I attended the “Cloud Computing and Beyond: The Web Grows up (finally)” conference hosted by SDForum in Santa Clara. Well done and was very informative. I was looking for the “beyond” part – pointers like “what are the barriers for wide adaption of cloud computing (especially in the enterprise)?” and “What are the opportunities?”. And got a lots of good ideas (as you will see in this – slightly long – blog!)
As a quick summary, I was impressed by the crispiness and clarity of what the users want from the current crop of cloud builders, especially Amazon. The shortcomings include latency variability, no stated SLA, proprietary interfaces and opacity. There are some requirements for seamless hybrid clouds, with spatial data and process flexibility between both sides of an enterprise firewall, ability to download control plane software (a.k.a the cloud OS) into on-perm servers (basically turn a set of arbitrary servers on this side of the firewall into a cloud, as needed!) as well as the declarative-policy based administrative control to achieve the “fluidity”. Projecting from this microcosm of users, I think, a few changes are in order for the cloud builders, like Amazon.
In short, the keynotes were inspiring, the vendors a little controversial, the users very passionate and somewhat frustrated and the vcs – well they acted like vcs – vague yet confident !
The first keynote, by James Staten of Forrester Research, set the tone for the day. He did a good job in framing the domain, even has a good definition!
The points I was able to grasp:
- Cloud Washing – a good term indicating the addition of cloud to an product in the market, especially from old web hosting companies.
- One implication of the cloud computing is the changes in business interactions including create markets faster (and the flip side fail faster), ability to try out newer concepts faster e al
- Business wants a place to experiment with fast integration, looser IT restrictions and faster responsiveness while It wants predictability, stability and early notice. So there is a level of impedance mismatch and cloud computing fill that void very well!
- I liked Forrester’s definition – “Cloud is a pool of scalable, abstracted infrastructure that hosts end-use applications, billed by consumption”
- Value proposition of clouds “Two persons, a laptop and a credit card = Web presence !”
- There are four major customers
- Start-ups/Individual accounts
- Gaming & Entertainment Companies
- This was news to me ! It seems maintaining a game infrastructure takes less resource but spikes when they introduce a new game. Excellent use case for clouds
- Looks like even Blu-ray movie releases have this challenge – Iron Man DVD release brought down Paramount’s servers completely (here and here) because the discs download content as soon as they are inserted into the player, which caused a stampede. Many folks returned the discs thinking they are defective ! ep an Amazon cloud could have helped here !
- Another use case is promotion sites for new movies (again very temporal!)
- Small businesses (web presence)
- Enterprises (very small percentage)
- Challenges to meet enterprise needs
- The current cloud offerings do not meet IT needs
- Not easy to control by IT
- Don’t support general as well as IT specific practices
Lew Tucker from Sun gave a keynote and had good points:
- Salesforce was able to turn an application into a platform
- Extending Carr’s book, Lew is of the opinion that distribution was the key for the adoption of electricity; a lesson the cloud community should consider
Another keynote was from Dr.Jayashree of IBM. Very insightful and interesting presentation. Some highlights:
- IBM’s cloud centers are interesting – they are actually listening posts ! Customers can come in, elaborate their business problems, prototype and then see if cloud fits their needs – a very neat way for IBM to engage and work with their customers, especially in a new domain !
- Use cases IBM is seeing from customers include innovation enablement, software development facilitation, virtual clustering and web 2.0 data intensive processing
I had to leave during the keynote by Russ (HP)
Good discussions on what is lacking in the current offerings – especially by amazon.
- Users still do not trust their data in the cloud. Some form of assurance – certification, best practices as well as audit might help. As I had mentioned in one of the e-mails, the two areas viz:
- Security and reliability – I think we need some form of certification based on (one or all of) best practices, audits, standards and specifications.
- Compliance – this is a legal issue as well. Again, need some form of industry audit/certification that makes a cloud infrastructure as “legally secure” as an internal infrastructure
- One surprise mentioned was how much work need to be done with Amazon’s offerings
- Another was the reliability issue – VMs go down and corrupt databases
- The flip side is that the limitations posed by Amazon’s offering has forced towards a better architecture (based on constraints like limited size VMs that can disappear) that scales well and keeps with organic growth. Otherwise designs would have migrated to bigger machines leading to not-so-scalable solution.
- Latency seems to bother a few folks, especially the variability. For many the VM movement seems arbitrary
- Looks like there are opportunities for optimization in VM distribution & migration that optimizes variability in latency – for example all related VMs nearby – based on interactions optimization. I think this is also related to affinity mentioned in a couple of papers.
- There was a discussion around the use of memcache and associated latency. I quite didn’t follow, but have a feeling this might be an area Amazon should look into
- Many, in the users panel, mentioned that Amazon is very opaque and that adds to the unpredictability nature of the latency and response. Of course, for a successful abstracted cloud, one should not know of the underlying implementation at all; as that will change. Moreover Amazon is constantly improving the control plane and the protocols and algorithmics running the substrate (I hope so)
- Users are at a stage now where they expect clear statement of SLAs and contractually stated variability. Amazon would be wise to heed this insight from the users, especially as many are about to negotiate on this point! Especially as the users have confidence in Amazon “Jeff’s team will be a lot better at running a cloud than we will ever be”
The vendor panel was very interesting. Again, good insights. Some notes form the vendor panel discussion:
- Cloud infrastructure should be viewed similar to the communication infrastructure (a view I also share)
- Cloud control plane software is in a different end of the spectrum
- One insight from the vendors “Build your app for the channel”
- Clouds provide an opportunity to innovate at the application layer
- Architecture portability rather than application portability across different offerings
- Couple of provocative comments came from the vendor panels (Jason Hoffman of Joyent)
- We need the ability to seamlessly transition between off-perm and on-perm clouds. For example one should be able to do dev and test using Amazon’s clouds and when the time comes for the production environment, one should be able to declaratively specify what components should reside where; and the same goes for data as well! Amazon then downloads it’s control plane into the on-perm servers as needed and manages this distribution seamlessly! Moreover one should be able to change the mix depending on the nature of the application, the compliance and regulatory climate one is in as well as other business policies.
- I think the “dev-stage-prod” cycle of traditional architecture is replaced by “dev-cloud tryout-cloud-stage-prod” cycle with more dynamism as to the on-perm-cloud/off-perm-cloud distribution of apps.
- There is also the mixing of namespaces, IP and VLAN mobility, load balancing across VMs which are in very different LANs and a host of other contextual complexities to be tackled in the cloud control plane – all of which we had made very strict assumptions in the traditional architecture! I repeat the axiom “robust chaos rather than brittle determinism” ! No more Newtonian machines !
Had some good discussions.
- They agreed that cost saving is not the essence of cloud computing (I also share this viewpoint. Clouds are about newer business models, business capabilities and so forth)
- There was a good discussion on how CDN was commoditized and whether clouds will follow the suit
- The see clouds as virtualizing the application infrastructure
- They are looking for companies that address long term pain points as well as have a short term promises
All in all, a good conference. I got what I was looking for. Couple of quick notes:
- The presentation from GoGird looked, out of place (sorry guys, I like you, but you need to be relevant at that context) – a very anemic sales pitch, no clear value proposition (their claim is that they run windows and are cheaper than AMZ – both very transitionary (AMZ will run Windows soon and pricing is relative anyway) and looked like a “cloud washing” by a hosting company looking for relevance as a cloud provider. I could be wrong(I hope I am!)
- The enterprise panel really didn’t address the “Challenges and Opportunities”
- Lots of discussion around disaster recovery as a value proposition for clouds. Don’t know how the discussions degenerated to this; I thought was a doozy ;o) Not that disaster recovery is a good segway (once folks get used to data outside their vision, they will be more comfortable with clouds in general) but disaster recover as a cloud feature is way far off, IMHO.
- I have a nagging feeling that VCs really do not get clouds – except as a “hot” area. While I have to assume that they know much more than I do and have a far better judgment, I didn’t hear any insightful comments. May be they are intentionally vague.
- Joyent’s Jason had some insightful comments;he also had a lot of controversial ones. It is interesting that the ones I agreed with him – I had complete agreement; and the rest I totally disagree; no shades of Grey !
- I have more pages to transcribe ;o) Will update this blog with a few more details …