My first attempt draft-sankar-oauth-headerhash-00a
April 3, 2009
March 14, 2009
March 2, 2009
Cloud Interoperability of the 5th Kind
In the last couple of days there have been a few good posts on Cloud Interoperability from Reuven, Greg and Stu. I also had written a blog about standard cloud couple of months ago. To understand the cloud interoperability, we do not have to go that far – from the clouds I mean – just taking a cue from the world of UFOs would be enough !
- Cloud Interoperability of the 1st Kind => Clouds exist and can be observed. This is the state we are in
- Cloud Interoperability of the 2nd Kind => Cloud services can talk to each other
- Cloud Interoperability of the 3rd Kind => Clouds can observe each other and some form of intelligent communication can happen
- Cloud Interoperability of the 4th Kind => Cloud are abducted by each other – just VM movement from one cloud to another
- Cloud Interoperability of the 5th Kind => This is where “joint, bilateral contact events produced through the conscious, voluntary and proactive … cooperative communication” happens. In essence, the clouds can work together
With this enlightenment, we can examine the POV of the cloud luminaries …
- Reuven says “Cloud Interoperability is to make it easier to use multiple cloud providers who share a common set of application interfaces as well as a consensus on the terminology / taxonomies that describe them” – Yes aliens and earthlings can talk the same language and understand each other, but we cannot change the fundamental nature of aliens and earthlings. So a common API layer does not work here and it should not.We do not want to convert aliens to earthlings or vice-versa. They both have their place in the universe and we should respect that. So Common API is a non-starter.
- Reuven defines “Cloud Portability” as mobility irrespective of OS/technology platform and format. Portability is Cloud interoperability of the 4th kind – mobility between homogeneous format and OS/technology platforms. We are not looking for antidote against platform lock-in, but are looking for cloud provider/vendor lock-in and btw, there is nothing like a cloud lock in. Unfortunately Reuven, in his blog, has a little confusion on this aspect. One has a choice of multiple OS platforms and one can always choose to write apps in one platform, if one wishes so. There are valid reasons for choosing one or more platforms. Windows folks will write in Azure, Google folks would like to work with Python and rest of the folks will use EC2. A cloud substrate should not mitigate the OS platform difference.
- Stu first talks about protocol interoperability only a network engineer can love, Yummy Stu ;o)
- His startling observation, that Microsoft has a clue, based on the .NET services architecture, itself is startling ! It is just interfaces between services. I do not think that is interoperability in any sense. OK, I will give it the Cloud Interoperability of the 2nd kind, if a cloud service on another OS/technology platform can talk to Azure services, and they can (I think)
- I liked Stu’s deployment flexibility and that is one of the key components for interoperable clouds – not translation, semantic or not, not common APIs but just deployment flexibility between cloud providers, on the same OS/technology platform.
- Stu has some interesting predictions – yep, no new run time (read my lips – those who remember that)
- As I have said earlier, we have to look at this from the four planes viz: policy, control, management & data. For Cloud Interoperability of the fifth kind, we need :
- A declarative policy plane that can be interpreted in a common way but implemented in proprietary ways as the platform sees fit
- A control plane that can be dynamic with rich attributes that are semantically equivalent across clouds (I do not care if they talk the same words so long as I know how to map them and do equivalence)
- A management plane – again built on semantically equivalent attributes so that I can make (collective and individual) inferences as and when needed
- And A data path which is the OS/technology of my choice and I do not want the cloud to change my data path characteristics.
- And A mature metering plane [Update - March 15,09 : James Urquhart convinced me that this is a fundamental requirement!]
Cheers
<k/>
February 22, 2009
Twitter Tips
I am collecting best practices, insights and just blogs with attitude on the usage of Twitter. I think this would help twitter newbies. So far I have :
- Twitter Etiquette: Five Dos and Don’ts
- Are you using Facebook, Linkedin and Twitter differently?
- Top 10 Twitter Tips for Beginners
Cheers
<k/>
February 14, 2009
A Berkeley View Of Cloud Computing : An Analysis – the good, the bad and the ugly
I read thru the technical report from UC Berkeley, Above the Clouds: A Berkeley View of Cloud Computing with interest. My analysis:
<summary>
- As an undergrad work on cloud computing, the paper gets an A+. But as a position paper from eminent academics, I can only give a C-. Granted it correctly identifies many of the trends and obstacles. But that material is widely available !
- With a title “A Berkeley view of cloud computing” the report misses the point. “A Berkeley observation…” is more like it – view requires original thinking and interpolation, which the report lacks.
</summary>
<the_good>
- The authors got some of the essentials of Cloud Computing right viz: infinite capacity, no up-front commitment and pay as you go.
- The three classes viz: amazon , Microsoft and the Google model is interesting. But there are more in-between.
- They have some good points on the cost advantage of power et al and leveraging that aspect by building datacenters at the appropriate locations.
- The new application models viz. analytics, parallel batch processing, compute-intensive desktop applications and so forth are excellent observations.
- They have done some good work in characterizing elasticity. Pages 10 and 11 are good read – the models are very simplistic, though.
- They also have done a good job in showing the economies of scale that can be achieved by a cloud computing infrastructure.
- I like their assertion that “there are no fundamental obstacle to make cloud-computing environments secure as well as compliant to data and processing rules. Declarative policy and enforcement thereof is my answer.
- They have correctly identified scalable storage as one of the bottlenecks. The BigTable(Google), Dymo(AMZ) and Cassandra(facebook) all are solutions for the challenge.
</the_good>
<the_bad>
- But, they got the model wrong ! The essentials of Utility Computing is the consumption model not the payment model. No doubt the pay-as-you-go model is attractive to startups, but the payment model is the second order effect. For enterprises and other organizations, the value proposition is the elasticity and the just-in-time availability of resources. Even for startups the pay as you go is attractive but elasticity is much more important.
- Argument about increase in performance and resultant cost reduction. This just Moore’s law and it is achievable within IT environments as well as a cloud computing space. I think computers are on a 5 year amortization schedule and depreciation. And a refresh can be done – with associated efficiency whether they are a cloud provider or an IT shop.
- I think the major disconnect in the paper is the basic definition of a cloud as public. The artificial separation of public/private clouds and the focus on payment were the two areas where their definition has gone awry. Cloud is an architectural artifact and a business model of computing. But clouds are clouds – internal or external, public or private. The internal vs. external is only a spatial artifact – which side of the firewall. Not worth a demarcation when we talk about the domain of cloud computing. Which side of the internet (firewall) does the cloud infrastructure lie, should not be the criteria. By their definition, they have disenfranchised the whole set of clouds inside organizations. The internal-external cloud integration across data, management, policy and compute planes is an important topic which this model conveniently skips. Also as I mentioned earlier, utility is the consumption not a payment model. A big organization can have a cloud computing infrastructure and it’s business units can leverage the elasticity – no need for a credit card, a charge back model will do.
</the_bad>
<the_ugly>
- I really didn’t get the “statistical multiplexing” they mention a few times. What exactly is this and what is the relevance ? Just a buzz word to jazz up the paper ?
- I literally got lost in their characterization of DDoS attack and the cost models there of on P.15. Really convoluted and it does not change for traditional vs. cloud. They found a break-even point for DDoS attack based on very slim assumptions.
- I do not think the data transfer bottleneck, as described in the paper (P.16), is an issue. Who is going to transfer 10TB of data routinely for cloud processing ? Looks like a force fit for some calculations done by someone.
- The report has no useful tables or equations. Equations 1 and 2 (which are the same, btw) are not right – in thesense that the datacenter cost includes the utilization and I do not think we need to accommodate for it additionally.
- I am sorry to say all the cost models and the equations look forced and very unnatural. Even the assertion of 1/5 to 1/7 cost advantage of a datacenter is at best questionable.No value what so ever – sorry folks
</the_ugly>
Updates:
February 10, 2009
Dynamic scalability for cloud computing
An interesting article on scalability. Three points caught my attention:
a) Scalability is fundamental aspect of a design
b) Architect for near-zero-cohesion among your components
c) Dynamic scalability – defined as the ability to increase capacity by adding instances (which would be possible only by zero-cohesion)
I thought there is a little mixup in the article with multi-threading, semaphores et al. IMHO, scaling witin a machine (by leveraging more cores) is different from scaling in a cloud – with more processes. We need both, but depends on the nature of processing. For example Zillow (which runs solely on EC2) would be more suited for cloud scaling while computationally intensive applications could use some multi-core scaling plus cloud scaling.
January 27, 2009
Cloud Standards – Putting the cart before the horse ?
I have been following some of the chatter in the “Cloud Standards” space. Looks to me, we have more talk about standardization work than actual cloud work. May be I am moving in the wrong circles ;o)
Anyway the reason for the blog is Bob Sutor’s notes from a Cloud Standards gathering . To be fair, I did not attend the meeting and I will be careful to separate the message from the messenger – Bob’s ideas vs. what he is reporting from the gathering.
A few things that caught my eye – some I agree, a lot I disagree:
- There could be potentially 100s of standards
- To quote Scooby-doo, “Yikes”. I think this is the wrong approach. The domain (Cloud Computing) is barely born and we are already talking about 100s of standards- new or old ! This will definitely slow down the progress or (most probably) everyone will ignore the “standards” and do whatever is necessary to run the business. We should strive for simplification. I propose that overstandardization was one of the reasons for the demise of SOA and Web Services. (Yep, I know lots of people will disagree. But before you start throwing stones, at least read thru Ann’s blog – http://apsblog.burtongroup.com/2009/01/soa-is-dead-long-live-services.html)
- Need “marketecture” & speed of standardization is important
- This is exactly where I think we have the cart before the horse. Usually standardization happens based on a few running instances and when practitioners realize that some parts need interoperability based on experience – let me say again, actual experience. Because of running code, working protocols, data fields et al, the participants know what to standardize and how. Without the extensive implementation experience to guide us, how are we going to standardize based on an imaginary marketecture ? A marketecture is just that and it should never be confused with actual hands-on experience.
- And we should never rush standards – usually standards are a little deliberate and systemic. Agreed, once we understand a domain and come across opportunities, we shouldn’t take infinite time. But I have never heard of standardization based on “marketecture” even before a domain is being developed ! (Until now, of course)
- Need to understand interchangeable parts vs. interoperability
- I think we are over analyzing here. To tell the truth, I have no clue what it means.
- “It is not acceptable to delay standardization until a particular provider establishes lock-in or a monopoly”
- Eh ? ;o) Very strange. Monopoly or duopoly or … comes via business relevance – not by standardization. Here also I am lost – are we saying that all the hoopla about standardization is against a known or unknown cloud provider ?
- “Cloud computing is here today, but we are very, very early”
- Agreed. Of course, this clashes with other statements where it says we already have a monopoly
- “We should not waste time having an official cloud computing definition of ‘interoperable.’”
- Agreed. Instead, may be we should define “interchangeable” and move on o;) (Sorry, couldn’t resist) Again, this is right – the domain is very new that we really do not know what it is. Me thinks, we should not smother and suffocate it with too much standardization. What says thee ?
January 20, 2009
Book Review – Programming Erlang, Software for a Concurrent World By Joe Armstrong
Note: This is the blog version of my review slated to be published in an upcoming issue of the Journal Of Functional Programming Volume 19, Issue 2, March 2009
Context
A funny thing happened on the microprocessors’ race to faster and faster clock speeds; somewhere along the way, the speeds and feeds hit a brick wall and the only way out is lateral – more cores rather than faster CPUs. It started with hyper threading and progressed to dual cores and to multi-core CPUs. While this paradigm shift solved the hardware challenges, a new bottleneck rose – in the software! Multi-threaded programming, which was mainly for leveraging I/O waits and SMPs couldn’t scale to the moderately massive parallelism – while the domain complexity of concurrent programming itself is interesting, the additional accidental complexity of threads and mutexes’ and semaphores in traditional programming just doesn’t cut it.
The answer it seems is “Pluralitas non est ponenda sine neccesitate” i.e. the singularity of immutability and stateless-ness of functional languages rather than the pluralities (of mutexes, semaphes, spin locks and code locking) – the Ockham’s Razor!
As a result, more and more folks are looking towards functional programming as the silver bullet for solving the software’s version of the Moore’s Law! In fact a timely article in Dr.Dobbs journal [1] wonders if “functional programming is on the verge of becoming a must-have skill”! And Erlang is at the forefront of this revolution – an article in Queue[2] is of the opinion that “… designed for concurrency from the ground up, the Erlang language can be a valuable tool to help solve concurrent problems.”.
In short, Joe Armstrong’s book “Programming Erlang-Software for a concurrent world” is at the right place and at the right time! As the title implies, the book’s focus is more on concurrency than functional programming but essential functional programming is the backdrop for the required functionalities.
Overview
The book is pretty thick – 20 chapters and 5 appendices, coming out around 500 pages. The subject is deep but the style makes it interesting and easy to comprehend. I felt that the book ended very fast ! In my opinion, this is not a traditional programming book – I consider this a systems book. It is difficult to separate Erlang the language from Erlang the system. The book follows the architecture of the Erlang system – makes sense, especially as the author is one of the originators of the language
Gory details
The book has a good logical progression and is logically divided into four parts:
- Part 1 – The important concepts of the language with enough syntax ending with compiling and running Erlang Programs
- Part 2 – Dives into bigger topics like concurrency and distributed programming plus files and sockets
- Part 3 – Is where the Erlang system concepts are detailed including OTP and databases.
- The book ends with Part 4, which talks about multi-core programming.
On the whole, the book is laid out well. I felt that this is not a book for the casual reader – the depth of the subject as well as the breadth of understanding required to “grok” Erlang is slightly more than other languages. Erlang is slightly difficult to get around for oo programmers. I have Cobol and Pascal background so was easier, still was corrupted by years of oo programming ;o)
One should read the book fully to get the benefit – some of the concepts become very clear at the later chapters – an artifact of the language-as-a-system characteristic of Erlang. There is another Erlang book coming from O’Reilly (http://www.amazon.com/gp/product/0596518188/) and am looking forward to see how the authors are handling this in that book. May be I will write a review.
Half way thru the book, I found it helpful to read thru the the Erlang language reference – just to get a feel for the full syntax. The Erlang language is very small language that is the beauty of the language. Another must-read is the History Of Erlang paper [6] as well as the presentations [7]. The history of Erlang is fascinating and a good read before or while reading the book.
A few observations from my reading of the book:
- Joe explained the Bit syntax very well. It could have helped me for a time sync protocol implementation (IEEE1588v2) with 48byte manipulations
- The database is called mnesia – interesting that it was called Amnesia – strange name for a database !
- No surprise – the chapters on concurrency are the strongest
- One area it doesn’t cover is the cloud computing which Erlang very strong – for example AmazonDB is supposed to be written in Erlang and the new cloud infrastructure project Verteba is in Erlang.
- The chapter on ets/dts keyvalue store is timely as that is very common in internet applications
- Finally want to mention two reviews of this book in the O’Reilly site [4].
Conclusion
Overall, an exceptionally good book on a very relevant language/system. I would suggest pre-reading the couple of papers – the Erlang syntax as well as history of Erlang before reading this book. Then lots of things would make sense as well as the reader would get a clear understanding.
Looking forward to Erlang Programming by Francesco Cesarini and Simon Thompson[5] and how they treat the introductions. May be I will follow-up with a review !
One interesting artifact of the immutability of data in Erlang is that handling traditional data structures like Linked lists. A companion book that would be of interest to the readers of this book is “Purely functional data structures”. In addition to Erlang, the other FP languages under consideration include Scala, F# and Clojure. In fact I am looking forward to reviewing the book “Programming Scala: Tackle Multi-Core Complexity on the Java Virtual Machine” which follows the concurrency paradigm covered in the Erlang book. Scala is a mixture of OO and functional programming and has many of the Erlang patterns like gen_server. Clojure might be of interest to readers because of it’s LISP origins.
There are concerns that functional programming – pure or part of an oo model – itself might not solve the massive concurrency problem; more than 16 cores might hit the memory wall[3]! Let us not worry about it now; most probably it is a topic for yet another article !
It is apt to quote the epilogue which says it all – “… large monolithic programs are becoming dinosaurs … the future has multiple cores and the future has distributed processors …”
Reference:
- [1]Dr.Dobb’s ournal “It’s Time to Get Good at Functional Programming – Is it Finally Functional Programming’s Turn?” http://www.ddj.com/development-tools/212201710;jsessionid=3MQLTTYJRPL3CQSNDLRSKH0CJUNN2JVN
[2] http://acmqueue.com/modules.php?name=Content&pa=showpage&pid=556
[3] http://arstechnica.com/news.ars/post/20081207-analysis-more-than-16-cores-may-well-be-pointless.html
[4] http://oreilly.com/catalog/9781934356005/#rr
[5] http://www.amazon.com/gp/product/0596518188/
[6] http://www.cs.chalmers.se/Cs/Grundutb/Kurser/ppxt/HT2007/general/languages/armstrong-erlang_history.pdf
[7] http://delivery.acm.org/10.1145/1240000/1238850/supp/Erlang.pdf?key1=1238850&key2=9573586811&coll=ACM&dl=ACM&CFID=15151515&CFTOKEN=6184618
October 27, 2008
Think Way Outside the box – Microsoft’s Azure into cloud computing
Finally Microsoft has entered into the cloud business formally with Windows Azure. These are my notes from MS PDC 2008.
Executive summary:
Windows Azure is not an OS but an offering. It consists of :
- A hosting environment to deploy your services (you define the rules & provide your code, the platform will take care of the rest!) for a spectrum of users – from hobbyists to enterprise developers
- Automated service management (abstracts hardware, load balancing and a host of other similar functions, based on the service model you create, which has things like service topology, size, health constraints and so on)
- Scalable storage
- A rich developer experience(This is where Microsoft has leverage- the Azure fits seamlessly into their development environment- you can write usual code, test it in their cloud simulation environment, debug the code and then deploy it to the cloud. So the current development skills are fully transferable! The deployment is so easy “even a CEO or a VP can do it!”)
- Windows Azure as a services layer with .NET services (service bus,access control and work flow services)
- They also offer SQL Services, SharePoint services and CRM on this platform
Their perspective of a cloud is very simple – “A set of connected servers;on which install and run services;and store and retrieve data” – and their offerings reflect that view of the world
Their view of the business requirements on a cloud infrastructure are:
- Interoperability and Business Processes
- Identity and Security
- Data management & Compliance
- Services Management
Some quick thoughts
- In terms of impact, Bob Muglia compares this year to PDC ‘92, when Microsoft announced Widows NT
- Microsoft characterizes this as 5th generation of computing
- Monolithic(70s), Client Server(80s),web(90s),SOA(now)-services(2009+)
- They are not embracing the term Cloud, but are calling it services exposed via web protocols!
- Their motto is software + services -> The power of choice
- They see Azure as helping to evolve existing paradigms to work with hybrid architectures
- Also suddenly there are fabric everywhere – Azure service management fabric, development fabric,… <KS>Neiman Marcus anyone ?</KS>
- All features are not currently available, they are “exposing functionality in a staged manner”
- Introducing the cloud, they have a little arrogance and one would come away with the feeling that Microsoft had been at it for years ! Ray did tip his hat to Amazon and Jeff, though.
- From a business perspective, Microsoft s formally in the infrastructure business
Gory Details:
Day 1
Keynotes
Ray Ozzie introduced the concepts and Azure, followed by Amitabh Srivastava, Bob Muglia and David Thompson detailing different parts of the offering.
- Current applications have enterprise as the scope and with cloud the scope has expanded – Cloud is the externalization of IT
- The separate roles of software developer and operations are intertwined with the cloud computing paradigm
- While some companies have the resources required for the operation discipline to run a global infrastructure,many find it a disproportionate burden
- Some challenges to be solved by cloud computing
- Meeting customer expectation of interactive, participatory web systems
- Operating across peaks and valleys
- Continuity Issues
- Loosely coupled architecture, data replication strategies, data partition strategies
- Ray calls clouds “overdraft protection for your web site”
- This “high scale internet infrastructure” is a new tier
- Desktop Tier – scope of a single machine
- Enterprise Systems – scope of the enterprise
- Web Tier – scope of the web
- Windows Azure is not an OS but Kernel of cloud platform. “Kernels do not demo very well”, so they showed demo of a few apps
- Business View of Azure
- Scalable infrastructure
- Ability to manage a large global datacenter infrastructure
- Federated DataCenter (This is one of their key themes)
- Automated Service Management
- Applications and OS managed separately
- Fabric Controller manages lifecycle
- The service model is one of the key concepts
- That is how one defines a service declaratively. The model includes roles.channels, adaptors, interfaces, configuration setting et al
- The service model is an XML file
- The services bus is another important piece because it securely connects the on perm and cloud seamlessly even through firewalls
- The have a good Identity substrate with connectors and gateways to transcend between on perm and cloud. <KS>This I thought was a good value proposition; seamless identity across enterprise and cloud is difficult. I also saw that they now support openID.Good move</KS>
Odds & Ends
- An interesting URL!
- Azure white paper
- Yep, I know – Azure is not a verb. May be it should ;o)