Cloud Standards – Putting the cart before the horse ?

I have been following some of the chatter in the “Cloud Standards” space. Looks to me, we have more talk about standardization work than actual cloud work. May be I am moving in the wrong circles ;o)

Anyway the reason for the blog is Bob Sutor’s notes from a Cloud Standards gathering . To be fair, I did not attend the meeting and I will be careful to separate the message from the messenger – Bob’s ideas vs. what he is reporting from the gathering.

A few things that caught my eye – some I agree, a lot I disagree:

• There could be potentially 100s of standards

• To quote Scooby-doo, “Yikes”. I think this is the wrong approach. The domain (Cloud Computing) is barely born and we are already talking about 100s of standards- new or old ! This will definitely slow down the progress or (most probably) everyone will ignore the “standards” and do whatever is necessary to run the business. We should strive for simplification. I propose that overstandardization was one of the reasons for the demise of SOA and Web Services. (Yep, I know lots of people will disagree. But before you start throwing stones, at least read thru Ann’s blog  – http://apsblog.burtongroup.com/2009/01/soa-is-dead-long-live-services.html)

• Need “marketecture” & speed of standardization is important

• This is exactly where I think we have the cart before the horse. Usually standardization happens based on a few running instances and when practitioners realize that some parts need interoperability based on experience – let me say again, actual experience. Because of running code, working protocols, data fields et al, the participants know what to standardize and how. Without the extensive implementation experience to guide us, how are we going to standardize based on an imaginary marketecture ? A marketecture is just that and it should never be confused with actual hands-on experience.

• And we should never rush standards – usually standards are a little deliberate and systemic. Agreed, once we understand a domain and come across opportunities, we shouldn’t take infinite time.  But I have never heard of standardization based on “marketecture” even before a domain is being developed ! (Until now, of course)

• Need to understand interchangeable parts vs. interoperability

• I think we are over analyzing here. To tell the truth, I have no clue what it means.

• “It is not acceptable to delay standardization until a particular provider establishes lock-in or a monopoly”

• Eh ? ;o) Very strange. Monopoly or duopoly or … comes via business relevance – not by standardization. Here also I am lost – are we saying that all the hoopla about standardization is against a known or unknown cloud provider ?

• “Cloud computing is here today, but we are very, very early”

• Agreed. Of course, this clashes with other statements where it says we already have a monopoly

• “We should not waste time having an official cloud computing definition of ‘interoperable.’”

• Agreed. Instead, may be we should define “interchangeable” and move on o;) (Sorry, couldn’t resist) Again, this is right – the domain is very new that we really do not know what it is. Me thinks, we should not smother and suffocate it with too much standardization. What says thee ?

Book Review – Programming Erlang, Software for a Concurrent World By Joe Armstrong

Note: This is the blog version of my review slated to be published in an upcoming issue of the Journal Of Functional Programming Volume 19, Issue 2, March 2009

Context

A funny thing happened on the microprocessors’ race to faster and faster clock speeds; somewhere along the way, the speeds and feeds hit a brick wall and the only way out is lateral – more cores rather than faster CPUs. It started with hyper threading and progressed to dual cores and to multi-core CPUs.  While this paradigm shift solved the hardware challenges, a new bottleneck rose – in the software! Multi-threaded programming, which was mainly for leveraging I/O waits and SMPs couldn’t scale to the moderately massive parallelism – while the domain complexity of concurrent programming itself is interesting, the additional accidental complexity of threads and mutexes’ and semaphores in traditional programming just doesn’t cut it.

The answer it seems is “Pluralitas non est ponenda sine neccesitate” i.e. the singularity of immutability and stateless-ness of functional languages rather than the pluralities (of mutexes, semaphes, spin locks and code locking)  – the Ockham’s Razor!

As a result, more and more folks are looking towards functional programming as the silver bullet for solving the software’s version of the Moore’s Law! In fact a timely article in Dr.Dobbs  journal [1] wonders  if “functional programming is on the verge of becoming a must-have skill”! And Erlang is at the forefront of this revolution – an article in Queue[2] is of the opinion that “… designed for concurrency from the ground up, the Erlang language can be a valuable tool to help solve concurrent problems.”.

In short, Joe Armstrong’s book “Programming Erlang-Software for a concurrent world” is at the right place and at the right time! As the title implies, the book’s focus is more on concurrency than functional programming but essential functional programming is the backdrop for the required functionalities.

Overview

The book is pretty thick – 20 chapters and 5 appendices, coming out around 500 pages. The subject is deep but the style makes it interesting and easy to comprehend. I felt that the book ended very fast ! In my opinion, this is not a traditional programming book – I consider this a systems book. It is difficult to separate Erlang the language from Erlang the system. The book follows the architecture of the Erlang system  – makes sense, especially as the author is one of the originators of the language

Gory details

The book has a good logical progression and is logically divided into four parts:

• Part 1 – The important concepts of the language with enough syntax ending with compiling and running Erlang Programs
• Part 2 – Dives into bigger topics like concurrency and distributed programming plus files and sockets
• Part 3 – Is where the Erlang system concepts are detailed including OTP and databases.
• The book ends with Part 4, which talks about multi-core programming.

On the whole, the book is laid out well. I felt that this is not a book for the casual reader – the depth of the subject as well as the breadth of understanding required to “grok” Erlang is slightly more than other languages. Erlang is slightly difficult to get around for oo programmers. I have Cobol and Pascal background so was easier, still was corrupted by years of oo programming ;o)

One should read the book fully to get the benefit – some of the concepts become very clear at the later chapters – an artifact of the language-as-a-system characteristic of Erlang. There is another Erlang book coming from  O’Reilly (http://www.amazon.com/gp/product/0596518188/) and am looking forward to see how the authors are handling this in that book. May be I will write a review.

Half way thru the book, I found it helpful to read thru the the Erlang language reference – just to get a feel for the full syntax. The Erlang language is very small language that is the beauty of the language. Another must-read is the History Of Erlang paper [6] as well as the presentations [7]. The history of Erlang is fascinating and a good read before or while reading the book.

A few observations from my reading of the book:

1. Joe explained the Bit syntax very well. It could have helped me for a time sync protocol implementation (IEEE1588v2) with 48byte manipulations
2. The database is called mnesia – interesting that it was called Amnesia – strange name for a database !
3. No surprise – the chapters on concurrency are the strongest
4. One area it doesn’t cover is the cloud computing which Erlang very strong – for example AmazonDB is supposed to be written in Erlang and the new cloud infrastructure project Verteba is in Erlang.
5. The chapter on ets/dts keyvalue store is timely as that is very common in internet applications
6. Finally want to mention two reviews of this book in the O’Reilly site [4].

Conclusion

Overall, an exceptionally good book on a very relevant language/system. I would suggest pre-reading the  couple of papers – the Erlang syntax as well as history of Erlang before reading this book. Then lots of things would make sense as well as the reader would get a clear understanding.
Looking forward to Erlang Programming by Francesco Cesarini and Simon Thompson[5] and how they treat the introductions. May be I will follow-up with a review !

One interesting artifact of the immutability of data in Erlang is that handling traditional data structures like Linked lists. A companion book that would be of interest to the readers of this book is “Purely functional data structures”. In addition to Erlang, the other FP languages under consideration include Scala, F# and Clojure. In fact I am looking forward to reviewing the book “Programming Scala: Tackle Multi-Core Complexity on the Java Virtual Machine” which follows the concurrency paradigm covered in the Erlang book.   Scala is a mixture of OO and functional programming and has many of the Erlang patterns like gen_server. Clojure might be of interest to readers because of it’s LISP origins.

There are concerns that functional programming – pure or part of an oo model – itself might not solve the massive concurrency problem; more than 16 cores might hit the memory wall[3]! Let us not worry about it now; most probably it is a topic for yet another article !

It is apt to quote the epilogue which says it all – “… large monolithic programs are becoming dinosaurs … the future has multiple cores and the future has distributed processors …”

Reference:

• [1]Dr.Dobb’s ournal  “It’s Time to Get Good at Functional Programming – Is it Finally Functional Programming’s Turn?”  http://www.ddj.com/development-tools/212201710;jsessionid=3MQLTTYJRPL3CQSNDLRSKH0CJUNN2JVN
  [2] http://acmqueue.com/modules.php?name=Content&pa=showpage&pid=556
  [3] http://arstechnica.com/news.ars/post/20081207-analysis-more-than-16-cores-may-well-be-pointless.html
  [4] http://oreilly.com/catalog/9781934356005/#rr
  [5] http://www.amazon.com/gp/product/0596518188/
  [6]  http://www.cs.chalmers.se/Cs/Grundutb/Kurser/ppxt/HT2007/general/languages/armstrong-erlang_history.pdf
  [7] http://delivery.acm.org/10.1145/1240000/1238850/supp/Erlang.pdf?key1=1238850&key2=9573586811&coll=ACM&dl=ACM&CFID=15151515&CFTOKEN=6184618