Cloud Encounters of the Erlang Kind …


I have started building (or more precisely hacking my way into) a robust control plane for the cloud infrastructure and an convinced that Erlang is the right language:

  • A Cloud Control Plane is very similar to the control plane of networks – in fact in many ways it is probable that the hypervisor/VM is the next network – actually it is a layer slightly above the VM. What is a network control plane – it is distributed, it is self configuring (to some degree, with hints from network admins) and lots of processes connected by a set of very intelligent protocols. If one looks at a Cloud Control Plane that way, choice of Erlang is not that illogical
  • Very relevant architectural concept – Subsumption. Really like the way Wikipedia says about it ” …the lowest layers can work like fast-adapting mechanisms, while the higher layers work to achieve the overall goal ..” in this case lower layers work on protocol primitives (or “layers of augmented FSMs” http://ai.eecs.umich.edu/cogarch2/specific/subsumption.html) for reliability, connectivity, code/data mobility et al while the upper layers work on the goal of a stable cloud infrastructure !
  • R.A.Brook’s Intelligence without representation is a good read. What is the relevance here one might ask – a cloud is a “very large, coordinated, distributed infrastructure”, as Red Hat’s Jim Whitehurst would say[1] and we need ‘Robust Chaos rather then brittle determinism’ – robust chaos maintained by highly distributed massively scalable concurrent processing and that is an Erlang substrate supported by appropriate intelligent protocols (like Paxos, like P2P gossip, like XMPP, like …)
  • Concurrent processes – whether they are on a single machine across multiple processors (or even on a single virtualized processor) or across multiple machines – and transparency thereof is required, by the nature of the beast – a cloud will span machines and processors
  • And the paradigms of high availability (not across a few systems, but on a massive scale) again requires us to go beyond the Turing machine and Von Newman architectures. (Am sure this argument needs more logic – I will add as I conceptualize and internalize more)
  • High scalability – how high one might ask. In a recent press release Citrix talks about “Web 2.0 style tagging and searching capability which allows … to assign metadata and virtual tags to workloads, …  Once tagged, virtual machines are easily located using powerful searching and sorting capabilities based on application type, qos, department, cost center, location, origin, owner, performance requirements, or any other important attributes…”

References:

[1]  [http://resources.zdnet.co.uk/articles/features/0,1000002000,39454819,00.htm]

Notes:

  • Am keeping this post at the architectural level and am collecting the development level pointers in the companion blog Erlang Chronicles

[Sep 14, 2008: Pardon me for the anemic start.This is WIP and so will develop and add more details in the next few days, as I plug along …]

About these ads

2 thoughts on “Cloud Encounters of the Erlang Kind …

  1. Pingback: Erlang Chronicles …. « My missives

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s