Yep, I know, it is 10 foot pole; and the origin is from “10-foot poles that river boatmen used to pole their boats with”[1]
Back to the main feature, I was reading an piece by Andrew J Burst at GigaOM that “Hadoop needs a better front-end for business users”
Yikes. This is terrible … I would argue, no, make that insist, that business users be kept as far away as possible from Hadoop (& similar frameworks)
Allow me to elaborate …
- Business users do need highly interactive analytic dashboards with knobs & dials into our deep machine learning models and sliders onto our AI machines, No doubt.
-
We don’t want to abandon our beloved business users with static-rigid-newtonian-deterministic artifacts; we want them to have living, (fire) breathing intelligent-inferential-predictive-models
- But that control & interactivity is into a business analytics beast that has multiple layers, not directly onto a Hadoop or hadoop-like system.
Also separating the “what” form the “how” by a declarative interface is very important
You see, analytics has at least four layers viz. Infrastructure, Intelligence, Inference & Interface
- Hadoop is Infrastructure, Spark is Infrastructure – The “How”
- Machine Learning algorithms are Intelligence – Again lots of “How”
- Models are Inference – the “What” Plus some “How”
- Dashboard is the Interface (usually) – definitely the “How”
-
Interface can be recommendations, financial predictions, ad forecasts or even actual devices that interface to predictive models
- And business needs knobs & dials at the Inference & Interface layers
- The Infrastructure then appropriately fires frameworks Hadoop or Spark or Java or iPython …
-
Digging deeper, Hadoop itself has three layers – none of them operable by a business user, but real work horses
-
HDFS – the distributed File System
-
MapReduce – the distributed data parallel computation engine
-
HBase – the NOSQL data store
-
Back to Andrew’s points, Hadoop (and it’s cousins) should remain as a tool for the Chefs; but diners do need to express their choices and have the ability to “tweak” the seasonings, portions or even the amount of cooking; a declarative interface (which tells what but not how) comes from the domain specific menus catered by the restaurants which focus on respective culinary styles or even a fusion !
Now I am getting Hungry ! On my way to downstairs (am at the Hilton – NY Fashion District) to my favorite Chipotle – who in fact gives me the declarative freedom, without getting into their kitchen and the need to handle the saucepans ;o) It is better that way because I am terrible with cooking and spice measures – I can tell less salt but not the amount !
[1] http://en.wiktionary.org/wiki/not_touch_something_with_a_ten_foot_pole
[2] Interface from http://img1.mxstatic.com/wallpapers/1bb91493c637d7c5ed6e1cefbef87ec1_large.jpeg
My missives 