What would you want AI to do, if it could do whatever you wanted it to do ?


P.S: Copy of my blog in Linkedin

Note : I am capturing interesting updates at the end of the blog.

AI-06-01

Exponential Advances:

An interesting article in Nature points out that exponential advanced in technological growth can result is a very alternate world very soon.

IBM X Prize:

 

AI-07

And the IBM AI X Prize is offering a chance to showcase powerful ideas that tackle challenges.

Got me thinking … What do would we want our machines/AI to do ?

I am interested in your thoughts. Pl comment on what you would like AI to do.

Earlier I had written about us not wanting our machines to be like us; understand us – may be, help us – definitely, but imitate us – absolutely not  …

So what does that mean ?

  • Driving cars ? – Definitely
  • Image recognition, translation and similar tasks ? – Absolutely
  • Write like Shakespeare just by feeding all the plays to a neural network like the LSTM ? – Definitely not !

I see folks training deep Learning systems by feeding them Shakespeare plays and see what the AI can write. Good exercise, but is that something we would get an X Prize for ? Of course, that is putting the cart before the horse !

We don’t write just by memorizing the dictionary and Elements of Style !!

  • We write because we have a story to tell.
  • The story comes before writing;
  • Experience & imagination comes before a story …
  • A good story requires both the narrative power as well as a powerful content with it’s own anti-climax, and of course the hanging chads ;o)
  • Which the current AI systems do not possess …
  • Already we have robots (Google Atlas) that can walk like a human – leaving aside the the goofy gait – which, of course, is mainly a mechanical/balance problem than an AI challenge
  • Robots can drive way better than a human
  • They translate a lot better than humans can (Of course language semantics is a lot more mechanical than storytelling)
  • Robots and AI machines do all kinds of stuff (Even though Mercedes Assembly plant found that they cannot handle the versatile customization!)

Is there anything remaining for an AI prize One wonders …

In the article “How Google’s impressive new robot demo will fuel your nightmares” , at 2:09, the human (very rudely) pushes the robot to the ground and the robot gets up on it’s own ! That proves that we have solved the mechanical aspects of human anatomy w.r.t movements & balance.

[Update 3/17/16] Looks like Google is pushing Boston Dynamics out of the fold !

AI-08

 

But a meta question remains.

  • Would the robot be upset at the human ?
  • Would it know the difference – if it was pushed to keep it away from harm’s way (say a falling object) vs. out of spite ?
  • And, if we later hug the robot (as the author suggests we do) would it feel better ?
  • Will it forget the insult ?

So there is something to be done after all !

Impart into our AI – the capability to imagine, the ability to understand what life is;  feel sadness & joy; understand what it is to struggle through a loss,…

This is important – for example, if we want robots to act as companions for the sick, the elderly and the disabled, may be even the occasional lonely, the desolate and for that matter even the joyous!

If the AI cannot comprehend sadness, how can it offer condolences as a companion ? Wouldn’t understanding our state of mind help it to help us better? 

AI-09

 In many ways, by helping AI to understand us, the ultimate utility might not be whether AI really comprehends us or not, but whether we get to understand us better, in the process !! And that might be the best outcome out of all of these innovations.

As H2O-ai Chief SriSatish points out,

Over the past 100 years, we’ve been training humans to be as punctual and predictable as machines; … we’re so used to being machines at work—AI frees us up to be humans again ! – Well said SriSatish

With these points in mind, it is interesting to speculate what the AI X-Prize TED talks would look like in 2017; in 2018. And what better way to predict the future than to invent it ? I am planning on working on one or two submissions …

And what says thee ?

[Update 3/12/16] Very interesting article in GoGaneGuru about AlphaGo’s 3rd win.

AI-11

  • AlphaGo’s strength was simply remarkable and it was hard not to feel Lee’s pain
  • Having answered many questions about AlphaGo’s strengths and weakness, and exhausting every reasonable possibility of reversing the game, Lee was tired and defeated. He resigned after 176 moves.
  • It’s time for broader discussion about how human society will adapt to this emerging technology !!

And Jason Millar @guardian agrees.

AI-12

Maybe all is not lost after all, WSJ says … !

AI-13.png

[Update 3/9/16] Rolling Stone has a 2-part report – Inside the Artificial Intelligence Revolution. They end the report with a very ominous statement.

AI-10

 

[Update 3/4/16] Baidu Chief Scientist Andrew Ng has insightful observations

  • “What I see today is that computer-driven cars are a fundamentally different thing than human-driven cars and we should not treat them the same”- so true !

[Update 3/6/16] An interesting post from Tom Devenport about Cognitive Computing.

  • Good insights into what Cognitive Computing is, as a combination of Intelligence(Algorithms), Inference(Knowledge) and Interface (Visualization, Recommendation, Prediction,…)
  • IMHO, Cognitive Computing is more than Analytics over unstructured data, it also has touches of AI in there.
  • Reason being, Cognitive Computing understands humans – whether it is about buying patterns or the way different bodies reacts to drugs or the various forms of diseases or even the way humans work and interact
  • And that knowledge is the difference between Analytics and Cognitive Computing !

I like Cognitive Computing as an important part of AI, probably that is where most of the applications are … again understanding humans rather than being humans !

Reference & Thanks:

My thanks to the following links from which I created the collage:

  1. http://www.nature.com/news/a-world-where-everyone-has-a-robot-why-2040-could-blow-your-mind-1.19431
  2. http://lifehacker.com/the-three-most-common-types-of-dumb-mistakes-we-all-mak-1760826426
  3. https://e27.co/know-artificial-intelligence-business-20160223/
  4. http://spectrum.ieee.org/tech-talk/computing/software/digital-baby-project-aims-for-computers-to-see-like-humans
  5. http://www.techrepublic.com/article/10-artificial-intelligence-researchers-to-follow-on-twitter/
  6. http://www.lifehack.org/309644/book-lovers-alert-8-the-most-spectacular-libraries-the-world
  7. http://www.headlines-news.com/2016/02/18/890838/can-ai-fix-the-world-ibm-ted-and-x-prize-will-give-you-5-million-to-prove-it
  8. http://www.lifehack.org/366158/10-truly-amazing-places-you-should-visit-india?ref=tp&n=1
  9. http://www.lifehack.org/articles/lifestyle/20-most-magnificent-places-read-books.html

Augmented Cognitive Intelligence


Have been working on this architecture for a couple of years. The idea is to build an AI machine that augments the human capabilities. I know IBM has Watson; Google, FB all have their own versions that address different domains.

The diagram below is more for my understanding and to clarify the thinking. I will write more as I get time. Hope you all find it useful.

AI-01

Google’s Jeff Dean on Scalable Predictive DeepLearning – A Kitbizer’s notes from Recsys 2014 (Note :


It is always interesting to hear from Jeff and understand what he is upto. I have blogged about his earlier talks at XLDB and at Stanford. Jeff Dean’s Keynote at RecSys2014 was no exception. The talk was interesting, the Q&A was stimulating and the links to papers … now we have more work ! – I have a reading list at the end.

Of course, you should watch it (YouTube Link) and go thru his keynote slides at the ACM Conference on Information and Knowledge Managment. Highlights of his talk, from my notes …

dean-recsys-01

  • Build a system with simple algorithms and then throw lots of data – let the system build the abstractions. Interesting line of thought;
  • I remember hearing about it from Peter Norwig as well ie Google is interested in algorithms that get better with data
  • An effective recommendation system requires context ie. understand the user’s surroundings, previous behavior of the user, previous aggregated behavior of many other users and finally textual understanding.

dean-recsys-02-01


  • He then elaborated one of the area they are working on — semantic embeddings, paragraph vector and similar mechanisms

dean-recsys-03

Interesting concept of embedding similar things such that they are nearby in a high dimensional space!

  • Jeff then talked about using LSTM (Long Short-Term Memory) Neural Networks for translation.

pic09-01

  • Notes from Q & A:
    • The async training of the model and random initialization means that different runs will result in different models; but results are within epsilon
    • Currently, they are handcrafting the topology of these networks ie now many layers, how many nodes, the connections et al. Evolving the architecture (for example adding a neuron when an interesting feature is discovered) is still a research topic.
      • Between ages of 2 & 4, our brain creates 500K neurons / sec and from 5 to 15, starts pruning them !
    • The models are opaque and do not have explainability. One way Google is approaching this is by building tools that introspect the models … interesting
    • These models work well for classification as well as ranking. (Note : I should try this – may be for a Kaggle competition. 2015 RecSys Challenge !)
    • Training CTR system on a nightly basis ?
    • Connections & Scale of the models
      • Vision : Billions of connections
      • Language embeddings : 1000s of millions of connections
      • If one has more data, one should have less parameters;otherwise it will overfit
      • Rule of thumb : For sparse representations, one parameter per record
    • Paragraph vector can capture granular levels while a deep lSTM might be better in capturing the details – TBD
    • Debugging is still an art. Check the modelling; factor into smaller problems; see if different data is required
    • RBMs and energy based models have not found their way into GOOGL’s production; NNs are finding applications
    • Simplification & Complexity : NNs, once you get them working, forms this nice “Algoritmically simple computation mechanisms” in a darkish-brown box ! Less sub systems, less human engineering ! At a different axis of complexity
    • Embedding editorial policies is not easy, better to overlay them … [Note : We have an architecture where the pre and post processors annotate the recommendations/results from a DL system]
  • There are some interesting papers on both the topics that Jeff mentioned (This my reading list for the next few months! Hope it is useful to you as well !):
    1. Efficient Estimation of Word Representations in Vector Space [Link]
    2. Paragraph vector : Distributed Representations of Sentences and Documents [Link]
    3. [Quoc V.lee ‘s home page]
    4. Distributed Representations of Words and Phrases and their Compositionality [Link]
    5. Deep Visual-Semantic Embedding Model [Link]
    6. Sequence to Sequence Learning with Neural Networks [Link]
    7. Building high-level features using large scale unsupervised learning [Link]
    8. word2vec Tool for computing continuous distribution of words [Link]
    9. Large Scale Distributed Deep Networks [Link]
    10. Deep Neural Networks for Object Detection [Link]
    11. Playing Atari with Deep Reinforcement Learning [Link]
    12. Papers by Google’s Deep Learning Team [Link to Vincent Vanhoucke’s Page]
    13. And, last but not least, Jeff Dean’s Page

The talk was cut off after ~45 minutes. Am hoping they would publish the rest and the slides. Will add pointers when they are on-line. Drop me a note if you catch them …

Update [10/12/14 21:49] : They have posted the second half ! An watching it now !

 Context : I couldn’t attend the RecSys 2014; luckily they have the sessions on YouTube. Plan to watch, take notes & blog the highlights; Recommendation Systems are one of my interest areas.

  • Next : Netflix’s CPO Neal Hunt’s Keynote
  • Next + 1 : Future Of recommender Systems
  • Next + 2 : Interesting Notes from rest of the sessions
  • Oh man, I really missed the RecSysTV session. We are working on some addressable recommendations. Already reading the papers. Didn’t see the video for the RecSysTV sessions ;o(

A Glimpse of Google, NASA & Peter Norvig + The Restaurant at the End of the Universe


I came across an interesting talk by Google’s Peter Norvig at NASA.

Of course, you should listen to the talk – let me blog about a couple of points that are of interest to me:

Algorithms that get better with Data

Peter had two good points:

Norvig-01

  • Algorithms behave differently as they churn thru more data. For example in the figure, the Blue algorithm was better with a million training dataset. If one had stopped at that scale, one would be tempted to optimize that algorithm for better performance
  • But as the scale increased, the purple algorithm started showing promise – in fact the blue one starts deteriorating at larger scale. The old adage “don’t do premature optimization” is true here as well. 
  • Norvig-02In general, Google prefers algorithms that get better with data. Not all algorithms are like that, but Google likes to go after the ones with this type of performance characteristic. 

There is no serendipity in Google Search or Google Translate

  • There is no serendipity in search – it is just rehashing. It is good for finding things, but not at all useful for understanding, interpolation & ultimately inference. I think Intelligent Search is an oxymoron ;o)
  • Same with Google Translate. Google Translate takes all it’s cue from the web – it wouldn’t help us communicate with either the non-human inhabitants of this planet or any life form from other planets/milky ways.
    • In that sense, I am a little disappointed with Google’s Translation Engines.  OTOH, I have only a minuscule view of the work at Google.

The future of human-machine & Augmented Cognition

And, don’t belong to the B-Ark !

The Curious Case of the Data Scientist Profession


Data Science & the profession of a Data Scientist is being debated, rationalized, defined and refactored … I think the domain & the profession is maturing and our understanding of the Mythical Data Scientist is getting more pragmatic. Earlier, I had proposed the idea of a Data Science Engineer last year with similar thoughts; and elaborated more at “Who or what is a Data Scientist?“, “Building a Data Organization that works with Business” & “The sense & sensibility of Data Science devOps“. Things are getting more interesting …

Now to the highlights:

1. Data Scientist is multi-faceted & contextual

  • Two points – It requires a multitude of skills & different skill sets at different situations; and definitely is a team effort.
  • This tweet sums it all
  • DataScienceTeam
  • Sometimes a Data Scientist has to tell a good business story to make an impact; other times the algorithm wins the day
    • Harlan in his blog identifies four combinations – Data Business Person, Data Creative, Data Engineer & Data Researcher
      • I don’t fully agree with the diagram – it has lot less programming & little more math; math is usually built-in the ML algorithms and the implementation is embedded in math libraries developed by the optimization specialists. A Data Scientist should n’t be twiddling with the math libraries
    • The BAH Field Guide suggests the following mix:
    • Data Scienc 03
    • I would prefer to see more ML than M. ML is the higher from of applied M and also includes Statistics
  • Domain Expertise and the ability to identify the correct problems are very important skills of a Data Scientist, says John Forman.
  • Or as Rachel Schutt at Columbia quotes:
    • Josh Wills (Cloudera)
      • Data Scientist (noun): Person who is better at statistics than any software engineer & better at software engineering than any statistician

    • Will Cukierski (Kaggle) retorts
      • Data Scientist (noun): Person who is worse at statistics than any statistician & worse at software engineering than any software engineer

2. The Data Scientist team should be building data products

3.  To tell the data story effectively, the supporting cast is essential

  • As Vishal puts it in his blog,
    • Data must be there & processable – the story definitely depends on the data
    • Processes & buy-in from management – many times, it is not the inference that is the bottle neck but the business processes that needs to be changed to implement the inferences & insights
    • As the BAH Field Guide says it:
    • Data Scienc 04
    • DS01

 4.  Pay attention to how the Data Science team is organized

5. Data Science is a continuum of Sophistication & Maturity – a marathon than a spirint

Let me stop here, I think the blog is getting long already …

Is it still “Artificial” Intelligence, if our Computers learn -to think- from the workings of our Brain ?


Image

  • In fact that would be Natural Intelligence ! Intelligence is intelligence – it is a way of processing information to arrive at inferences, recommendations, predictions and so forth …

May be it is that Contemporary AI is actually just NI !

Point #1 : Machines are thinking like humans rather than acting like Humans

  • Primitives inspired by Computational Neuroscience like DeepLearning are becoming mainstream. We are no more enamored with Expert Systems that learn the rules & replace humans. We would rather have our machines help us chug through the huge amount of data.

We would rather interact with them via Google Glass – a two-way, highly interactive medium that act as a sensor array as well as augment cognition with a digital overlay over the real world

  • In fact, till now, our computers were mere brutes, without the elegance and finesse of the human touch !
  • Now the computers are diverging from Newtonian determinism to probabilistic generative models.
  • Instead of using greedy algorithms, the machines are now being introduced to Genetic Algorithms & Simulated Annealing. They now realize that local minima, computed via exhaustive brute force, are not the answers for all problems.
  • They now have knowledge graphs and have the capability to infer based on graph traversals and associated logic

Of course, deterministic transactional systems have their important place – we don’t want a probabilistic bank balance!

Point #2 : We don’t even want our machines to be like us

  • The operative word is “Augmented Cognition” – our machines should help us where we are not strong and augment our capabilities. More later …
  • Taking a cue from the contemporary media, “Person Of Interest” is a better model than “I,Robot” or “Almost Human” – a Mr.Spock, rather than a Sonny; Logical but resorts to the improbable and the random, when the impossible has been eliminated !

Point #3 : Now we are able to separate Interface from Inference & Intelligence

AI-03

  • New Yorker asks, “Why can’t my computer understand me?” Finding answers to questions like “Can an alligator run the hundred-meter hurdles?” is syntax.
  • NLP (Natural Language Processing) and it’s first cousin NLU(Natural Language Understanding) are not intelligence, they are interface.
  • In fact, the team that built IBM Watson realized that “they didn’t need a genius, … but build the world’s most impressive dilettante … battling the efficient human mind with spectacular flamboyant inefficiency”.

Taking this line of thought to it’s extreme, one can argue that Google (Search) itself is the case and point of an ostentatious and elaborate infrastructure for what it does … no intelligence whatsoever – Artificial or Natural ! It should have been based on knowledge graph rather than a referral graph. Of course, in a few years, they would have made huge progress, no doubt.

  • BTW, Stephen Baker has captured the “Philosophy of an Intelligent Machine” very well.
  • I have been & am keeping track of the progress by Watson.
  • Since then, IBM Watson. itself, has made rapid progress in the areas of Knowledge Traversal & Contextual Probabilistic Inferences i.e. ingest large volume of unstructured data/knowledge & reason about it
  • I am not trivializing the effort and the significance of machines to understand the nuances of human interactions (speech, sarcasm, slang, irony, humor, satire et al); but we need to realize that, that is not an indication of intelligence or a measure what machines can do.

Human Interface is not Human Intelligence, same with machines. They need not look like us, walk like us, or even talk like us. They just need to augment us where we are not strong … with the right interface, of course

  • Gary Markus in New Yorker article “Can Super Mario Save AI” says “Human brains are remarkably inefficient in some key ways: our memories are lousy; our grasp of logic is shallow, and our capacity to do arithmetic is dismal. Our collective cognitive shortcomings are so numerous … And yet, in some ways, we continue to far outstrip the very silicon-based computers that so thoroughly kick our carbon-based behinds in arithmetic, logic, and memory …

Well said Gary. Humans & Machines should learn from the other and complement … not mimic each other … And there is nothing Artificial about it …

I really wish we take “Artificial” out of AI – Just incorporate what we are learning about ourselves into our computers & leave it at that !

Finally:

AI-04-01

Deep Learning for the masses


Updates:

Back to the main feature …

An interesting blog in GigaOm by Derrick Harris on Deep Learning for the masses. What interested me most was Jeremy Howard from Kaggle.

DL-Dean-12

  • “…It’s going to enable whole new classes of products that have never existed before …”
  • But there’s a catch: deep learning is really hard. So far, only a handful of teams in hundreds of Kaggle competitions have used it. Most of them have included Geoffrey Hinton or have been associated with him.
    • Yep, it is hard. We are trying to bootstrap an application system and haven’t even scratched the surface – so it seems
  • If data scientists in places outside Google could simply (a relative term if ever there was one) input their multidimensional data and train models to learn it, that could make other approaches to predictive modeling all but obsolete.
    • Yep. Deel Learning is being applied in image recognition, translation et al. It would be interesting to see how the technologies can be applied to retail, banking, manufacturing et al

I also think the broader architecture of the three amigos viz Interface,Inference & Intelligence needs to come together

Finally,

Smarter Models = Smarter Apps – Yep, definitely !