The Art of NFL Ranking, the ELO Algorithm and FiveThirtyEight


In this blog, I will focus on the NFL Ranking based on the ELO algorithm that Nate Silver’s FiveThirtyeight uses. The guys at 538 have done a good job.The ELO and NFL ranking was part of my workshop at the Global Big Data Conference this Sunday. The full presentation is in slideshare


ELO – the algorithm made famous by Facebook & depicted in the movie Social Network

gbdc-r-04-P30


 Basic ELO

gbdc-r-05-P20

The k-Factor is the main leverage point to customize the algorithm for different domains.

  • For example Chess has no notion of a season; Soccer,Football & Basket ball are dependent on seasons – teams change during different seasons
  • Chess has no score to consider except WIn,Lose or Draw; but ball games have scores that need to be accommodated
  • For Chess k=10; for soccer it varies from 20 to 60; 20 for friendly matches to 60 for World Cup Finals
  • As we will see later, NFL adjusts k with the Margin Of Victory Multiplier
  • NFL also adjusts k to weigh recent games more heavily, w/ exponential decay
  • There are also mechanisms for weighing playoffs higher than regular season games (We will see this in Basketball)

538’s take on ELO

gbdc-r-05-P21

gbdc-r-05-P22


NFL 2014 Predicts & Results

The R program ELO-538.R is in Github

2014 Ranking Table

gbdc-r-05-P27

gbdc-r-05-P29

gbdc-r-05-P31

gbdc-r-05-P32


To Do

  1. Exponential decay with more weight for recent games – later in the season
  2. Calculate the rankings from 1940 to present, draw graphs like this from 538

Business Users Shouldn’t touch Hadoop even with a 99-foot pole !


Yep, I know, it is 10 foot pole; and the origin is from “10-foot poles that river boatmen used to pole their boats with”[1]

Back to the main feature, I was reading an piece by Andrew J Burst at GigaOM that “Hadoop needs a better front-end for business users”

Yikes. This is terrible … I would argue, no, make that insist, that business users be kept as far away as possible from Hadoop (& similar frameworks)

Allow me to elaborate …

  • Business users do need highly interactive analytic dashboards with knobs & dials into our deep machine learning models and sliders onto our AI machines, No doubt.
  • We don’t want to abandon our beloved business users with static-rigid-newtonian-deterministic artifacts; we want them to have living, (fire) breathing intelligent-inferential-predictive-models

  • But that control & interactivity is into a business analytics beast that has multiple layers, not directly onto a Hadoop or hadoop-like system.

Also separating the “what” form the “how” by a declarative interface is very important

You see, analytics has at least four layers viz. Infrastructure, Intelligence, Inference & Interface

I4

  • Hadoop is Infrastructure, Spark is Infrastructure – The “How”
  • Machine Learning algorithms are Intelligence – Again lots of “How”
  • Models are Inference – the “What” Plus some “How”
  • Dashboard is the Interface (usually) – definitely the “How”
  • Interface can be recommendations, financial predictions, ad forecasts or even actual devices that interface to predictive models

  • And business needs knobs & dials at the Inference & Interface layers
    • The Infrastructure then appropriately fires frameworks Hadoop or Spark or Java or iPython …
  • Digging deeper, Hadoop itself has three layers – none of them operable by a business user, but real work horses

    • HDFS – the distributed File System

    • MapReduce – the distributed data parallel computation engine

    • HBase – the NOSQL data store

Back to Andrew’s points, Hadoop (and it’s cousins) should remain as a tool for the Chefs; but diners do need to express their choices and have the ability to “tweak” the seasonings, portions or even the amount of cooking; a declarative interface (which tells what but not how) comes from the domain specific menus catered by the restaurants which focus on respective culinary styles or even a fusion !

Now I am getting Hungry ! On my way to downstairs (am at the Hilton – NY Fashion District) to my favorite Chipotle – who in fact gives me the declarative freedom, without getting into their kitchen and the need to handle the saucepans ;o) It is better that way because I am terrible with cooking and spice measures – I can tell less salt but not the amount !


[1] http://en.wiktionary.org/wiki/not_touch_something_with_a_ten_foot_pole

[2] Interface from http://img1.mxstatic.com/wallpapers/1bb91493c637d7c5ed6e1cefbef87ec1_large.jpeg

Building a Data Organization that works and works with business


One thing that caught my attention on Netflix’s Neil Hunt’s interview with Gigaom was this :

A Data Organization that works & works with business

Well said. That explains Netflix’s data Science in a nutshell that all Data Scientists should emulate !

From a Chief Data Scientist’s perspective, I really like their way of looking at Data Science viz:

  • The folks who do data Science for the whole business
  • The folks who build algorithms & 
  • The folks who do data engineering

In fact I had a blog on this specialization of Data Science skills

Netflix is putting more weight on actual behavior ! Interesting, we are also seeking similar effects ie differentiate between falling asleep on a couch vs. actually watching a TV show !  It is hard inference … Netflix has the blocker, I have nothing ;o(

Binge watching … interesting … We are actually working on algorithms to figure that out and change the ad mix. I plan to talk more at the TM Forum Digital Disruption Panel on December 9th !

bdtc-py-18-P76Finally, the fact that importance of Netflix recommendation engine is underrated is so true. In many ways the recommendation algorithms and engines are core to many systems.

In fact, we have a reverse recommendation strategy ! We recommend users to ads !

The Sense & Sensibility of a Data Scientist DevOps


The other day I was pondering the subject of a Data Scientist & model deployment at scale as we are developing our data science layers consisting of Hadoop, HBase & Apache Spark. Interestingly earlier today I came across two artifacts – a talk by Cloudera’s @josh_wills and a presentation by (again) Cloudera’s Ian Buss.

The talks made a lot of sense independently, but add a lot more insight – collectively !  The context, of course, is the exposition of the curious case of data scientists as devops. The data products need an evolving data science layer …

It is well worth your time to follow the links above and listen to Josh as well as go thru Ian’s slides. Let me highlight some of the points that I was able to internalize …

Let me start with one picture that “rules them all” & summarizes the  synergy. The “Shift In Perspective” from Josh & the Spark slide from Ian

JW-01

The concept of Data Scientist devops is very relevant. It extends the curious case of the Data Scientist profession to the next level.

Data products live & breath in the wild, they cannot be developed and maintained with a static set of the data. Developing an R model and then throwing it over the wall for a developer to translate won’t work.  Secondly, we need models that can learn & evolve in their parameter space.

 

JW-02

 I agree with the current wisdom that Apache Spark is a good framework that spans the reason,model & deploy stages of data. 

Other interesting insights from Josh’s talk.

Finally,

The virtues of being really smart is massively overrated; the virtues of being able to learn faster is massively underrated

Well said Josh.
P.S: Couldn’t find the video of Ian’s talk at the Data Science London meetup. Should be an interesting talk to watch …

Data Science Folk Knowledge & Words of Wisdom


I have collected some words of wisdom on Data Science & Machine Learning for my pycon 2014 Tutorial “Data Wrangling”.
Posting as a blog. The pdf is at Slideshare. Would appreciate comments, insights & corrections.

Slide01

Slide02

  • Slide11
  • Slide12
  • Slide13
  • Slide14
  • Slide15
  • Slide16
  • Slide17
  • Slide18
  • Slide19
  • Slide20
  • Slide21
  • Relevant Papers To Read

  • Slide22
  • An ordered list of mooc links & books. This was my answer in Quora

  • Slide23

A Glimpse of Google, NASA & Peter Norvig + The Restaurant at the End of the Universe


I came across an interesting talk by Google’s Peter Norvig at NASA.

Of course, you should listen to the talk – let me blog about a couple of points that are of interest to me:

Algorithms that get better with Data

Peter had two good points:

Norvig-01

  • Algorithms behave differently as they churn thru more data. For example in the figure, the Blue algorithm was better with a million training dataset. If one had stopped at that scale, one would be tempted to optimize that algorithm for better performance
  • But as the scale increased, the purple algorithm started showing promise – in fact the blue one starts deteriorating at larger scale. The old adage “don’t do premature optimization” is true here as well. 
  • Norvig-02In general, Google prefers algorithms that get better with data. Not all algorithms are like that, but Google likes to go after the ones with this type of performance characteristic. 

There is no serendipity in Google Search or Google Translate

  • There is no serendipity in search – it is just rehashing. It is good for finding things, but not at all useful for understanding, interpolation & ultimately inference. I think Intelligent Search is an oxymoron ;o)
  • Same with Google Translate. Google Translate takes all it’s cue from the web – it wouldn’t help us communicate with either the non-human inhabitants of this planet or any life form from other planets/milky ways.
    • In that sense, I am a little disappointed with Google’s Translation Engines.  OTOH, I have only a minuscule view of the work at Google.

The future of human-machine & Augmented Cognition

And, don’t belong to the B-Ark !