The Bridges of Pittsburgh County … That Autonomous Cars can’t SLAM through !

Same as my post in Linkedin

Autonomous cars do bring out interesting nuances to the normal things that we take for granted and don’t think twice about !

Business Insider’s article “Here’s why self-driving cars can’t handle bridges” fits this category.

“Bridges are really hard,… and there are like 500 bridges in Pittsburgh.”

Of course, it is the infinite (or near infinite) context that we, humans can process and machines aren’t even close … But, one would think bridges would be easier – no distractions, well designed straight roads; of course with the current GPS accuracy, the car might think that it is in the water and start rolling out it’s fins !!

“You have a lot of infrastructure on the bridge above the level of the car that we as humans take into account, … But when you sense those things with a sensor that doesn’t have the domain knowledge that we do … you could imagine that the girders coming up from the side of the bridge and that kind of thing would be disturbing or possibly confusing.”

Pittsburgh does have bridges, lots of them … There is even a BBC documentary! Even how the city deals with the bridges is interesting.

In fact Pittsburgh is called “The City of Bridges”, even though some have different interpretations (we will come to that discussion in a minute)

While we are on the subject, I do have a couple of books for the Uber Car to read ! It can even order them through it’s robotic friend Alexa ! or drive to wherever fine books are sold, on it’s own time – Uber might not pay for the impromptu solo drive.

  1. Bob Regan’s Book is the first one to read
  2. Next is Pittsburgh’s Bridges (Images of America)
  3. The book Bridges… Pittsburgh at the Point… a Journey Through History gives interesting perspectives the riders would enjoy (of course, the ones with enquiring minds…)
  4. Finally, the hardcore bridge fans would be thrilled to hear from Pittsburgh’s Bridges: Architecture and Engineering

Now, to SLAM, it is the set of algorithms collectively called Simultaneous Localization And Mapping – a very interesting topic by itself.

In short, a SLAM system needs known points in addition to unknown points, to reason about & figure out it’s trajectory – bridges have less of known points it can rely on …

We can definitely employ Deep Learning ConvNets as well as traditional computer vision with a dash of contextualization is a good start … that is a topic for another time (sooner than later…). Probably an interesting opportunity for or

For those snappy Machine Learning experts, there is even a Pittsburgh Bridges Data Set at UCI, to start with ! Probably nowhere near the data needed to train modern Convolutional Nets, but one can augment the images with algorithms like Flip, Jitter, Random Crop and Scale et al.

If we think Pittsburgh is difficult, wait until Uber starts autonomous driving in Amsterdam ! While Pittsburgh has 446 bridges, many sources put Amsterdam with over 1000 bridges that cars can travel. There are many bicycle and pedestrian bridges in Amsterdam that an Uber car wouldn’t be interested in – except, of course, to pick up the tired pedestrians ;o). The which-city-has-max-number-of-bridges discussions can be followed here:



  • The bridges in Pittsburgh are not painted Yellow (as one might tend to think) but Aztec Gold !
  • And yep, it is Allegheny County ! But Pittsburgh rhymes better ;o)



Autonomous Vehicles – On the road to a Standardization : 5 Questions

In view of the recent developments, the Automated Vehicle Symposium 2016 and the SAE On-Road Automated Vehicle standards work is more relevant. The Connected Car meetup is organizing a meeting with the standards committee; of course, I plan to attend the meetup.

As I was contemplating, extending my experience from OASIS, W3C, IETF and EU FP6 STFs, thought of a few questions that would be good to get clarified. I will try to get answers on Monday, and if you have any, pl add to the comments.

P.S: I will post notes from the meetup next week.

1. Software Telematics

  • One of the important aspects of standardization is the ability to look inside a system declaratively, i.e. one doesn’t have to know “how” a system implements “stuff” but the “what” should be inspected, understood, analyzed, verified and debated (if necessary); the software equivalent of the CAN bus

Question 1 : Will the SAE define the essential semantics of the software pins – a basic curated schema with capabilities for extensibility?

Thinking more, this probably is a bigger effort that what is evident at a first glance, as we have to define the software model before we can define the telematics.

2. Verification of behavior by Induction vs deduction

  • We can either say that a system works because it has worked for the last 130,000 miles (deduction) or strive to prove the correctness of a system by analysis(induction). The proverbial deduction vs induction.

Question 2 : Will the standards effort address the mechanics of behavior verification ?

3. Definitions

  • Of course, definitions are the essential and fundamental ingredients of standardization. And definitions what a system is NOT is equally or more important.
  • As we are seeing in the media, defining AI and autonomous behavior are way more difficult and subject to multiple interpretations. For example, in the autonomous world, we can define AI like so.

  • Probably we are not looking for a humanoid, but we still need intelligent interactivity with the environment which includes pedestrians, drivers, intelligent infrastructure and other vehicles. We also need the Robots Rules of Order.

Question 3 : How deep does the committee plan to define the concepts & components ?

4. Simulation

  • Simulation is another interesting topic that we need to address. Companies do claim 100s of thousands of miles of virtual driving, but in order to characterize, compare and contrast, the simulation frameworks need to be equalized.

Question 4 : Does the committee plan to specify the essentials of simulation ?

  • Probably, a broader framework, with some of the software pins, would be a good start

5. MHI – Machine to Human Interface

  • The machine to human interactions are also another essential aspect.
  • When we drive a different car, e.g. a rental car, we don’t need to study new interfaces or vocabulary. What we know about brakes, accelerator, R and D positions – all are valid.
  • A similar set of ontologies and taxonomies are required for autonomous driving. To make a point, compare the driving controls with the rest of the controls eg infotainment systems, climate controls;  sometimes it takes a lot of effort to understand the infotainment systems that we are not familiar with.  That is OK for connecting iTunes via USB to a car, but not OK for autonomous driving.

Question 5 : Does the committee plan to standardize the interfaces – not just the control but also the metadata ?

What are your thoughts ? Do you have more questions ?

P.S: We haven’t addressed a host of related domains like v2v protocols, v2I protocols(vehicle-to-intelligent infrastructure), security mechanisms, embedding behaviors and extending to the world of drones !

5 Lessons on AI from the Tesla Autopilot Fatality

Unfortunately it takes extreme repercussions for us to feel in our bones, the limitations of our technologies.

Three points :

  1. I have included relevant links about this incident at the end of the blog (incl the AutoPilot v8 with Radar). Informative read
  2. One of my parent died in an automobile accident; so I do know, first hand, the human toll – I do not take this lightly; in fact the reverse is true
  3. And the views expressed in my writing are my own and do not reflect any organization I am part of … now or in the future …


Lesson 1 : Our machines inherit our faults (so far …)

As I pointed out in one of my AI blogs:



Lesson 2 : Many domains are not forgiving to byzantine failures

We are learning that painful lesson whether they are rockets, airplanes or cars. Even though we freak out of snapchat is down for an hour, we can survive that, but not these. The drivers need to understand the downside of technologies and be alert.

Lesson 3 : Mission Critical Systems should have redundancy, over coverage & independency

For example multiple sensor sources & probably independent situational interpretation. I saw the following from somewhere where the Japanese Ministry talks about “correcting the wrong train of thoughts”:

Lesson 4 : Swarm Intelligence

Lesson 5 : This might lead to some level of Standardization & Legalization

  • Standardization of components & protocols
  • Legislation/Standardization of algorithms or semantic behaviors incl image recognition, policies and pragmas …
  • Even driver education and certification to dive autonomous vehicles !



  7. Tesla’s Response to fortune’s Article
  22. Finally AutoPilot v8 with Radar !


Robot’s Rules of Order

As a designer of AI & autonomous behaviors, this week was pretty interesting:

Both are must read & discuss for the AI community. The abstracted 8 rules are :

This could beg the question, what exactly is an AI ? Let me make an attempt from an autonomous vehicle (cars, drones et al) perspective, which might not be complete or sufficient for other situations ….

What says thee ?