Now is not the Winter of Our Discontent (but it may be for others)

2019 is an interesting time in the world of autonomous vehicles (AV). Interesting times in the Chinese sense for most companies out there, but interesting in a different and very good way for us here at Perrone Robotics. I'll explain.

2017: The Pre-Launch Party

Hey! we're all different together!

In 2017 there was an almost giddy sense of autonomous vehicles around every corner and in every garage. More AV startups joined existing OEMs and all parties were making noise and doing demos. Analysts (just picking one of many here) fairly took stock of all that energy and started forecasting autonomous vehicles to arrive by 2020.

However, almost all of these companies had followed the same plan:

• Hire some smart guys out of research or out of university who know a lot about AI & neural net software;
• Download open source ROS components to provide a fast onramp to gaining control of a vehicle and to do basic sensing with cheap sensors like cameras;
• Obtain a drive-by-wire car or something easy to work with;
• Add some special sauce to improve some module of the original ROS code;
• Spend money on a cool demo in a well-controlled environment;
• <repeat>

This isn't a bad kick-off plan, but building a truly autonomous car isn't something you jump into overnight. Problems with this plan become apparent after you travel a few miles down this road. More to follow.

2017 for Perrone Robotics found us very upbeat as we moved into our new location complete with a private test track, and celebrated our 14th birthday as well as the holidays. We focused our efforts to take our unique, patented autonomous software engine called MAX into different domains - automotive, industrial, and robotic. We did a very public demo of our capabilities at the TU-Automotive show in June where we showed our MAX engine running on three different processor architectures. We drove a vehicle around a course and performed pedestrian avoidance, pacing, and passing stalled traffic. All of this on a MAX platform that had been refined for over a decade.

2018: The Headache/Hangover

After such an exuberant 2017 in the industry, 2018 figured to provide many more demos and live presentations of autonomous vehicles driving everywhere. However as the year wore on, the commentary grew more subdued and noted that autonomy was still coming, but it might take longer than expected. While progress was made, most companies discovered that the 2017 plan led to a consistent set of problems:

Last year was fun, but oww...

• Autonomous driving is hard. Having smart guys on your team helps, but what helps more is someone with years of experience in robotic controls with a focus on sensor integration and fusion. Automotive autonomy is not just simulations running on cloud servers, it is about moving tons of steel and rubber in the real world.
• AI is non-deterministic. Everyone started to discover that while they could get their car to run well down a test street, moving a few blocks away caused failures due to minor variations in the environment that occur over the course of a day or a month or a season. Deep learning systems can make perception mistakes because all they have is the set of data they were trained with. It's a complicated world out there even for homo sapiens with hundreds of millennia of visual training. The bottom line is that you never know what will come out of an AI system - often it's good or great, but you never know for sure.
• Re-training AI systems for widely diverse conditions turns out to be governed by "the law of diminishing returns". AI systems are fantastic at statistically matching specific patterns within a given domain. However, vehicular autonomy requires different streets, roads, vehicles, pedestrians, lights, signs, bikes, birds, open car doors, bouncing balls, etc. Trying to retrain AI systems to handle each new case causes extensive rounds of test and re-test to ensure everything that used to work still works. Supporting new areas or new modes of travel gets increasingly harder.
• Real-time control is also hard. Control is much more manageable than trying to retrain AI systems all the time, but in order to stay in control, information must be received and processed and acted upon at least 10 times a second in the general case. Some control cases need all this to happen 100 times a second or more, but those cases have much narrower inputs and control domains than the "car" as a whole. So when AI systems move out of the labs and onto the street, they have to get a lot of work done in a very specific period.

By contrast, 2018 for Perrone Robotics was about continuing our momentum by delivering on projects for a European automaker and our mining haul truck customer Liebherr as well as partnering with SAE for demos in Tampa and in Ft. Myers. The SAE demos required us to show up, plot a course, and test it in a day or less and then run it without error for a week while giving members of the public rides. If you are highly dependent on AI localization, I just don't think that's happening - it's more like you need a couple of weeks to record, learn, and test.

For us, 2018 was another year of refinement. With our modular architecture we were able to address new scenarios and obstacles and even new models of operating without adversely impacting the work that had gone on before. This is not to say we can't improve - but rather, our architecture allows us to continually improve, and even replace, modules that are needed to address new challenges. What's more, since we are able to use the same exact autonomous engine in every domain, each implementation can "learn" from the others and the core platform gets better and better. Work done in one domain can be applied directly to another domain without having to rework the full solution.

Finally, at the end of 2018, we announced our TONY initiative which aims provide autonomous shuttles for daily use. Our differentiation from other autonomous shuttle companies out there is that we use standard shuttle vehicles - already road proven and approved for road use. We combine our autonomous software engine with controls applied to the vehicle to create an autonomous solution out of any shuttle vehicle available. We also announced that we would be going live in 2019 with a daily autonomous shuttle in our hometown of Crozet, VA. Stay tuned for the launch announcement soon.

2019: The Winter of Discontent - for Some - but not for Perrone Robotics

Hard to localize on roadside artifacts when they are covered with snow...

As 2019 emerges, many OEMs are adjusting their AV deployment timeframe, and there is broad discussion about Uber's and Lyft's true AV capabilities. As well, Tesla is talking about fully autonomous production vehicles, but there seems a fair amount of uncertainly about how well they work. So far, it seems a bit of repeat of 2018.

In February a very interesting article came out from Radio Free Mobile that crystallized for me this overarching notion that "Winter is Coming" for AI as used in autonomous vehicles. In the article, Richard Windsor raises the above points and more in a very concise, cogent fashion.

All of this is not to say that using AI in autonomy is bad - the question is: how is it used? Is it the core functionality for the vehicle such that the vehicle must be trained how to drive? Or is it a tool in a toolbox which can be brought out when needed? At Perrone Robotics, we are strongly in the second camp. Our approach is to use our own algorithms and rules developed over 15 years of use and enable the vehicle to follow a course, avoid obstacles, and obey the rules of the road. For us, AI and deep learning are useful as tools to consult, but we are not tied down by them.

For example, our MAX solution uses a neural net solution to find traffic lights (which can be on the left, the right, over the road, etc.) and then to determine its state (red/yellow/green). In this model, our core MAX autonomous engine handles movement of the vehicle up to the light, staying in the line of traffic, ensuring that the intersection is clear when reached - all while consulting the traffic light AI to understand the state of the light. Still green? Yellow? Now red? We can use AI in a very focused way (find the traffic light in this image), and only invoke it when needed (we are at a traffic light intersection). This is important for overall power consumption as running AI algorithms is thirsty work for processors that will reduce the range of an EV.

I mentioned use of simulation above as an issue, but it is also a tool we use and which is a great benefit for us to examine new maneuvers quickly and easily. Particularly in the case of our mining truck job where some maneuvers are unsafe with a 330 ton vehicle! But we spent the time with our simulation partner Cyberbotics to very closely match the dynamics of the specific vehicle so that we knew we could trust the outcomes. And then of course a lot of testing in the real world.

So, in 2019 Perrone Robotics will continue to move ahead with our proven and tested model that allows us to quickly set up in new areas, execute autonomy well, and continually improve our performance over time. If you have some discontent with what you're trying to do, contact us and enjoy the Spring time of autonomy!