The Uber car saw her six seconds early
The NTSB's preliminary report on the Tempe crash shows the sensors worked. Classification flickered, braking was disabled, and the human was supposed to be the backup.
The NTSB released its preliminary report on Thursday about the Uber test vehicle that killed Elaine Herzberg in Tempe in March, as she walked a bicycle across a dark road. I read it twice. It’s short, and it’s worth reading in full if you build any system that’s supposed to be safe.
The car’s radar and lidar detected her about six seconds before impact. So this wasn’t a sensor failure. The system saw something in the road with plenty of time to stop.
What went wrong was everything after detection. According to the report, the software classified her first as an unknown object, then as a vehicle, then as a bicycle, each with different expectations about where she’d go next. Each time the classification changed, the prediction of her path changed with it. At 1.3 seconds before impact, the system decided emergency braking was needed. But Uber had disabled the Volvo’s own emergency braking while the car was under computer control, to reduce erratic behavior, and its own system wasn’t designed to brake hard on its own. It was designed to rely on the safety driver. The system also had no way to alert the driver that it wanted to brake. The driver was looking down, and in the video released earlier she looks up just before impact.
Two things stand out to me as an engineer.
First, a perception system that’s unsure what something is should still know it’s something. Six seconds of “there’s an object in my lane, I’m not sure what it is” is a very good reason to slow down. As I read the report, uncertainty about the category led to a failure to act on the more basic fact. People do the opposite. If you see a shape you can’t identify on the highway at night, you brake first and figure it out later. The object’s physical presence matters more than its label.
Second, the safety design depended on a human who had nothing to do. Anyone who has run operations knows what happens when you ask a person to watch an automated system for hours and step in within a second when it fails: they stop watching. That’s not a character flaw, it’s how attention works. A backup that depends on sustained vigilance isn’t much of a backup.
I wrote two years ago that the hard part of driving is the long tail of rare situations. This case adds something: even when the system notices the rare situation, the way it’s built to handle uncertainty and hand off responsibility determines whether anyone survives it. A self-driving stack is a chain of perception, prediction, planning and a human, and the failure here was in the connections between them.
We make much less dangerous things at Amanda, but the lesson carries over. When our system isn’t sure, the right move is a safe fallback, like “please see the desk,” not a confident guess and not silently waiting for someone to notice.