Forget millions of test miles—it’s about the *right* miles. Empiric Earth’s Luc Vincent reveals why current autonomous vehicle testing falls short and how collecting rare, dangerous real-world events is crucial for true self-driving safety. Learn how specialized data and targeted validation are becoming the new standard for robo-taxis and autonomous trucks to navigate unpredictable environments, from Philadelphia’s unique traffic lights to Wyoming’s wildlife.
There is no universally accepted safety test for autonomous vehicles before they operate on public roads — a gap that Empiric Earth, a road data company formed this year from the merger of Nexar and Nauto, is working to close. The company is developing a framework called Apex that would function as a driver’s license for self-driving cars and autonomous trucks, scoring systems across a range of rare, dangerous scenarios rather than average driving conditions.
The stakes for freight are significant. As autonomous trucking expands and debates intensify over CDL requirements and carrier safety scores, there is no equivalent evaluation standard for the software systems replacing human drivers. Empiric Earth’s special advisor Luc Vincent argues the industry cannot wait for a legal crisis to force the issue.
Vincent said the core problem with how most autonomous vehicle developers validate their systems today is over-reliance on simulation. Training and testing within simulated environments creates what he called an echo chamber that causes performance to drift over time. Real-world data — specifically rare-event data — is what exposes the gaps simulation cannot fill.
“In the US, fatalities on highways occur only every 100 million miles, thankfully. So to experience, to observe some, to maybe see events that are that rare, you need to drive a lot,” said Vincent.
Empiric Earth has collected roughly 10 billion miles of driving data, from which it has harvested what Vincent described as the uniquely interesting events: near misses, debris on the road, flooded streets, jaywalking pedestrians, and wildlife crossings. He noted that conditions vary sharply by geography — Austin’s horizontal traffic signals, Boston’s notorious left-turn behavior, and Halloween pedestrian surges all represent edge cases that a system trained elsewhere may never have encountered.
The Apex scorecard would measure safety performance across a wide range of such situations and produce both scenario-level scores and an integrated overall rating. “You can then have a scorecard that a regulator or any kind of agency could use to decide whether it is safe to deploy some self-driving fleet or system on their roads,” Vincent said. The framework is also designed to evolve as new road users — he cited scooters as a recent example — and new behaviors emerge.
Vincent said the AV landscape is accelerating, with multiple robotaxi fleets now visible on San Francisco streets and AI approaches becoming more standardized and cheaper to implement. That growth makes a formal testing regime more urgent, not less. “As a result, I think it’s possible that in the future there will be more and more robotaxi companies… and at some point, when you have dozens or more, how do you know that developers haven’t cut corners and built something unsafe?” he said. Empiric Earth believes its rare-event dataset positions it as one of the few organizations capable of building a credible, third-party benchmark for the industry.
- Empiric Earth’s Apex framework aims to serve as a standardized safety scorecard for autonomous vehicles, filling a regulatory gap no universal test currently addresses.
- The company has collected 10 billion miles of real-world driving data, focusing on rare events such as near misses, debris, and extreme weather that simulation cannot replicate.
- U.S. highway fatalities occur only once per 100 million miles, underscoring how much data is needed to validate autonomous systems against the most dangerous edge cases.
This Summary is generated thanks to a transcription of the interview, for the full interview please enjoy the video above.
