Cruise co-founder and CTO, Kyle Vogt, said Friday that recall reports published annually by California regulators are not a representation of the commercial preparation or safety of driverless cars.
Vogt, in a long publication in Medium, asked for a new metric to determine if an autonomous vehicle is ready for commercial deployment. The publication suggests that the autonomous vehicle company, which had a valuation of $ 19 billion as of May, is already developing more comprehensive metrics.
The California Department of Motor Vehicles, which regulates permits for autonomous vehicle testing on public roads in the state, requires companies to submit an annual report detailing "disconnections," a term that means the number of times that Drivers have had to take control of a vehicle. car. The DMV defines a disconnection as each time a test vehicle operating on public roads changes autonomously to manual for an immediate reason related to safety or due to a system failure.
"It is unfortunately unsuitable for most uses beyond those of DMV," Vogt wrote. "The idea that disconnections give a significant signal about whether an AV is ready for commercial deployment is a myth."
These withdrawal reports will be published in a few weeks. Cruise shared some of its disconnection data, specifically the number of miles traveled per disconnection event, between 2017 and 2019.
The so-called race to market autonomous vehicles has involved a good amount of theater, including demonstrations. This lack of data has made it almost impossible to determine whether a company's autonomous cars are safe enough or are ready for the large and very real stage of moving people from point A to point B on city streets. Withdrawal reports as defective as they could have been were one of the only data to which the public and the media have access.
How safe is safe enough?
While that data may provide some ideas, it does not help answer the fundamental question for all AV developers who plan to implement robotics for the public: "How safe is it safe enough?"
Vogt's comments point to Cruise's efforts to find a practical means to answer that question.
"But if we cannot use the disconnection rate to measure commercial readiness, what can we use? Ultimately, I believe that for an AV operator to deploy AV at scale in a fleet of shared trips, the general public and Regulators deserve empirical evidence that an AV has a performance that is super human (better than the average human driver), that the deployment of AV technology has a positive overall impact on automotive safety and public health.
This requires a) data on the true performance of human and AV drivers in a given environment and b) an objective comparison of apples with apples with statistically significant results. We will deliver exactly that once our AVs are validated and ready for implementation. Hope to hear more from us about this important issue soon. ”
The competitors agree
Cruise is not the only company that questions withdrawal reports, although this could be the strongest public call to date. Waymo told Newsdio that he has a similar vision.
Reports have long been a source of distress among AV developers. The reports provide information that may be useful to the public, such as the number of vehicles tested on public roads. But it is not a complete picture of any company's technology.
The reports are very different; Each company provides varying amounts of information, all in different formats. There is also disagreement about what is and what is not a disconnection. For example, this problem received more attention in 2018 when Jalopnik questioned an incident related to a Cruise vehicle. In that case, a driver took manual control of the wheel when passing through an intersection, but it was not reported as a disconnection. Cruise told Jalopnik at the time that he said he did not meet the standard for California regulations.
The other problem is that disconnections do not provide a comparison of "apples with apples" technology, as these test vehicles operate in a variety of environments and conditions.
Disconnections also tend to increase and decrease along with the scale of the tests. Waymo, for example, told Newsdio that his disconnections will likely increase as he expands his tests in California.
And finally, more companies are using simulation or virtual tests instead of sending car fleets on public roads to test each new version of software. Disconnection reports do not include any of that data.
The Vogt publication also called on the industry to carefully carry out "curated demonstration routes that avoid urban areas with cyclists and pedestrians, restrict geofences and pick-up / drop-off locations, and limit the types of maneuvers the AV will attempt during the travel".
The take could be interpreted as a take on Waymo, which recently conducted driverless demonstrations on public streets in Chandler, Arizona, with reporters. Newsdio was one of the first to have a driverless trip last year. However, demonstrations are a common practice among many other autonomous vehicle startups, and are particularly popular at events like CES. Cruise has conducted at least one demonstration, which was with the press in 2017.
Vogt suggested that unedited driving images that "cover long periods of handling in real-world situations" are difficult to falsify and are a more qualitative indicator of technological maturity.