The comparison isn't really like-for-like. NHTSA SGO AV reports can include very minor, low-speed contact events that would often never show up as police-reported crashes for human drivers, meaning the Tesla crash count may be drawing from a broader category than the human baseline it's being compared to.
There's also a denominator problem. The mileage figure appears to be cumulative miles "as of November," while the crashes are drawn from a specific July-November window in Austin. It's not clear that those miles line up with the same geography and time period.
The sample size is tiny (nine crashes), uncertainty is huge, and the analysis doesn't distinguish between at-fault and not-at-fault incidents, or between preventable and non-preventable ones.
Also, the comparison to Waymo is stated without harmonizing crash definitions and reporting practices.
TFA does a comparison with average (estimated), low-speed contact events that are not police-reported by humans, of one incident every 200,000 miles. I think that's high - if you're including backing into static objects in car parks and the like, you can look at workshop data and extrapolate that a lower figure might be closer to the mark.
TFA also does a comparison with other self-driving car companies, which you acknowledge, but dismiss: however, we can't harmonize crash definitions and reporting practices as you would like, because Tesla is obfuscating their data.
TFA's main point is that we can't really know what this data means because Tesla keep their data secret, but others like Waymo disclose everything they can, and are more transparent about what happened and why.
TFA is actually saying Tesla should open up their data to allow for better analysis and comparison, because at the moment their current reporting practice make them look crazy bad.
Tesla could share real/complete data at any time. The fact that they don't is likely and indicator the data does not look good.
I've actually started ignoring all these reports. There is so much bad faith going on in self-driving tech on all sides, it is nearly impossible to come up with clean and controlled data, much less objective opinions. At this point the only thing I'd be willing to base an opinion on is if insurers ask for higher (or lower) rates for self-driving. Because then I can be sure they have the data and did the math right to maximise their profits.
This is from elektrek. You cannot believe anything you see from them regarding Tesla. Completely corrupt site.
I think it's fair to put the burden of proof here on Tesla. They should convince people that their Robotaxis are safe. If they redact the details about all incidents so that you cannot figure out who's at fault, that's on Tesla alone.
electrek.co has a beef with Tesla, at least in the recent years.
Oh. Well then. May we see the details of these minor contact events so that people don’t have to come here and lie for them anymore?
How corrupt and unaccountable to the public is the city of Austin Texas, even, for allowing them to turn in incident reports like this?
"insurance-reported" or "damage/repair-needed" would be a better criteria for problematic events than "police-reported".
> The comparison isn't really like-for-like.
This is a statement of fact but based on this assumption:
> low-speed contact events that would often never show up as police-reported crashes for human drivers
Assumptions work just as well both ways. Musk and Tesla have been consistently opaque when it comes to the real numbers they base their advertising on. Given this past history of total lack of transparency and outright lies it's safe to assume that any data provided by Tesla that can't be independently verified by multiple sources is heavily skewed in Tesla's favor. Whatever safety numbers Tesla puts out you can bet your hat they're worse in reality.
oh hacker news, never change. "crashes 3x as much as human driven cars" but is that REALLY bad? who knows? pure gold
I find it interesting the Lemonade insurance just began offering a 50% discount for Tesla with FSD.
Insurance companies are known for analytics and don't survive if they use bad data. This points to your comment being correct.
Good analysis. Just over a month ago, Electrek was posted here claiming that Teslas with humans were crashing 10x more than with humans alone.
That was based on a sample size of 9 crashes. In the month following that, they've added one more crash while also increasing the miles driven per month.
The headline could just as easily be about the dramatic decline in their crash rate! Or perhaps the data is just too small to analyze like this, and Electrek authors being their usual overly dramatic selves.
All of your arguments are expounded upon in the article itself, and their conclusions still hold, based on the publicly available data.
The 3x figure in the title is based on a comparison of the Tesla reports with estimated average human driver miles without an incident, not based on police report data. The comparison with police-report data would lead to a 9x figure instead, which the article presents but quickly dismisses.
The denominator problem is made up. Tesla Robotaxi has only been launched in one location, Austin, and only since July (well, 28th June, so maybe there is a few days discrepancy?). So the crash data and the miles data can only refer to this same period. Furthermore, if the miles driven are actually based on some additional length of time, then the picture gets even worse for Tesla, as the denominator for those 9 incidents gets smaller.
The analysis indeed doesn't distinguish between the types of accidents, but this is irrelevant. The human driver estimates for miles driven without incident also don't distinguish between the types of incidents, so the comparison is still very fair (unless you believe people intentionally tried to get the Tesla cars to crash, which makes little sense).
The comparison to Waymo is also done based on incidents reported by both companies under the same reporting requirements, to the same federal agency. The crash definitions and reporting practices are already harmonized, at least to a good extent, through this.
Overall there is no way to look at this data and draw a conclusion that is significantly different from the article: Tesla is bad at autonomous driving, and has a long way to go until it can be considered safe on public roads. We should also remember that robotaxis are not even autonomous, in fact! Each car has a human safety monitor that is ready to step in and take control of the vehicle at any time to avoid incidents - so the real incident rate, if the safety monitor weren't there, would certainly be even worse than this.
I'd also mention that 5 months of data is not that small a sample size, despite you trying to make it sound so (only 9 crashes).