back
77 comments
"Something unexpected where the machine drove like a machine rather than a person..." is nonsense. There were over 1500 reports of humans driving the wrong way on Arizona roads last year. Humans drive that way all the time.
Do we think it's acceptable to target a level of driving proficiency for automation that could be featured in dashcam footage that gets uploaded to youtube because it's so extremely inept or are we aiming for a higher target that might be a middling amount of driving proficiency?
Most autonomous failures aren't failures in a way that make good video though. It's like your grandma or 16yo daughter that gets stuck at a yield for no good reason. Nobody is gonna watch that so nobody uploads it.

A robotaxi that has a "low enough to be acceptable" frequency of the above failure mode is likely to have enough occasional "full send" failure modes to make for Youtube fodder when deployed at scale even if they're comparatively rare compared to humans or the other failure type, or some other standard.

That is a very disingenuous take on the comment. We should of course target a higher level of proficiency than that, but the point is that many humans make stupid driving decisions every day. We can hold machines to a higher standard, but perfection is an unrealistic standard.
I don't generally have to entrust myself to humans doing crazy things, though. An actual person driving a taxi is familiar enough with the roads that they are unlikely to do something like that. Probably also Uber/Lyft drivers. And there is usually an option to rent a car and drive yourself, in which case any craziness the driver suffers from is at least my own fault.
Driving yourself onto tracks is natural selection. AI driving a person onto tracks is artificial selection.
> Humans drive that way all the time.

Why do we hold calculators to such high bars? Humans make calculation mistakes all the time.

Why do we hold banking software to such high bars? People forget where they put their change all the time. Etc etc.

Tens of thousands of humans kill each other every year on the roads in the US alone. So these analogies are completely bankrupt.
This is such a terribly ineffectual comparison, but okay... and I'd jump out of their car if I was a passenger and they were ignoring my pleading with them to stop.
But how many humans drove on train tracks? Yes, humans make scads of mistakes, but they don’t typically make this one.
> But how many humans drove on train tracks?

I have seen this twice in my life. One person who freaked out because they stopped in the tracks and then turned onto them, the second I still have no idea how they got there.

I think there are two really big issues with the roll out of self-driving cars that are going to be hard for us to overcome:

1. Their mistakes are going to be highly publicized, but no one is publicizing the infinite number of dumbass things human drivers do every day to compare it to.

2. They're going to make mistakes that are extremely obvious in hindsight or from a third party perspective, that most humans will say no human would have ever done. It is likely that a human has and would have made similar and worse mistakes, and makes them at a higher rate, and we will have to accept these as a reality in a complex world.

These are light rail/tram tracks, not railroad tracks. The road is the same type of road that you normally drive up, they just have train tracks embedded in the road surface, signs telling you not to drive there, and every now and then a tram drives along it.

Functionally, they're no different than bus lanes or a wide shoulder. Humans drive on them all the time, because there's no traffic on them and they can get to where they're going faster. They shouldn't, it's illegal, and they can get ticketed for it, but they do it anyway. If you load up google street view in Phoenix/Tempe/Gilbert you can see a few people driving on them.

I don't know if there are stats for this but it wouldn't surprise me if there were non zero incidents of it. Drivers that are high / drunk, mentally impaired etc. More broadly, lots of cars driven by humans collide with trains, which is the at least one of the core issues here.

EDIT: anecdotally at least for this type of ground level light rail, I've seen people drive on similar streetcar tracks (that are not shared with cars) in Toronto more than one time.

Seen this exact thing happen a handful of times myself lol (I live in Phoenix)
Spending 2 minutes on google news or youtube equipped with keywords such as "car", "train tracks" and "stuck" will show you otherwise.
I’ve seen it personally in San Jose. Guy turned left but instead of continuing onto the crossing road, he turned onto the VTA rails in the middle of the road. Then proceeded to get stuck on the concrete partition once the intersection was over, and work crews had to come out to fix the mess.
There are many pictures of cars driving in bike lanes in my city (and yes, the bike lanes are very small and well-signed).
I absolutely believe that humans would drive on train tracks. There is no shortage of terrible, insane, ignorant, and purely self-interested drivers on the road. Just look at any dashcam video compilation!

The difference, of course, is that when a human does it we just say "what an idiot!" But when a machine does it, some people say "well, obviously machines can't drive and can never drive as well as a human can," which is silly.

1500 out of a lot. It's probably more accurate to say that those 1500 didn't drive like humans either.
By definition, they did. Humans do weird, irrational things sometimes. It's part of being human.
Do they continue to drive that way if they have a passenger yelling, "we are on train tracks?" to the point that said passenger has to bail out?

(Sure, within the wide boundaries of mental health, it is not impossible.)

Bingo. Humans can make mistakes, but they can also recognize that something's gone wrong and change tactics to recover from it. Current self-driving systems can't do that effectively; they'll just keep going, even when they probably shouldn't.
>Do they continue to drive that way if they have a passenger yelling, "we are on train tracks?"

On a non-separated rail like in the video where you can just turn off at any time I can see a lot of people continuing to do it just to spite their spouse for screeching about the obvious.

Or on the other side of the coin I can see a lot of people just say nothing because it's probably fine and they'd rather not have the argument.

That would open up some very fun avenues for mischief if you could start yelling at the Waymo and pleading for it to change course
lol. I knew someone was going to come in with this reasoning for dismissing it. hn has become a parody of itself.
If the "It is difficult to get a man to understand something, when his salary depends on his not understanding it" and "notice me senpai" memes had a baby raised by an automaton nanny it would be the majority HN user.
The comments in this thread are wild.

Folks in this thread are trying to compare Waymo to human driving as some sort of expectation setting threshold. If humans can’t be perfect why should we expect machines to be?

We don’t expect humans to be perfect. When a human breaks the law we punish them. When they are sued civilly and found liable, we take their money/property.

There’s also a sense of self-preservation that guides human decision making that doesn’t guide computers.

Until we account for the agency that comes along with accountability, and the self-preservation mechanisms that keep humans from driving someone else onto a light rail track, we are making a false equivalence in saying that somehow we can’t expect machines to be as good as humans. We should expect exactly that if we’re giving them human agency but not human accountability, and while they still lack the sense of preservation of self or others.

Because we necessarily need higher standards for a self-driving system than for humans. A human failure is isolated; a machine failure is systemic.

I, as a somewhat normal driver, am not personally at much risk if some other driver decides to drive on the rails. That won't be true if I'm in a Waymo and there's nothing I can do about its bugs.

And I don't blame people who are skeptical that Waymo will be properly punished. In fact, do you suppose they were punished here?

> A human failure is isolated; a machine failure is systemic.

Some truth to this, but a machine failure can be patched for all cars. There's no effective way to patch a problem with all human drivers.

I totally agree the comment threads are definitely wild. People always assume we are against the technology. We expect more precision from machines, and that’s why they exist. Many people getting killed in car accidents is irrelevant to what we saw on the video. We expect more from them and their creators, and we do not want to see experiments on roads where we drive our kids.
The biggest issue for the wide deployment of autonomous cars is the legal liability for stuff like this, where it’s obvious that a human was unlikely to make this mistake. The robot can make human mistakes, but it can’t make robot mistakes.
If a human kills or seriously injures somebody with a car, the insurance company will pay out 6 or 7 figures.

If Waymo et al can successfully limit payouts to that, they'll be fine.

OTOH, if every death or serious injury comes with a 9 figure+ punitive damage award, they'll be in trouble.

Exactly. The bots are probably saving lives at this point just being more reliable than humans on the basics, but if they make mistakes that lead to outsized payouts, then the market won’t be insurable and will fail for legal reasons, not for technical reasons. I suspect we’ll figure it out in the long run because it’s worth it to do so, but the short run might be pretty rocky.
Many state minimum insurance limits are laughably low. Like $30,000.
As an aside, light rail that shares roadways with cars seems like such a dumb idea. More expensive and less flexible than buses, can get stuck in traffic, lots of accidents because drivers don't expect it, etc.
Leaders see the cost of a raised right of way or even just a dedicated lane and balk. Which is really dumb because after spending all the money on rail you screw up the last 1/10 by having a tram get stuck behind single occupancy cars. One person illegally parking can hold up dozens of riders. But our politicians still can exploit our bias of rail = good, bus = bad and show up for the ribbon cutting ceremony.
> But our politicians still can exploit our bias of rail = good, bus = bad

There's also a lot more campaign donation money for huge train projects from engineering and construction firms.

The expectation that a machine should not make these mistakes is reasoned, humans might make these mistakes but they will generally realise their mistake. The machine doesn't know its wrong. A human learns to drive within 20-40 hours with reasonable competency, versus the effectively millions of hours of computation that's used to train the models, Given this the expectations aren't as unrealistic as some on here claim.
Around the same time not far from there I saw a Waymo car partially blocking a lane of traffic as if it had frozen partway through turning. Wonder if they temporarily shut down all cars in that area until they could figure out what was going on.
I am glad I was not the passenger. I am blind and would not even know we are driving down tracks.
Ooh boy, that would be a good civil lawsuit. They should actually address that edge case (and related ones) before people are killed/maimed. Hopefully they have
"The society will tolerate some amount of deaths to AI vehicles"
That's the correct answer; the problem is that society may not tolerate those deaths.
>“I actually felt a little sorry for the car. It obviously made a bad decision and got itself in a difficult place,” said Andrew Maynard, an emerging and transformative technology professor at ASU.

Oof, psychopath much?

This is literally the opposite of psychopathic behavior.
Feeling empathy (for a machine) is... psychopathic?
"Waymo passenger flees after remote operator drives on Phoenix light rail tracks"

There, fixed it.