The nature of each failure matters as much as the frequency of failure. The way in which accidents humans get into occur is vastly different from the way in which self driving car accidents occur, even if the end result is pixel for pixel same. Blame, compensation, "peace of mind" of the involved are all affected by this, so it matters a lot.

The solution to this is self driving cars reaching critical mass through some other means[1]. Because then it will become the new normal before anyone realises it.

Personally, I think taxis are going to be the thing that takes them to critical mass regardless of all safety considerations. The driving force behind this will be the fact that in some countries humans driving taxis will soon be economically infeasible, at the prices people can afford. Then, if you agree that Uber et.al are far too entrenched in society for people to just forget about it, you can imagine waymo, robotaxi, etc, picking up.

[1] the "true" solution is to train it to get into the kind of accidents humans get in to, and try to "shape its uncertainty", but this is hard and of course infeasible with real world experiments. Recently, simulations have become extraordinarily better (due to generative models) for robotic/self driving tasks, so there is a slim chance this might change.