“before you can teach a robot to drive, you need to teach then to be human”, Why?

You completely missed the point of the article. Everyone already knows about the rewrite, and it offers nothing as a contribution to the discussion.

Yes, lots of things have been done with “computation”, but also there are lots of things, which are proven to be unsolvable by brute computation – the so called NP-complete problems… Just an example.
And there is another classical type of problems, which are very difficult to solve with computation – the so called “curse of dimensionality” problems. And self-driving appears too be one of them.

These cars are watched by a team of people with remote control…

Thanks for writing out in a very thought through way what I am thinkning about autonomous driving. One combination that you did not touch on, but I think could be viable, is remote driving, where a human being operates several level 4 veichles, this solves the issue of legal responsibility and to some degree the reaction to new situations. But especially in transportation it would reduce the cost of human labour.

“The mistakes they make are far less common and far less dangerous than the everyday accidents caused by texting, distracted driving, and bad driving practices that abound on our roads.”

Killed your own argument there. Someone who is texting or distracted is already not paying attention. You can’t compare a computer system that pays full attention with a person who isn’t. A human driver with his/her attention on the road beats every computer system in existence, and so it will be forever. You cannot come anywhere near the computing power of the brain with artificial contraptions. It’s just not possible. Time to drop this dumb self-driving idea and instead focus on things that actually matter.

How come Tesla still doesn’t know not to crash into sideways tractor trailer years after a Tesla fanboy’s life was sacrificed by autopilot? It’s not simple as you think it is. Taking myself as an example, I have very poor sports/ reflexes. Yet I have driven my car for nearly 40 years in east coast and west coast uner all kinds of road conditions without any accident at all. Can Model S top my performance despite having “significant better car control”? WIthout stong AI, autonomous cars will never approach safety level of a good human driver.

The only relevant metric is not some imaginary and marketing-ish levels, but who will take the financial and criminal responsibility for accidents and death. Note I make a difference between finance and criminal responsibility. Because one can make a case that some deaths from autonomous driving systems will be judged as criminal neglect and at least involuntary manslaughter. In such cases somebody will have to go to prison, not only pay the big bucks.

I also wouldn’t ignore it, even more, I think a closer look gets us to the key point of differentiation between level 4 and level 5 autonomy, as the metric is the average human driver. This includes less mindful people who drive drunk or under drug abuse. Hell yeah autonomous vehicles will soon be better than them. The next step are less trained drivers, like in the US, where you can get behind the steering wheel, starting somewhere between 14 and 16 years old. Yet further you have to compare autonomous vehicles to driver training standards in Austria and Germany, then to more experienced drivers, and I think we should absolutely not avoid thinking about racing drivers like Sebastien Loeb or Sebastien Ogier.

So, let me derive a key argument from that: my understanding of automotive safety is to have systems for the worst drivers, to be as good as and preferably even better as the best drivers. Think of stability control, emergency brake assist, etc.

To get back to your comment, I absolutely agree with you that we have to use such a metric, however, in benefit of Ben Dickson I think it would be a big mistake to pin level 5 autonomy to such a poor statistic.

The following doesn’t fit your point, but let me bring in my thoughts on the initially stated differentiation between level 4 and 5: I think that it is comparably easy to get level 4 autonomy, meaning full autonomy (level 5) in situations as freeways (autobahn). But the things I have seen in my short drivers life on highways, smaller streets, country roads or even small villages and the stupid forms of traffic accidents produced by Tesla lights big red warning lights when speaking of level 5 autonomy. To further stress the topic, I concur with many scientists and automotive engineers, when they say that level 5 autonomous cars might be a romantic dream of our generation and depending on the focus on this topic in respect to our world economy, it might take around 50 years, until we can say that vehicles are level 5 to the high standards I elaborated above.

I am not sure about US, but in most of other developed World there is a special process and requirements for insurance companies. I assume US is the same. Currently in EU, Japan, Korea… Tesla would not be able legally to sell insurance. I don’t see any indications Tesla is making steps to get into approval process in any of these makers. Moreover, in many markets you can not just put anything on the road. There are basic legal requirements for car safety and again Tesla is not starting the process – and thus will be a difficult process.

If they have to rewrite the code now, this is a very bad indication for the quality of the software development process.

Tesla will offer insurance, effectively backing their own product.

https://electrek.co/2020/07/02/elon-musk-talks-tesla-autopilot-rewrite-functionality/

Same here.
How would the system allow crossing the centre line in a British village with oncoming traffic which is part of daily life?
I have a M3SR+ with basic autopilot and in the Victorian countryside false speed limits abound causing sudden strong braking which as worrying if someone of size is following.
Vehicles almost 100m ahead having almost completely cleared your path but then delayed strong braking with similar concerns.
Literally ‘shaving’ parked vehicles and even oncoming over dimension heavy vehicles such that I simply won’t use ap under such circumstances.
Clumsy cornering and surging on TACC (done better in our Suzuki Vitara).
I’m starting to wonder if the talk is more to do with harming the ‘shorts’ by talking up the share price than actual reality

It is very simple – if the AI driver producer claims that the probability for extent X is Y, then they have to offer an insurance of 1/Y for the event X. If the average Joe insures his car paying 1000 dollars, he has to receive 1000/Y dollars. If the calculation makes ridiculous claims for very low Y and this is wrong, the insurer will go bankrupt very fast.

So basically you admit that the benchmark level has to be lowered for the AI. This by itself would be in some sense an admission of defeat.

And you reason that maybe the society will gain even from less performant AI driver. But if we start to make such global goal, maybe there are alternatives solutions instead – for example good public transport is nearly non existent in US, but abundant in many other places. I have lived in South Korea more than 10 years and never had a driving license, so I could intoxicate myself without risking anybody. Nearly the same level of public transport is available in Europe. Why would a consumer select to invest in less than perfect AI driving car and risk killing somebody unintetionally if he can simply use public transport? What would such societies with food public transport gain from a handicapped AI driver?

I like your idea. Deep learning autopilot systems should be able to bring down the probability of accidents and serious injury too.
What bothers me is that non-tech people will never trust hard data, such as “autopilot reduces accident probability to x accidents per million miles”, but rather they will look at the ugly accidents caused by it, and blame it as a flawed system.
People will not see the avoided accidents, because that will never make the news. No one can see an accident that didn’t happen.

Deep learning systems may not be as safe as a fully attentive driver but what if the combination of probability of an accident and the probability of serious injury in case of an accident can be brought down to such a low level that it is acceptable? Gone are the days when driving was a pleasure. Most now sees it as a chore that they are more than willing to give up. While there may be few cases of good drivers getting hurt because of deep learning systems there will be many more cases of inexperienced and intoxicated drivers being saved by it. Demand would drive this forward than the system being as good as an attentive driver.

Good, then who will take this risk – who will be ready to sell insurance to the self driving level 5 vehicles? Who will be responsible for the accidents and the eventual fatalities?

That said, many think that Level 5 is not really necessary and that good solid L4 is all that’s needed, and honestly I think that’s what Musk really means by what he calls L5, at least from everything I’ve heard him say. If the car can behave safely within the current context–react to surrounding traffic and stay on a recognized roadway, plus adapt to unexpected obstacles appearing in the road–and stay within a known infrastructure via geofencing, that would cover a massive majority of scenarios. I think Tesla is more right than say Waymo about their geofencing approach though: while Waymo rely on fully LIDAR mapped environments as their playground, Tesla think that a looser map like Google Maps plus solid situational awareness are all that’s needed. Waymo still have to implement the same situational awareness despite their LIDAR, coping with sudden obstacles in the path, their full 3D mapping doesn’t help with that.

The cases you cited a examples for why neural networks aren’t the answer I think are poor, because they all merely demonstrate flaws in recognizing the environment, not inherent AI issues. Not seeing the white truck against the low sun could be addressed with additional sensors–the radar that’s there already, or perhaps non-visual-spectrum cameras, or yes, LIDAR, and being able to classify the elephant as such is also not important in order to successfully avoid crashing into it. If that elephant were to move at the speed and in the direction of traffic, should the AI care that it’s an elephant? Alternatively, if a bedsheet were to be lowered into traffic from a cable above the street, would you as a human not stop anyway despite recognizing that your car would probably be ok driving through it? Why should the AI be more aggressive than that? As soon as you recognize an exception in the traffic flow, you just react to it in the most conservative and prudent way possible and that should be ok for L4. There will still be tons of edge cases, but I still think that the vast majority of them can be handled with higher level generic classification. I think better-than-human driving safety can still be achieved that way.

The current Autopilot is still at the baby stage. One of the biggest flaws in my view is its very poor to nonexistent handling of lateral approaches, vehicles veering into your lane from next to you. The side cameras seem to have huge blind spots at the B pillar on both sides, as can easily be seen on the sentry videos. You can also observe that in real life, where the car simply doesn’t react at all to vehicles right next to you coming dangerously close. I’m wondering to what extent it’s even using the ultrasonic sensors for Autopilot. Are there any at the B pillar pointing sideways?

Interesting you mentioned recognizing stop signs. I don’t think Teslas recognize stop signs. They just know where stop signs are. The reason I say this is that on a recent drive on Autopilot in my Model 3, I had to brake for a flag man displaying and regulation stop sign at a spot where a repair crew was working. Conversely, the car tells me that there’s a stop sign 500 feet ahead all the time, even when trees or a curve in the road makes the actual stop sign invisible to the car’s cameras.

I’m a new Tesla driver using the latest software update on my Model 3. Last week, I was driving on Autopilot on a city street when an all white semi pulled out of a parking lot in front of me. My car didn’t “see” it. That is, it didn’t show up on my car’s video display, and I had to do the braking myself in order to avoid a collision. So this situation of a white truck perpendicular to the travel lane is still not in the learning curve of the Tesla AI despite previous accidents and at least one driver intervention. I suspect that I’m not the only Tesla driver who has had to brake to avoid crashing into a perpendicular white truck.

In short – people who believe self driving is within reach are mislead by the growing computing power. It looks to them that we are within the range of the human brain power. However the brain is incredibly sophisticated device and has much more than speed and storage.

I am not even going close to the legal and insurance problems… They alone appear very big to me.

Musk is a great innovator and a blessing for.the humanity, but he is wrong about.self driving.

And the China example? Take any random American and plop them in a car in China and I guarantee their driving performance is going to suffer significantly, and for basically the same reason as a Tesla AI.

Look, I get the underlying point – AI is not going to be the completely the same as a human driver anytime soon, and probably not ever (IMO). It has it’s own set of pros/cons, but already shows potential for statistically better than human performance in metrics that matter (e.g. safety), and that’s what matters.

It’s like comparing humans to calculators in the 1950’s. Even now computers are not better than mathematicians at every task, but they have long since surpassed our ability to do arithmetic.

We have machines that can detect cancer, read lips, play chess and go way better than any human. It’s irrelevant if we can duplicate a jellyfish. Machines that can only do one specific thing really well exist. Self driving requires many things at the same time, but still just a limited number of independent things. The issue is the unforeseeable and the lack of causality. Although it’s unlikely that recognizing an elephant is important, but identifying a broken stop sign is. It’s at least a few more years before the long tail is addressed. We also need to consider security, such as a malicious person holding a fake 1000 mph sign, or a fake green light.

A jellyfish is a very simple organism that has about 10,000 neurons. As far as I know, AI cannot even fully achieve level 5 jellyfish. How can you possible expect to achieve level 5 driving?

And I’d even argue Tesla is also Level 3+, just paralyzed from releasing it because of the political/public perception implications of any accident caused by it. This fear would be much less if people, including articles like this, drove home the single metric that matters – safety relative to human drivers. Tesla would release features they’ve developed as soon as they’re sure that they satisfy this relevant metric, thereby saving lives.