Burn it With Fire

The original

The xkcd version from a few years back

A systems engineer who works on self-driving cars was being driven home by her boyfriend's dad in a Tesla.

She requested that autopilot not be used, and when he put on the driver assist, she demanded to be let off………  immediately ……… in the middle of the street if need by.

She requested this loudly, and dropped and swore profusely, and was wondering if she was being unreasonable.

She was not.

She followed up on her post explaining why she found Tesla's self driving is so profoundly alarming:

(disclaimer, this is my personal opinion and is not a statement on behalf of my current or past employers. This should not be taken to indicate any current or past employer's position.)

A number of companies are working on self driving tech. I have, and would, get into a number of their vehicles. Just not Teslas.

1. Many of the competing companies are at a similar level of technology development. However every other company is either... manning every vehicle with a highly trained safety driver / monitoring engineer pair who know the system inside out and is trained to avert emergencies, and is extremely focused. They usually take short (like 15 minute) shifts because it's hard to keep that extreme focus. Or..
   
   
2. Operating in a very very severely limited domain (in terms of location, weather, etc..) with 1:1 remote monitoring with a highly trained person monitoring every vehicle and able to intervene in a moment. These people also take short shifts. Or...
   
   
3. Operating away from people (I.E. hauling vehicles for mining operations, which do not use public roads and where the mine roads are laid out so no people are permitted on the autonomous hauling routes.

But tesla does not do any of that. They don't have trained safety drivers, live remote monitoring, or constrained operational domains. They don't limit how long a single person can operate the vehicle, despite widely accepted evidence on how focus degrades over time.

They have similar (or arguably worse) self driving technology to other companies. But while the other 30 or so companies in this space are being responsible for the safety of their vehicles with live monitoring, and trained engineer intervention, Tesla is just letting random consumers take on that role (and liability)

It's not a good place to be, as a consumer, to be doing the same dangerous job that other people are being paid 6 figures to do. Without training. For free. Hell, worse than free, you're paying to take on a corporation's liability.

This allows them to avoid paying for some of the usual research and development costs like safety drivers and live monitoring, and it also offloads a lot of liability of testing a very very new and unreliable technology from them as a corporation onto their consumers.

It also helps them appear "ahead" in technology development, while honestly under the hood, I'd say they're behind and simply appearing ahead because they're bypassing safety practices and engineering rigor that other companies are following.

For example, every time a driver claims FSD or Autopilot error, and Tesla claims driver error, that is a shifting of liability that the other companies in this space would not do. All the other companies take the liability of their technology failing on themselves.

Tesla is pushing liability onto the end-user and their insurance policies which are generally unequipped to handle claims regarding AV malfunction, and publicly denying any accusations of malfunction. However other companies developing similar tech are purchasing insurance themselves at very high rates, and being transparent with their insurers about the risks they're undertaking.

I don't see that as ethical, as they're pushing the danger and accident liability onto the consumer, at a very early stage of technology development. I also don't see that as safe for the consumer.

It's a joke in my field that if you pay for FSD, you're paying to be a crash test dummy...

Even if Elon were not an evil right wing asshole, his car company is profoundly anti-safety and anti-consumer, so buying a Tesla seems to be to be a fool's errand.

It's interesting that this systems engineer had exactly the same response as one gets from software folks over internet voting, as is so entertainingly shown by Randall Munroe.