Over the years, Tesla’s short-sighted management culture, characterized by a grossly negligent mindset to dump product into the market as quickly as possible despite known flaws (essentially, “burn toast now, scrape later”), has resulted in some troubling instances. A glaring example is evident in the recent NHTSA recall case.

Tesla, Inc. (Tesla) is recalling certain 2023 Model S, X, and Y vehicles equipped with full self-driving computer 4.0 and running a software release version 2023.44.30 through 2023.44.30.6 or 2023.44.100. Software instability may prevent the rearview camera image from displaying.

In this safety regulator’s report it’s clear how a giddy “science fiction” dream of abruptly eliminating physical mirrors to use only software for visibility turned up as a predictably stupid and unsafe implementation.

The 2023 Tesla manufacturing processes, while aiming for a sleek design, pushed 200,000 vehicles onto public roads with unsafe visibility issues due to buggy software, essentially rendering drivers blind.

“Software instability”

That phrase is a HUGE dig by the NHTSA at counter-safety culture of Tesla. The car brand often bleats and gloats about rapidly throwing problems into software development hacks, ignoring regulations, instead of using traditional multi-modal automotive safety practices.

Any guess how Tesla will address this recall? With… wait for it… just more and more and more software instability.

One of the first to report problems with the update was 2023 Tesla Model Y Long Range owner Brandon Yang, who told us he has owned his electric SUV for less than four months. Yang was driving his Tesla when he noticed the car’s active safety and driving assists weren’t working, so he arranged with a Tesla service center to have his car updated. But when the vehicle tried to update itself to build 30.8, Yang’s car got stuck downloading the update and wouldn’t shut off, repeatedly cycling through attempts that wouldn’t complete.

This occurred over a period of more than 72 hours, during which the car gradually drained its battery. Yang eventually disabled Wi-Fi to break the cycle, but the car enabled an internal LTE antenna to continue trying (and failing) to update. After much back-and-forth with the service center, Yang was told his car needed a new Autopilot computer to fix the problem—though on pickup a week later, he was told a software script had fixed it after all

While trying to sort it all out, Yang posted to Reddit for advice, and was met with numerous other owners with the same issue. {…] Tesla however has shipped bad updates before, in one case recalling a Full Self-Driving Beta update that caused cars to slam on their brakes with alarming frequency.

The ballooning technical debt that is producing amateur-level script failures in 2023 Tesla cars should be no surprise. The more they depend on “rapid patch” fixes all the time and everywhere, instead of sound engineering principles and safety regulations, the more likely this all ends for them in financial and moral bankruptcy. Tesla is digging itself into a giant predictable engineering disaster, placing everyone in and around the car in danger.

In related news:

Tesla hacked, 24 zero-days demoed at Pwn2Own Automotive 2024

TWENTY FOUR ZERO-DAYS.

It’s being reported in Australia as a hit and run.

The dashcam footage also seemingly shows the car driver allegedly leaving the scene without providing help to the injured rider.

Note the Tesla brake lights aren’t on until after the collision, suggesting another design safety failure.

Definitely not a safe car to be in or around at any time.

The crash happened just after 8 a.m. off Gulf Lab Road, near the intersection of Freeport Road and Route 910.

Police said the car, a Tesla, traveled at a high rate of speed across the parking lot of a Primanti Bros. restaurant and struck a parked car before going into the creek.

First responders found the man dead inside the car, police said.

When a car acts more like a loitering munition — an uncontrollable kamikaze missile operated in public — than being even remotely like a safe transit option, then we see yet again the design is garbage and engineers working on it are unethical.

Police have been allegedly investigating a charging station.

Related: FL Tesla Kills One, Police Unsure When

Mandiant is reporting that analysis of vCenter logs shows backdoors being installed for at least two years without detection.

While publicly reported and patched in October 2023, Mandiant has observed these crashes across multiple UNC3886 cases between late 2021 and early 2022, leaving a window of roughly a year and a half that this attacker had access to this vulnerability. Most environments where these crashes were observed had log entries preserved, but the “vmdird” core dumps themselves were removed. VMware’s default configurations keep core dumps for an indefinite amount of time on the system, suggesting the core dumps were purposely removed by the attacker in an attempt to cover their tracks.

The lines Mandiant offers for an example are pretty clear, and beg a question of alarms, as well as immutability and availability of core dumps.

2022-01-01T01:31:55.419+00:00| | I125: Notify vMon about vmdird dumping core. Pid : 1558 

2022-01-01T01:31:55.421+00:00| | I125: Successfully notified vMon.

2022-01-01T01:31:55.927+00:00| | I125: Successfully generated core file.

Dumping core is an indicator the environment is untrustworthy. VMware vCenter dumping the LDAP Database “vmdird” core screams security danger. Mandiant illustrates that after the core dump the vpxuser credentials for all ESXi hosts were sent in clear text to the attacker, which sounds very much like what Pentera was ringing alarms about back on March 29, 2022.

Another table is the table ‘vpx_host’ containing details for a user called ‘vpxuser’ and its password phrase. The vpx_host table holds a record for each managed ESXi, each containing a user called “vpxuser” and a unique password phrase. So we retrieved the password phrase, using the command:

/opt/vmware/vpostgres/current/bin/psql -d VCDB -U vc -w -c ‘SELECT user_name, password FROM vc.vpx_host;’

[…]

Once decrypted, the compromised root account vpxuser confirms complete takeover of the ESXi server and a new zero-day is born.

To be fair vpxuser is a non-root account, used in ESXi for privilege elevation. Thus the high-privilege credential leak was used by attackers to install their VirtualPIE/VirtualPITA backdoors into all the ESXi.

Anyone remember those two backdoor names from the VMware report on September 29, 2022?

Mandiant found no evidence that a vulnerability in a VMware product was exploited to gain access to ESXi during their investigations. They have named the malware artifacts as VirtualPITA (ESXi & Linux), VirtualPIE (ESXi), and VirtualGATE (Windows).

Oops, I guess? This all reads to me like Mandiant now sees better what they and VMware missed at least a year ago.

The VMware public advisories on October 24 and 25, 2023 describing VMSA-2023-0023 (Trend Micro credited for the discovery), included a claim there was no known exploitation campaign tied to the critical (9.8) severity vulnerability.

How hard did they look?

The Mandiant report on January 19, 2024 completely blew up this understanding after re-examining vCenter logs going back two years, forcing VMware to update their FAQ.

As of January 18, 2024 VMware is aware of exploitation “in the wild.”

However, note VMware record keeping on this FAQ fails to mention that significant change:

Changelog

2023-10-24, 1930 PDT (-0700): Initial publication.

2023-10-25, 11:50 PDT (-0700): Updates to improve clarity.

2023-10-31, 0930 PDT (-0700): Updates to the VMware Cloud messaging.

Meanwhile the VMware advisory text for VMSA-2023-0023 says they were notified on January 17, a day earlier.

Issue Date:
2023-10-25

Updated On:
2024-01-17

VMware has confirmed that exploitation of CVE-2023-34048 has occurred in the wild.

Log management failure in multiple ways here. This is not how we used to run security at VMware.