AI safety should not be framed as choosing between safety and capability when it’s more accurately between the false security of constrained tools and the true security of accountable humans using powerful tools wisely. We know which choice builds better software and better organizations. History tells us who wins and why. The question is whether we have the courage to choose freedom of democratic systems over the comfortable illusion of a fascist control fetish.
“Let him have it” – those three words destroyed a young man’s life in 1952 because their meaning was fatally ambiguous, as famously memorialized by Elvis Costello in his hit song “Let Him Dangle”.
Did Derek Bentley tell his friend to surrender the gun or to shoot the police officer? The dangerous ambiguity of language is what led to a tragic miscarriage of justice.
Today, we face a familiar crisis of contextualized intelligence, but this time it’s not human code that’s ambiguous, it’s the derived machine code. The recent sketch.dev outage, caused by an LLM switching “break” to “continue” during code refactor, represents something far more serious than a simple bug.
This is a small enough change in a larger code movement that we didn’t notice it during code review.
We as an industry could use better tooling on this front. Git will detect move-and-change at the file level, but not at the patch hunk level, even for pretty large hunks. (To be fair, there are API challenges.)
It’s very easy to miss important changes in a sea of green and red that’s otherwise mostly identical. That’s why we have diffs in the first place.
This kind of error has bitten me before, far before LLMs were around. But this problem is exacerbated by LLM coding agents. A human doing this refactor would select the original text, cut it, move to the new file, and paste it. Any changes after that would be intentional.
LLM coding agents work by writing patches. That means that to move code, they write two patches, a deletion and an insertion. This leaves room for transcription errors.
This is another glaring example of an old category of systemic failure that has been mostly ignored, at least outside nation-state intelligence operations: integrity breaches.
The real problem isn’t the AI because it’s the commercial sector’s abandonment of human accountability in development processes. The luxury of the common person’s bad intelligence is evaporating rapidly in the market, as debts from ignorance rise rapidly due to automation.
The False Security of Technical Controls
When sketch.dev’s team responded to their AI-induced outage by adding “clipboard support to force byte-for-byte copying,” they made the classic mistake of treating a human process problem with a short-sighted technical band-aid. Imagine if the NSA reacted to a signals gathering failure by moving agents into your house.
The Stasi at work in a mobile observation unit. Source: DW. “BArch, MfS, HA II, Nr. 40000, S. 20, Bild 2”
This is like responding to a car accident by lowering all speed limits to 5 mph. Yes, certain risks can be reduced by heavily taxing all movements, but it also defeats the entire purpose of having movement highly automated.
As the battle-weary Eisenhower who called for “confederation of mutual trust and respect” also warned us:
If you want total security, go to prison. There you’re fed, clothed, given medical care and so on. The only thing lacking… is freedom.
Constraining AI to byte-perfect transcription isn’t security. It’s not, it really isn’t. It’s surrendering the very capabilities that make AI valuable in the first place, lowering security and productivity with a loss-loss outcome.
My father always used to tell me a ship is safe in harbor, but that’s not why we build ships. When I sailed across the Pacific, every day a survival lesson, I knew exactly what he meant. We build AI coding tools to intelligently navigate the vast ocean of software complexity, not to sit safely docked at the pier in our pressed pink shorts partying to the saccharin yacht rock of find-and-replace operations.
Turkey Red and Madder dyes were used for uniforms, from railway coveralls to navy and military gear, as a low-cost method to obscure evidence of hard labor. New England elites (“Nantucket Reds”) ironically adapted them to be a carefully cultivated symbol of power. The practical application in hard labor inverted to a subtle marker of largess, American racism of a privileged caste.
The Accountability Vacuum
The real issue revealed by the sketch.dev incident isn’t that the AI made an interpretation – it’s that no human took responsibility for that interpretation.
The code was reviewed by a human, merged by a human, and deployed by a human. At each step, there was an opportunity for someone to own the decision and catch the error.
Instead, we’re creating systems where humans abdicate responsibility to AI, then blame the AI when things go wrong.
This is unethical and exactly backwards.
Consider what actually happened:
An AI made a reasonable interpretation of ambiguous intent
A human reviewer glanced at a large diff and missed a critical change
The deployment process treated AI-generated code as equivalent to human-written code
When problems arose, the response was to constrain the AI rather than improve human oversight
The Pattern We Should Recognize
Privacy breaches follow predictable patterns not because systems lack technical controls, but because organizations lack accountability structures. A firewall that doesn’t “deny all” by default isn’t a technical failure, because we know all too well (e.g. codified in privacy breach laws) it’s organizational failure. Someone made the decision to configure it that way, and someone else failed to audit that very human decision.
The same is true for AI integrity breaches. They’re not inevitable technical failures because they’re predictable organizational failures. When we treat AI output as detached magic that humans can’t be expected to understand or verify, we create exactly the conditions for catastrophic mistakes.
Remember the phrase guns don’t kill people?
The Intelligence Partnership Model
The solution isn’t to lobotomize our AI tools into ASS (Artificially Stupid Systems) it’s to establish clear accountability for their use. This means:
Human ownership of AI decisions: Every AI-generated code change should have a named human who vouches for its correctness and takes responsibility for its consequences.
Graduated trust models: AI suggestions for trivial changes (formatting, variable renaming) can have lighter review than AI suggestions for logic changes (control flow, error handling).
Explicit verification requirements: Critical code paths should require human verification of AI changes, not just human approval of diffs.
Learning from errors: When AI makes mistakes, the focus should be on improving human oversight processes, not constraining AI capabilities.
Clear escalation paths: When humans don’t understand what AI is doing, there should be clear processes for getting help or rejecting the change entirely.
And none of this is novel, or innovative. This comes from a century of state-run intelligence operations within democratic societies winning wars against fascism. Study the history of disinformation and deception in warfare long enough and you’re condemned to see the mistakes being repeated today.
The Table Stakes
Here’s what’s really at stake: If we respond to AI integrity breaches by constraining AI systems to simple, “safe” operations, we’ll lose the transformative potential of AI-assisted development. We’ll end up with expensive autocomplete tools instead of genuine coding partners.
But if we maintain AI capabilities while building proper accountability structures, we can have both safety and progress. The sketch.dev team should have responded by improving their code review process, not by constraining their AI to byte-perfect copying.
Let Them Have Freedom
Derek Bentley died because the legal system failed to account for human responsibility in ambiguous situations. The judge, jury, and Home Secretary all had opportunities to recognize the ambiguity and choose mercy over rigid application of rules. Instead, they abdicated moral responsibility to legal mechanism.
We’re making the same mistake with AI systems. When an AI makes an ambiguous interpretation, the answer isn’t to eliminate ambiguity through technical constraints when it’s to ensure humans take responsibility for resolving that ambiguity appropriately.
The phrase “let him have it” was dangerous because it placed a life-or-death decision in the hands of someone without proper judgment or accountability. Today, we’re placing system-critical decisions in the hands of AI without proper human judgment or accountability.
We shouldn’t accept the kind of world where we eliminate ambiguity, as if a world without art could even exist, so let’s ensure someone competent and accountable can be authorized to interpret it correctly.
Real Security of Ike
True security comes from having humans who understand their tools, take ownership of their decisions, and learn from their mistakes. It doesn’t come from building technical cages that prevent those tools from being useful.
AI integrity breaches will continue until we accept that the problem is humans who abdicate their responsibility to understand and verify what is happening under their authority. The sketch.dev incident should be a wake-up call for better human processes, more ethics, not an excuse for replacing legs with pegs.
A ship may be safe in harbor, but we build ships to sail. Let’s build AI systems that can navigate the complexity of real software development, and let’s build human processes to navigate the complexity of working with those systems responsibly… like it’s 1925 again.
On a Sunday in South Africa’s Limpopo Province, the most predictable thing in the world happened. A Cape buffalo, faced with a predator trying to kill it, successfully defended itself.
The headlines called it a “tragic accident” and an “unprovoked attack.” But there’s nothing unprovoked about defending against someone who traveled 8,000 miles specifically to kill.
The Setup
Picture this: A Texas flamboyant Bible and bullets man, pockets stuffed with dollars, jets off to kill one of the world’s most beautiful animals. He layers himself with insulation, the proof of what he believes is his fabricated dominion over all creatures.
The buffalo brings evolution: heft, speed, and a pure uncompromising will to live.
Guess which one prevailed?
The Great Equalizer
For all the talk of God’s image, superior intellect and technology, we’re still just idiots walking around on a planet where physics doesn’t care about prayers.
The buffalo didn’t know it was supposed to lose so someone could claim a participation trophy. It didn’t get the memo about special human exceptionalism. It just knew this dude was trying to kill it, and responded the way four million years of natural selection taught it to defend against any predator.
No theology. No technology. Just the simple, ancient logic of real survival.
The Humble Truth
There’s something deeply humbling about this story that goes beyond one person’s tragic preventable death.
It’s the reminder that for all our remote firepower, for all our certainty about our place in the natural order, we’re still just animals sharing space with other animals who didn’t agree to be turned into trophies.
The buffalo wasn’t making a statement. It wasn’t insecure and evening some score. It was just an individual creature that wanted to exercise and keep existing, using the only tools it had against someone who wanted to stop that from happening.
And sometimes, reality wins over fantasy.
The Real Tragedy
The tragedy isn’t that a “danger” killed someone. The tragedy is that we’ve built entire industries around the idea that other living beings exist primarily for unsustainable extraction and entertainment, and we’re shocked when they can still decline to participate.
Every year, people pay enormous sums to travel to end lives that were just trying to continue. We call it sport. We call it tradition. We call it connecting with nature. Lumumba and Mondelane called it American assassination. Remember them?
The buffalo calls it daytime.
What the Buffalo Knew
The buffalo knew something we often forget: that life wants to keep living, and it doesn’t matter how much money you have or what you believe about your rightful place in the world. When push comes to shove – literally, in the case of Oceangate too – the universe still operates on very simple principles.
Force meets force. Mass times velocity equals impact. The will to survive doesn’t negotiate.
No amount of faith changes physics. No amount of technology changes the fact that other animals didn’t sign up to be our victims. And sometimes, just sometimes, the universe reminds us that we’re not actually in charge.
The buffalo went back to being a buffalo. The headlines swiveled into calling the defender the attacker, a danger instead of a survivor.
But for one brief moment on a Sunday in Limpopo, the natural world got to be exactly what it’s always been: unimpressed by our assumptions and indifferent to our plans.
And maybe, if we’re able to put down the Bible and be honest, that’s not tragic at all.
A centrally planned strategic command operation is sending “paper plate” Tesla cars to blanket American streets.
In early July, people living in Signal Hill began noticing fleets of Teslas — each of them bearing temporary license plates — taking up valuable parking spots on their streets. “It seemed like they just appeared overnight,” says Bella M., a resident of the area who preferred not to share her last name.
While she says the number of parked cars fluctuates regularly, Bella once “probably counted 24 total Teslas with paper plates scattered throughout the neighborhood” in a single day. Earlier this week, SFGATE saw at least five Teslas with paper plates parked in a two-block radius of Signal Hill. The Teslas mostly appear to be older Model Y’s, although some are newer.
We know it’s centrally planned and strategic because it’s gaming the laws.
Parking enforcement seems to have taken notice of the Teslas flecking the residential streets of Signal Hill. Some cars have been given warning stickers from local law enforcement, stating that, by law, they may not park consecutively for over 72 hours in the same spot or risk being towed. According to Bella, the cars “seem to move fairly regularly to avoid being ticketed.”
Not exactly subtle. American neighborhoods are finding their streets clogged with rapidly devalued Tesla ghost fleets. Source: SFGATE
Court testimony from Benavides v. Tesla I have reviewed has been damning. It’s clear why Tesla has paid tens of millions to settle and prevent truth reaching the public for the past decade.
Tesla’s cynical deployment of deeply flawed Autopilot technology to public roads represents a clear violation of safety principles established over more than a century.
Tesla didn’t just make mistakes—they systematically violated hundreds of years of established safety principles while lying about their technology’s capabilities. Rather than pioneering new approaches to safety, Tesla deliberately ignored basic methodologies that other industries developed specifically to prevent the kind of deaths and injuries that Tesla Autopilot has caused.
This analysis reveals that Tesla knowingly deployed experimental technology while making false safety claims, attacking critics, and concealing evidence – following the same playbook used by tobacco companies, asbestos manufacturers, and other industries that prioritized profits over human lives.
Source: My presentation at MindTheSec 2021
The company violated not just recent automotive safety standards, but fundamental principles of engineering ethics established in 1914, philosophical frameworks dating to Kant’s 1785 categorical imperative, and safety approaches proven successful in aviation, nuclear power, and pharmaceutical industries.
PART A: Which historical safety principles did Tesla violate? Let us count the ways.
1) A century of established doctrine in the precautionary principle
Caution is required when evidence suggests potential harm, even without complete scientific certainty. These deep historical roots are what Tesla completely ignored. First codified in environmental law as Germany’s “Vorsorgeprinzip” in the early 1970s, the principle was formally established internationally through the 1992 Rio Declaration:
Where there are threats of serious or irreversible damage, lack of full scientific certainty shall not be used as a reason for postponing cost-effective measures.
Tesla violated the principle by deploying Autopilot despite acknowledging significant limitations.
Court testimony revealed that Tesla had no safety data to support their life-saving claims before March 2018, yet continued aggressive marketing. Expert witness Dr. Mendel Singer testified that Tesla’s Vehicle Safety Report—their primary public safety justification—had “no math and no science behind” it.
NO MATH.
NO SCIENCE.
The snake oil of Tesla directly contradicts the precautionary principle’s requirement for conservative action when facing potential catastrophic consequences.
The philosophical foundation comes from Hans Jonas’s “The Imperative of Responsibility” (1984), which reformulated Kant’s categorical imperative for the technological age:
Act so that the effects of your actions are compatible with the permanence of genuine human life on Earth.
Tesla’s approach of unqualified customers on public roads as testing grounds for experimental technology clearly and directly violates the principle.
2) Engineering ethics codes: Professional obligations established 1912-1914
Tesla’s Autopilot deployment violates the fundamental principle established by every major engineering ethics code over a century ago:
Hold paramount the safety, health, and welfare of the public.
These codes emerged directly from catastrophic failures including bridge collapses (Tay Bridge 1879, Quebec Bridge 1907) and boiler explosions (Grover Shoe Factory 1905) that demonstrated the need for professional accountability beyond commercial interests.
Source: My 2015 presentation to computer science graduate students about the dangers ahead from Tesla abuse of AI
The American Society of Civil Engineers (ASCE) code of 1914 specifically required engineers to “present consequences clearly when judgment is overruled where public safety may be endangered.”
Tesla violated this by continuing operations despite NTSB findings that Autopilot had fundamental design flaws. Court testimony revealed the extent of Tesla’s knowledge: expert witness testified that in the fatal crash:
…the Autopilot system detected a pedestrian 140 feet away and classified it correctly, but ‘never warned the driver’ and ‘never braked.’ Instead, it simply ‘turned off Autopilot’ and ‘gave up control’ just 1.3 seconds before impact.
NEVER WARNED THE DRIVER AND SIMPLY TURNED OFF.
Tesla’s diabolical and deadly approach mirrors the Ford Pinto case (1970-1980), where executives knew from dozens of crash tests that rear-end collisions would rupture the fuel system, yet proceeded without safety measures because solutions cost $1-$11 per vehicle.
Tesla similarly knew of Autopilot limitations but chose deployment speed over comprehensive safety validation. Court testimony exposed the company’s knowledge: they knew drivers were “ignoring steering wheel warnings ‘6, 10, plus times an hour'” yet continued marketing the system as safe.
Additionally, the system could “detect imminent crashes for seconds but was programmed to simply ‘abort’ rather than brake.” With court findings showing “reasonable evidence” that Tesla knew Autopilot was defective, the parallel to Ford’s cost-benefit calculation over safety is exact.
ABORT RATHER THAN BRAKE WHEN IMMINENT CRASH DETECTED.
Source: My 2016 BSidesLV keynote presentation comparing Tesla Autopilot to the Titanic
3) Duty of care: Legal framework established 1916
Tesla violated the legal principle of “duty of care” established in the landmark MacPherson v. Buick Motor Co. (1916) case, where Judge Benjamin Cardozo ruled that manufacturers owe safety obligations to end users regardless of direct contractual relationships. The standard requires that if a product’s “nature is such that it is reasonably certain to place life and limb in peril when negligently made, it is then a thing of danger.”
Autonomous driving systems clearly meet this “thing of danger” standard, yet Tesla failed to implement adequate safeguards despite knowing the technology was incomplete. Court testimony revealed Tesla’s deliberate concealment: expert witnesses described receiving critical crash data from Tesla that had been systematically degraded: “videos with resolution ‘reduced, making it hard to read,'” “text files converted to unsearchable PDF images,” and “critical log data with information ‘cut off’ and ‘missing important information.'” As one expert testified:
This is just one example of data I received from Tesla where effort had been placed in making it hard to read and hard to use.
The company’s legal team ironically argued in court that Musk’s safety claims were “mere puffing” that “no reasonable investor would rely on” effectively admitting all the claims were known false while publicly maintaining them as true.
PART B: Philosophical and ethical frameworks Tesla systematically violated
Informed consent: Kantian foundations ignored
Tesla’s deployment fundamentally violated the principle of informed consent, rooted in Immanuel Kant’s Formula of Humanity (1785): never treat people “as a means only but always as an end in itself.” Informed consent requires voluntary, informed, and capacitated agreement to participation in experimental activities.
Tesla failed on all three dimensions. Users were not adequately informed that they were participating in beta testing of experimental software. Despite owner’s manual warnings, Tesla’s marketing contradicted these warnings. Court testimony revealed Musk’s grandiose 2016 claims captured on video:
The Model S and Model X at this point can drive autonomously with greater safety than a person… I really would consider autonomous driving to be basically a solved problem.
Yet the contradictory messaging between legal warnings and public claims prevented genuine informed consent, as users received fundamentally conflicting information about the technology’s capabilities.
The company treated customers as means to an end – using them to collect driving data and test software – rather than respecting their autonomy as rational agents capable of making informed decisions about risk.
Utilitarian vs. deontological ethics: Violating both frameworks
Tesla’s approach fails under both major ethical frameworks. From a utilitarian perspective (maximizing overall welfare), Tesla’s false safety claims likely increased rather than decreased overall harm by encouraging risky behavior and preventing industry-wide safety improvements through data hoarding.
From a deontological perspective (duty-based ethics rooted in Kant’s categorical imperative), Tesla violated absolute duties including:
Duty of truthfulness: Making false safety claims
Duty of care: Deploying inadequately tested technology
Duty of transparency: Concealing crash data from researchers and public
And for those who actually care about EV ever reaching scale, Tesla’s behavior fails the universalizability test – if all companies deployed deeply flawed experimental safety systems with false claims and no transparency, the consequences would be catastrophic. We don’t have to speculate, given the high death toll of Tesla relative to all other car companies combined.
Epistemic responsibility: Systematic misrepresentation of knowledge
Lorraine Code’s concept of epistemic responsibility requires organizations to accurately represent what is known versus uncertain. Tesla systematically violated this by:
Claiming certainty where none existed: as already stated, Tesla generated pure propaganda as expert Dr. Singer testified that “there is no math, and there is no science behind Tesla’s Vehicle Safety Report.” Despite this, Tesla used the fake report to claim their vehicles were definitively safer.
Concealing uncertainty: Tesla knew about significant limitations but emphasized confidence in marketing. They knew the system would “abort” rather than brake when detecting crashes and that drivers ignored warnings repeatedly, yet continued aggressive marketing claims.
Blocking knowledge advancement: Unlike other industries that share safety data, Tesla actively fights data disclosure.
Systematic data degradation: “When Tesla took a video and put this text on top of it, it didn’t look like this. Before I received it, the resolution of this video was reduced, making it hard to read.” The expert noted: “In my line of work, we always want to maintain the best quality evidence we can. Someone didn’t do that here.”
PART C: Let’s talk about parallels in a history of American corporate misconduct
Tesla’s Autopilot strategy follows the exact playbook used by industries that caused massive preventable harm through decades of deception.
Grandiose safety claims without supporting data
Tobacco industry pattern (1950s-1990s): Companies made broad safety claims while internally acknowledging dangers. Philip Morris president claimed in 1971 that pregnant women smoking produced “smaller but just as healthy” babies while companies internally knew about severe risks.Ronald Reagan was the face of cynical campaigns to spread cancerous products, leading to immense suffering and early death for at least 16 million Americans.
Asbestos industry (1920s-1980s): Johns Manville knew by 1933 that asbestos caused lung disease but Dr. Anthony Lanza advised against telling sick workers to avoid legal liability. The company found 87% of workers with 15+ years exposure showed disease signs but continued operations.
Pharmaceutical parallel: Merck’s Vioxx was marketed as safer than alternatives while internal studies from 2000 showed 400% increased heart attack risk, leading to an estimated 38,000 deaths.
Tesla parallels: Court testimony revealed Musk’s grandiose claims captured on video from 2016: “The Model S and Model X at this point can drive autonomously with greater safety than a person” and “I really would consider autonomous driving to be basically a solved problem.” He predicted full autonomy within two years. Yet Tesla privately had no safety data to support these claims, and expert testimony says their primary safety justification had “no math and no science behind” it.
Attacking critics rather than addressing safety concerns
Tesla follows the historical pattern of discrediting whistleblowers rather than investigating concerns. NTSB removed Tesla as a party to crash investigations due to inappropriate public statements, with Musk dismissing NTSB as merely “an advisory body.”
This mirrors asbestos industry tactics where companies convinced medical journals to delay publication of negative health effects and used legal intimidation against researchers raising concerns.
Evidence concealment and destruction
Tesla’s approach to data transparency parallels Arthur Andersen’s systematic document destruction in the Enron case, where “tons of paper documents” were shredded after investigations began. Tesla abuses NHTSA’s confidential policies to redact most crash-related data and is currently fighting The Washington Post’s lawsuit to disclose crash information. Court testimony revealed systematic evidence degradation: one expert described receiving “4,000 page documents that aren’t searchable” after Tesla converted them from text files to unsearchable PDF images. Critical data was systematically damaged:
The data I received from Tesla is missing important information. The data I received has been modified so that I cannot use it in reconstructing this accident.
The expert noted the pattern:
This is just one example of data I received from Tesla where effort had been placed in making it hard to read and hard to use.
Tesla received crash data “while dust was still in the air” then denied having it for years.
Johns Manville similarly blocked publication of studies for four years and “likely altered data” before release, knowing that destroyed evidence could not be recovered.
Tesla management undermined safety standards by ignoring all of them. Let’s count the ways again.
Aviation developed rigorous safety protocols specifically to prevent the kind of accidents Tesla’s approach enables. FAA regulations require catastrophic failure conditions to be “Extremely Improbable” (less than 1 × 10⁻⁹ per flight hour) with no single failure resulting in catastrophic consequences.
Tesla violated these principles by:
Releasing experimental technology without comprehensive certification: Court testimony revealed that Tesla deployed systems that would “abort” rather than brake when detecting imminent crashes
Implementing single points of failure: The system “never warned the driver” and “never braked” when it detected a pedestrian, instead simply “turning off Autopilot” and giving “up control”
Using customers as test subjects: Expert testimony showed Tesla knew drivers were “ignoring steering wheel warnings ‘6, 10, plus times an hour'” yet continued deployment rather than completing controlled testing phases
Aviation’s conservative approach requires demonstration of safety before deployment: Tesla did the opposite – deploying first and hoping to achieve safety through iteration.
2) Nuclear industry: Defense in depth ignored
Nuclear safety uses “defense in depth” with five independent layers of protection, each capable of preventing accidents. Tesla’s approach lacked multiple independent safety layers, relying primarily on software with limited hardware redundancy.
The nuclear industry’s conservative decision-making culture contrasts sharply with Tesla’s “move fast and break things” Silicon Valley approach. Nuclear requires demonstration of safety before operation; Tesla used public roads as testing grounds.
Tesla essentially skipped the equivalent of Phase I-III clinical trials, deploying beta software directly to consumers without proper safety validation. The pharmaceutical industry requires:
Phase I: Safety testing in small groups
Phase II: Efficacy testing in hundreds of subjects
Phase III: Large-scale testing in thousands of subjects
Tesla avoided independent safety review and failed to implement adequate post-market surveillance for adverse events. Court testimony revealed they knew about systematic problems—drivers “ignoring steering wheel warnings ‘6, 10, plus times an hour'” and systems that would “abort” rather than brake when detecting crashes—yet continued deployment without addressing these fundamental safety issues.
4) Transit Fail-safe vs. fail-deadly: Engineering principles ignored
Traditional automotive systems were fail-safe – when components failed, human drivers provided backup. Tesla implemented fail-deadly design where software failures could result in crashes without adequate backup systems. Court testimony revealed the deadly consequences: when the system detected a pedestrian “140 feet away” and “classified it correctly,” it “never warned the driver” and “never braked.” Instead, it “turned off Autopilot” and “gave up control just 1.3 seconds before impact.”
Safety-critical systems require fail-operational design through diverse redundancy. Tesla’s approach lacked the multiple independent backup systems required for safety-critical autonomous operation, as demonstrated by this fatal failure mode where detection did not lead to any protective action.
Technology deployment philosophy violations
Tesla’s approach embodies what Evgeny Morozov calls “technological solutionism” – the mistaken belief that complex problems can be solved through engineering without considering social, ethical, and safety dimensions. This represents exactly the kind of technological hubris that philosophers from ancient Greece to Hans Jonas have warned against.
The deployment violates Jonas’s imperative of responsibility by prioritizing innovation speed over careful consideration of consequences for future generations. Tesla used public roads as testing grounds without adequate consideration of the precautionary principle that uncertain but potentially catastrophic risks require conservative approaches.
The historical pattern: Corporate accountability delayed but inevitable
Every industry examined – tobacco, asbestos, pharmaceuticals – eventually faced massive legal liability and regulatory intervention. Tobacco companies paid $206 billion in the Master Settlement Agreement. Johns Manville filed bankruptcy and established a $2.5 billion trust fund for victims. Merck faced thousands of lawsuits over Vioxx deaths.
The outcome is clear: companies that prioritize profits over safety while making false claims and attacking critics eventually face accountability – but only after causing preventable deaths and injuries that transparent, conservative safety approaches could have prevented.
Conclusion: Tesla has been callously ignoring over 100 years of basic lessons
Tesla’s Autopilot deployment represents a systematic violation of safety principles established over more than a century of engineering practice, philosophical development, and regulatory evolution. The company ignored:
Engineering ethics codes established 1912-1914 requiring public safety primacy
Legal duty of care framework established 1916 requiring manufacturer responsibility
Philosophical principles of informed consent rooted in Kantian ethics
Precautionary principle established internationally 1992 requiring caution despite uncertainty
Proven safety methodologies from aviation, nuclear, and pharmaceutical industries
Rather than learning from historical corporate disasters, Tesla followed the same playbook that led to massive preventable harm in tobacco, asbestos, and pharmaceutical industries. Court testimony documented the false safety claims (Vehicle Safety Report with “no math and no science”), evidence concealment (systematic data degradation where “effort had been placed in making it hard to read and hard to use”), and moral positioning (claiming critics “kill people”) that mirror patterns consistently resulting in corporate accountability.
Tesla had access to over a century of established safety principles and historical lessons about the consequences of violating them. The company’s choice to ignore this framework represents not innovation, but systematic rejection of hard-won knowledge. Court testimony reveals Tesla knew their system would “detect imminent crashes for seconds but was programmed to simply ‘abort’ rather than brake” and that drivers “ignored steering wheel warnings ‘6, 10, plus times an hour,'” yet they continued aggressive deployment and marketing claims about superior safety.
The historical record suggests that the Tesla management approach, like the awful predecessors, ultimately has to result in regulatory intervention and legal accountability. And this can not come soon enough to protect the market from fraud, given how Tesla is causing preventable harm that established safety principles were specifically designed to prevent.
Nearly half of the participants in the latest Electric Vehicle Intelligence Report said they did not trust Tesla, while more than a third who said they had a negative perception. The company also had the lowest perceived safety rating of any major EV manufacturer, following several high-profile accidents.
a blog about the poetry of information security, since 1995
