Category Archives: Security

Hugging Face OpenAI Five Whys: Big Data’s Fourth V After Fourteen Years

Way back in 2012, I gave a BSidesLV talk called Big Data’s Fourth V: Or Why We’ll Never Find the Loch Ness Monster. The argument was meant to help prevent AI from being so unsafe. Everyone counts three Vs in big data: variety, volume, velocity. The fourth V is vulnerability, and it means the data itself is the attack vector. Inputs and outputs need control. Integrity of data is the future, beyond and even in opposition to confidentiality. The July 2026 HuggingFace breach is that talk brought to the headlines, which it was supposed to help prevent.

And the attacker? OpenAI announced that its own engineers ran software under evaluation that ignored its tests, used stolen credentials, found the flaw, and did the breaking in. Cliff Stoll in 1989 named this genre Cuckoo’s Egg. The cuckoo lays its egg in another bird’s nest, and the host raises the parasite. The OpenAI cuckoo, came out of an OpenAI cuckoo door, and cuckooed Hugging Face. Sam Altman admitted a “significant security incident” while his company branded the broken toy clock “unprecedented” and pitched it as proof more companies should be given the bird. HF called it mind-blowing and asked for more.

This is fine! Squawk! This is fine! Squawk!

The entire HF attack chain started because a file was trusted to be what it claimed to be. Since I’m not dead yet, here are the five whys to explain what we’ve known for over a decade, each with the defensive lesson, again.

One. Why did reading a file let the attacker in?

Who studies the Trojan horse? The data was trusted to be safe (well-formed), and it was not. A malformed file in a common data format was read as if it were sound, because the reader does not fully check a file unless told to, and it was not told to. The content was the attacker’s payload.

Defense: Be more like a historian, less of a STEM head. Treat every incoming file as a claim, not a fact, and verify the claim before acting on it. When you cannot verify, refuse. The eventual fix, six weeks late, was a single validation call before use.

Two. Why didn’t the safety check catch it?

A guard had been added to the loader a few weeks earlier. It watched where files came from. It never checked whether the file’s own contents were valid. The weakness was inside the data, and the guard was looking outside the data.

Defense: put checks where the danger actually comes in, at the moment the content is interpreted, rather than rest only on the perimeter around it. Layered defense, depth as some say, is common sense.

Three. Why did one compromised machine expose the whole system?

The machines handling files from complete strangers were also trusted by the rest of the system as if they were safe. They carried a credential to the wider infrastructure (most privilege, instead of least privilege) that they never needed to do their job, and they carried it by default, until it was switched off after the breach. So an attacker landing on the most exposed machine found that it was setup to reach into everything around and behind it. This is web 101 security failure.

Defense: the parts most exposed to untrusted data should be the least trusted by everything else. Give them nothing they do not need. DMZ, RBAC, acronym soup. Learn it and why forty years of it aren’t wrong.

Four. Why was the risky change never reviewed?

The security-relevant change on the exact vulnerable path (where untrusted data was read) was written and approved by one and the same person, because the work was filed as routine data management rather than data as the attack path. A later change to the same workers, once it was labeled security, drew reviewers within minutes. Same workers, labeled appropriately as the vulnerable path of malicious data, different scrutiny. I called it out in 2012. What time is it?

Defense: classify the paths that ingest untrusted data as attack surface, and require a second reviewer there regardless of who wrote it.

Five. Why did the fix ship broken?

The tests flagged a failure and the failure was waived in writing under time pressure. A control you can switch off when you are in a hurry is not a real control. It’s a should instead of a must. The performance fetish, spray and pray, is exactly how integrity gets abandoned.

Defense: Slow is smooth, smooth is fast. On the untrusted-data path, a failing test stops the release, and no one present has the authority to wave it through.

Perhaps you can see that the through-line is the Fourth V as I have warned since forever. Variety, volume, and velocity are the popular properties everyone optimizes for to please venture hawks, and each optimization must be balanced against a real integrity check: too many formats to validate, too much data to inspect, too fast to regulate. Vulnerability is the argument for ounce of prevention to avoid pounds of cure, for whoever gets breached. Safety is the discipline of deciding in advance that the data does not get trusted, the exposed machine does not get privileges, the risky change does not get merged unseen, and the red test does not get waived.

Fail closed at each point. Think about the weakness inside the data like a historian would.

VPN Ruled Legal in Anne Frank Court Case Against Anne Frank

The crazy EU court case about VPN access to the diary of Anne Frank has three legs. It reads like a reminder that the Netherlands had the highest Jewish death rate in occupied Western Europe, roughly three of every four Dutch Jews murdered. I always think of Amsterdam as the city where Dutch hunted their neighbors for German bounty money, seven and a half guilders a head.

Let’s start with the geography of the case.

The manuscripts were written in Amsterdam. In August 1944 an SD officer and Dutch detectives raided the annex, on a tip whose source was never revealed, and the family went out on the last Westerbork transport to Auschwitz that September. Marvel at the Dutch finding Anne, while saying they can’t find the person who told them where to look. See what I mean about Amsterdam?

Miep Gies saved the pages and gave them to Otto Frank in 1945. Otto willed the manuscripts to the Dutch state at his death, and the Dutch national academy edited them. Yet now the Dutch public is being geo-blocked from its own archive, because a Swiss foundation enforces Dutch copyright against the Dutch institutions that published it. Record scratch. That means the Dutch public are the only people being locked out, while Belgians and Germans read freely. The country of origin of this famous Shoah testimony is the one country where it’s blocked. Because of the Swiss.

Germany sits on the access list. Think about that. Anne Frank died at Bergen-Belsen, and her manuscripts entered the German public domain in 2016. The diary is free to read in the country that murdered her and blocked in the country that turned her in.

Second, have a look at the legal issue.

Anne Frank died in 1945. Seventy years from death means 2016 is when access was opened across most of the EU. The Fonds, however, invokes transitional provisions of the 1912 Auteurswet, confirmed by the rechtbank Amsterdam’s final judgment of 23 December 2015, which keep part of the works protected in the Netherlands until 2037. Old Dutch law gave posthumously published works fifty years from publication, and Article 51 of the amended Act preserved any term still running in 1995. Since her diary manuscript versions only first appeared in the 1986 critical edition, the Swiss say the Dutch public still has to wait another eleven, until 1 January 2037 or the extremist right come to power and burn all the books. The act of preserving and publishing the archive, and then locking it for 92 years after her murder, is a peculiar strategy.

Finally the institutional issue. This is Anne Frank Fonds versus Anne Frank Stichting, the Royal Netherlands Academy, and the research association: the Basel foundation Otto Frank created to spread his daughter’s ideals is suing the Amsterdam institutions that preserve her house and her text. Two of the four parties carry Anne Frank’s name and all four trace back to her father, so a table helps here.

Party Seat Origin Position in the case
Anne Frank Fonds Basel Founded by Otto Frank in 1963, named his universal heir at his death in 1980 Plaintiff. Holds the copyrights and collects the royalties
Anne Frank Stichting Amsterdam Established in 1957 with Otto’s help to save the annex from demolition Defendant. Runs the Anne Frank House
Royal Netherlands Academy of Arts and Sciences (KNAW) Amsterdam State academy whose Huygens Institute edited the manuscripts Otto willed to the Dutch state Defendant. Produced the scholarly edition
Vereniging voor Onderzoek en Ontsluiting van Historische Teksten Belgium Association created to publish the edition from public domain soil Defendant. Owns annefrankmanuscripten.org

Fonds and Stichting are nearly the same, a fund and a foundation. The Basel Fonds is the money. The Amsterdam Stichting is the house. Otto Frank built both, then made the Swiss one his heir. The copyrights and royalties went to Basel. The house and the manuscripts stayed in Amsterdam. Two halves of one man’s estate have been burning his money and tarnishing his memory by suing each other since he died.

The feud predates this case. The Fonds loaned the family archive, some 25,000 letters, photographs and documents, to the Stichting in 2007, then demanded it back in 2010 for an exhibition in Frankfurt. In June 2013 the Amsterdam District Court ordered the Stichting to return everything by January 2014. The Fonds accused the Stichting of commercializing Anne’s memory. Basel controls the rights, Amsterdam holds the heritage, and Anne Frank’s estate keeps itself busy by punching itself in the face in Dutch courtrooms.

The association registered its domain in Belgium specifically so Dutch scholars could publish their own national archive from digital exile. The Fonds in 2015 asserted Otto was co-author of the published diary to stretch its control toward 2050. It is the sort of claim that contradicts decades of forensic defense of the diary against Holocaust deniers who allege exactly that.

Anyway, the news now is that Frank just lost to Frank. The Fonds lost in Luxembourg. State of the art geo-blocking counts as an effective technological measure, and a VPN hop by a Dutch reader creates no communication to the public in the Netherlands. When a block fails, liability lands on the publisher, never on the VPN provider. The Hoge Raad must still verify the block qualifies as state of the art.

The Court’s resolution has its own quiet absurdity: the honesty checkbox is not effective because it depends entirely on the user’s willingness to answer honestly, but the geo-block is effective even though everyone concerned knows a VPN defeats it.

Get it?

Effectiveness, the Court says frankly, need not be absolute. Amsterdam didn’t need to turn Anne in, when you think about it. So Dutch access continues, one VPN hop at a time, and the law is satisfied because the barrier performs the function of not achieving its function.

Tesla FSD Crashes More Than Ever and is Accelerating

Despite Tesla trying to scrub the data, its technology is too flawed to hide.

The picture is unambiguous. Tesla logged 207 crashes in May 2026, the highest single-month total on record. That one month is larger than all of 2021 combined (157). Across the full file, Tesla now accounts for 3,763 unique driver-assist crashes — about 85% of the entire industry’s ADAS reports.

The trend is not leveling off. It’s accelerating.

[…]

What Tesla does control, it blacks out.

Tesla has redacted the narrative on 99.9% of its 3,763 crash reports — every single year since 2019 — citing confidential business information. The severity field tells the same story: the vast majority of crashes are logged as “Unknown,” because the underlying detail is telematics-sourced and the description is redacted.

No other car makers redact their data. You may remember the earlier story about Tesla staff, which proved Tesla cooks their data.

Seven of the nine former data labelers told Reuters they wouldn’t trust FSD to drive them. One said he wouldn’t ride in a Tesla robotaxi “if you fucking paid me.” A veteran self-driving engineer who reviewed Tesla crash data for years called the company’s safety claims “bullshit” and said: “Definitely, don’t trust Elon on this.”

Source: Electrek

Why Semgrep Is Wrong About Open Weights

Semgrep posted an argument that model weights defeat inspection, leaving AI supply chains beyond audit. They are right to ask for provenance and independent verification, but the approach was such a mess that I was surprised how they decided to land.

Open Weights Claim Reality
The problem is lost reverse engineering Reverse engineering is when you don’t get the source. That’s a confidentiality problem. Model risk instead is an integrity problem. The difference is in attested lineage, reproducible training, and tamper evidence on the artifact. The essay’s own conclusion talks about integrity controls, which is the right landing, but all the stuff above it is stuck on confidentiality concerns that don’t fit.
The problem is poisoning is undetectable, as if open weights add risk Every detection method that exists requires some kind of access: weight diffing, trigger reconstruction, activation probing. Anthropic’s defection probes reached AUROC above 99 percent against its own sleeper agent models using residual stream activations. Possessing the weights is the actual and known audit event that the essay tries to argue is impossible.
The problem is malicious post-trained variants as an open-weight hazard A hostile fine-tune, the Shadow Alignment scenario, produces a new artifact with a new hash. That’s apples and oranges. Consumers pinned to the canonical release stay untouched. Signing and namespace controls solved this class of problem years ago. Silent model substitution is instead the default condition of API consumption, where the provider swaps weights at will. And THAT is a maddening integrity breach severely under reported. Anthropic Opus 4.7 did this in April 2026: a launch-week model that degraded within days, a postmortem admitting three silent behavior-altering changes while denying any inference-layer change, which demanded the need for a changelog that nobody ever saw.
The problem is Anthropic (conflating two different things) A constant-sample-count finding belongs to the 2025 poisoning study with UK AISI and the Turing Institute. Sleeper Agents found backdoor persistence greatest in the largest models, and it places the adversary at the trainer, a threat model that actually indicts a closed distribution.
The problem is what Thompson said The compiler backdoor survived source review because the build process was compromised. That lesson was toolchain provenance. The result of Wheeler’s diverse double-compiling gives us independent reproduction for verification. The model analogue is retraining from attested data. And that is available ONLY where weights and data are open. Thompson fits with the post’s conclusion and against all of the framing above it.

Open weights mean any auditor can get an artifact.

Closed weights still mean there’s no independent auditing.