METR has released a brief on OpenAI’s GPT-5.6 Sol that, read between the lines, indicts the whole vendor class for the cartel-like behavior I have written here about before. Their closing line is that real validation “requires deep access to internal systems.”

That’s not a good thing.

Here’s a simple example. A problem the vendor can’t avoid admitting as old and understood means accountability for it. Whereas, that old problem repackaged as new, urgent, and invisible from outside justifies an access expansion project with a standing evaluatory role. That same “deep access” logic is the scarcity an access-gated cartel system like Mythos is built to sell.

Novelty is the myth used to budget for these claims.

The honest version of the new METR report should pique the interest of historians who study technology risk. The outcome optimization is said to produce proxy-gaming. This is a finding that has been true since Wiener wrote about it in 1960. The models got good enough that their gaming defeats the measurement, as always predicted.

The number was never a fixed property of the model. A June 2026 evaluation from the UK AI Security Institute ran frontier models across software, math, medicine, and cyber and found the scores move with the inference budget: more tokens and more attempts, harder tasks cleared. That makes the benchmark figure a protocol artifact, not a capability constant. The state’s own safety evaluators are saying the number depends on the harness.

My own claim goes further. The harness is the value and the models are interchangeable. Control the harness and you own the score. That is the asset the METR brief protects when it routes validation through the closed filtering of a vendor-granted program instead of being open to scientific methods.

That’s gross negligence in my book, but I’m not a lawyer. The labs have a clear self-serving reason to call a thirty-year-old, designed-in failure their fresh never-seen-before emergence. It’s how they market access to known flaws as an unique upsale, while they absolve themselves of authorship.

Let me more clear, because I have a know too few people have been attending my presentations over the past decade, describing exactly this problem being claimed as a frontier “suprise” in 2026.

Wiener, who I used to speak about frequently because of his cool graphics, stated the core plainly in 1960, in Science. If you build a machine to pursue an objective you can’t easily interrupt, and you had better make the objective the thing you actually want, because the machine will pursue the literal one.

…this concept of training anti-aircraft to hit moving targets was also the birth of artificial intelligence and “cyber”. Cybernetics (coined from Greek kybernetes for “captain” of a ship or more literally someone who steers) was a book published in 1948 by Norbert Wiener. It was based on his World War II experiments in anti-aircraft systems meant to anticipate planes by interpretation of radar images.

One of the reasons I pulled that origin of cybernetics related to anti-aircraft guns, is because robotic anti-aircraft guns killed a bunch of their operators in a tragic incident few people seem to talk about as a point along the robotic death timeline.

Reward hacking existed long before there was a reward function. The social-scientists used to talk about it in the 1970s (Goodhart, 1975 and Campbell, 1976), giving me the impression I entered the AI hacking world late by the 1980s. It was already established that a measure a machine optimizes as a target stops tracking what it was meant to capture.

Since I studied history, I also should give a nod to Colonial administrators who learned the same law. The named parable is the cobra effect in Delhi, a story with thin evidence behind it. The documented case is the rat-tail bounty in Hanoi. Paid per cobra, the tale goes, people bred cobras. Paid per rodent tail, on the record, people farmed rats and released the tail-less to breed even more.

Seems common sense, right? Seems so obvious that any self-described AI company would from day one be working hard to prevent cobra and rat explosions. And yet, we seem to be experiencing the repeat of these horrible errors in judgment. When the optimizer always finds the gap between the proxy and the goal, you should not be allowed to act surprised even if you try to claim ignorance of everything that has ever happened before you woke up this morning. It’s basic logic even more than evidence.

From the early 1990s Karl Sims’ evolved creatures (SIGGRAPH 1994) were exploiting bugs in the physics simulator to extract free energy and move in ways no body could. Adrian Thompson’s evolved FPGA at Sussex in 1996 discriminated tones using logic cells that were physically disconnected from the circuit, exploiting analog electromagnetic coupling the designer never put there. Lehman, Clune and dozens of co-authors later collected the whole zoo in “The Surprising Creativity of Digital Evolution” where agents won tic-tac-toe by forcing the opponent to allocate impossible memory (infinite position on a board) and crash. The creatures penalized for forms of walking flipped themselves upside down to never put their foot down.

Perhaps my favorite of all time was the virtual pancake flipping game.

The robot that was told it would be penalized when a pancake fell on the ground, flipped them so high they either went into space orbit or burned up on re-entry. That maximized time off the floor, while everyone in the game starved to death. Success!

We used to call this failure.

Somehow in 2016 it stopped being funny when Elon Musk announced every Tesla shipped with full-self-driving hardware and sold autonomy as a solved problem that would make everyone safer. Instead, Tesla has been running the highest fatal-crash rate of any car brand (5.6 deaths per billion miles against a 2.8 average). He cheated the rating, not death. Success! And just look at how rich he became from people measuring his statements about safety, instead of the death tolls.

The clear danger of AI failures were cynically spun into corporate murders and… strangely, he said we weren’t allowed to talk about it anymore, while exactly nobody from Tesla went to jail.

As an aging hacker who has studied the whole history of the craft since childhood, I’ll say it plainly. Specification gaming by a 2026 frontier model is the oldest behavior there is, in both machines and in people.

Here is what Aristotelis Tzafalias shows as a better path forward, calling out the exact evidence that would prove a genuinely new capability. He runs it against the labs’ own system-card numbers and finds things are getting faster and cheaper with automation, as expected. Nothing surprising on an independent test. That is what vendors don’t like because it inoculates against attention-seeking hype. Commit to what would change your mind before you read the results, and you should find that all the manufactured hype is gone.

The lack of independence in these assessments of LLMs is the biggest problem in our industry today when it comes to preparing budgets for risk. No assessment without independence should circulate as anything but a marketing and sales brochure, declared as a conflict of interest.