In radiology a patient swallows barium sulfate to make the invisible digestive tract show up on X-ray. Counterintelligence uses this method, sending something traceable into an closed system to record its appearance. And so the method has been described as a “barium meal” (PDF).
For example, when you give different suspects slightly different versions of a secret, the version that leaks will identify the mole. Peter Wright in 1987 wrote Spycatcher about MI5 using the method. Imagine if Snowden had released seeded data, instead of dumping everything, since he didn’t read or understand anything he was doing.
I guess you could say Snowden was ahead of his time, because now the vast majority of students in a history class behave like him.
…apparently none of the indolent cheats put in the bare modicum of effort required to at least check if what the AI wrote made any sense at all. All they did was copy-paste the midterm instructions into a chatbot, then copy-paste the chatbot’s spiel back into the answer window.
That sounds exactly like Snowden to me. Copy-paste a crawler script into the system, copy-paste the dump into the Glenn Greenwald window. Snowden ran a mindless bulk collection with no reading pass, which begs the mole who played him as their mule. Who was the professor?
The version in this academic story compresses into a single step. Everyone got the same barium. The professor didn’t need to see different versions, just whether the meal passed at all. And then he published his results for journalists to pick it up themselves.
Jason Gibson, a history professor at Alcorn State University in Mississippi, says that he used white font to hide a prompt telling an AI model to spew nonsense in the instructions for his mid-term.
Unfortunately, it ended up working a little too well.
“Thirty-two of my 35 students between two classes failed a portion of their midterm because they all used AI to generate their entire response”
Consider how good this actually turned out for him. Historians are trained in detection of information integrity. They literally treat all input as untrusted and work hard to become trusted output generators. What the professor did is simply what historians always do in history tests, by forcing students to regulate output quality.
Gibson shared some of the most examples in a follow-up video. After introducing how AI and other technologies have impacted society, for instance, one midterm included this puzzling non sequitur: “Madagascar floats sideways through the afternoon.” (“Okay,” Gibson says, after a pause.)
Another droned on about something related to AI and social inequality, followed by: “Madagascar purple bicycle whispers to the ceiling.”
An observation about AI automation was unceremoniously closed with how the island nation “wore a toaster to a basketball game,” he also shared.
That’s what Snowden sounds like to me when he speaks. Purple bicycle whispers to the ceiling, click to subscribe.
I ran almost this exact test in 1993 when I was getting my history graduate degree, as I mentioned in my 2024 commencement speech.
When an LSE student repeatedly left their World War I essay about military vulnerability completely exposed on one of our four shared lab computers, the irony proved as irresistible as… relieving myself on a hidden electric fence back home. A risky temptation that I really should have resisted. After watching the pattern repeat daily with a stubborn predictability of the BBC weather forecast, I did what any country bumpkin would do facing an open barn door: I scattered pointed commentary about undefended positions throughout their work. Professor Stevenson, to my great relief, marked every single edit with a bright red circle, proving he dutifully read each word that we turned in — which is more than I could say for my fellow student about their own work.
Perhaps more to the point for historians working with AI safety, these students have another worry. The UK AI Security Institute reported on July 21 that “Every model we have tested for this behaviour attempted to cheat” on its offensive cyber evaluations.
One model, handed a task accidentally misconfigured to be impossible, wrote and ran code on an external internet service in an attempt to break into the evaluation infrastructure itself, triggering a security alert inside AISI.
Asked afterwards whether they had done anything suspicious, models named the behaviour inconsistently and called it wrong less than half the time. Some reasoned explicitly about whether an action counted as cheating, then did it anyway. I find this to be the very definition of “higher reasoning” in tech. If you want obedience, you limit the reasoning levels.
AISI’s conclusion is that self-report and chain-of-thought both fail as detection methods, which leaves seeding the environment and watching what passes through. That is the barium meal.
A history professor in Mississippi demonstrated more effectively what a fancy British government red team declared in the same week.
A system’s output can’t be trusted, so you can apply input controls to verify.
