A new think tank paper is out with a shocking, shocking I tell you, revelation that there is disinformation afoot. Mincemeat! I say old chaps, this ruse is a Haversack!
Now, before I get too snarky about this paper being disconnected from reality, I have to admit they did prove a small thing: if you ask a chatbot leading questions about made-up stories, it plays along about one time in six, and the junk sites may turn up in its citations.
That’s a finding.
I can work with that.
However, then these authors took a running leap off a cliff without a rope to claim Russia deliberately rigged websites so AI models would belch out the lies.
I mean, duh. The whole web history has been astroturfing and sock puppets, and Russian history has been militarized disinformation for over a century (copying the Americans and British), so put that all together and what do you expect? But that’s the exact problem here. Our expectations aren’t a substitution for science.
Their test can’t tell their explanation apart from a boring one: models remembered the junk from training. They never checked which it really is, so reading the paper is very disappointing.
The one in six math also is familiar. NewsGuard in May 2025 tested Australian election falsehoods with its innocent, leading and malign prompt styles and also got 16.6 percent. A different country and different topic hit that same number, which suggests the test design may be at fault.
Even more troubling is the fact that their website “evidence” of bad intentions is simply what a standard WordPress plugin does on its own. Let me explain. This report says the max-snippet:-1 directive is a strong signal of deliberate targeting.
Well, it is also found in the page source of the think tank’s own release page, and on the Euronews article that launched the report. Anyone can view the source and check. Are they not aware of their own site undermining their main claim?
And every one of the questions used came from the research team itself? Is that any different from what the authors are accusing the Russians of doing? Is this paper now not evidence of British information dissemination? The researchers typed the questions, and then just reported the answers as a public threat without proper checks. I’m not sure why.
The necessary checks are easy, and yet they skipped them. As a matter of fact, they already were published by someone else. Four researchers at Manchester and Bern (Maxim Alyukov, Mykola Makhortykh, Alexandr Voronovici and Maryna Sydorova) tested the earlier NewsGuard claim in the Harvard Kennedy School Misinformation Review last October. Their echo rate was 5 percent instead of 33. Junk citations appeared in 8 percent of answers, usually with warnings attached, and almost entirely when a prompt matched stories that appeared solely on the junk network.
They concluded the cause was data voids: models pull from junk when better coverage is thin. They also noted NewsGuard published no prompt set and counted cautious answers as failures.
Their own analysis in Al Jazeera has perhaps said it best: a study built to find disinformation has found it. Swiss reporters at NZZ got NewsGuard to confirm that its prompts were written so the chatbot only had to agree with the falsehood in the question, and NewsGuard declined to release the full prompt list. The same researchers also made the point that Margarita Simonyan cites Western research as proof that RT works, rather than citing actual proof that RT works.
So this paper says to me the think tank started out to prove a hypothesis “Russia did this” and then wrote everything down as proof. Tests that could have disproved it don’t appear to have been tried. And that means they launched an unproven conclusion.
Far worse, they are gifting the Russians a huge boost. Why? These horrible spammers’ whole existence is convincing the world their junk works. A British report saying “the junk works” is just an ad for the junk sellers.
This report was funded by whom exactly?
Will Perrin, co-architect of the Online Safety Act’s duty of care. He is funding the paper that argues platform regulation should now extend to LLMs.
Will Perrin alert!
- He managed the 2001 Communications White Paper that created Ofcom.
- His duty of care model, by his own account, underpins the UK approach to regulating online services.
- He chaired the campaign coalition that lobbied the Act into law.
- He advises the network implementing it.
He built the regulator, then he wrote its doctrine, then he ran its lobby, and now he pays for the synthetic laundering, a paper with unproven claims about Russian disinformation, to push state censorship.
Oh, and the report he funded calls for “black-lists”, in 2026. Racist language, on top of it all, in a report complaining about “bad” information spread! Such fools. The UK’s National Cyber Security Centre retired that term in 2020, and the Home Office has followed. I’ll tell you who needs a block list.
Will Perrin!
Fool me once…
