AI Security Researchers Getting Better at Crying Wolf About Privacy

A new Swiss LLM paper about attacking privacy demonstrates only casual pseudonymity, yet researchers want us to be very worried.

We demonstrate that LLMs fundamentally change the picture, enabling fully automated deanonymization attacks that operate on unstructured text at scale

Oooh, so scary.

Someone who posts under an anonymous handle without thinking about correlation risks is vulnerable to a motivated attacker with LLM tooling.

Yeah, that’s as real as someone wearing an orange camo vest in a forest and thinking they can’t be seen. This is not news. I get that most people don’t think enough about privacy in advance of being breached, or maybe are misled by vendors (looking at you “VPN” and WhatsApp). But it’s still just Narayanan and Shmatikov’s original point in 2008 with better automation (updated in 2019). The cost of unmasking anonymous users dropped, yet the capability has NOT fundamentally changed.

Ok, security economics time.

Their whole argument is that LLMs made attacks cheap. Well, active profile poisoning makes defense cheap too. You don’t need to maintain perfect compartmentalization anymore because you just need to inject enough contradictory micro-data that the Reason step can’t converge. A few false location signals, some deliberately inconsistent professional details, and the attacker’s 45% recall at 99% precision collapses because their confidence calibration can’t distinguish genuine matches from poisoned ones.

Boom. Their paper is toast.

Integrity breaches are everything now in security. Poisoning inverts the economics because LLMs are weak at being integrous. Their entire pipeline trusts that what people post about themselves is true, the kind of classic academic mistake that makes a good paper useless in reality.

The paper needed a Section 9 about what happens to their pipeline when even 5% of the candidate pool is actively adversarial.

This is a genuine lack of a threat model in a paper supposedly about threats. It blindly asserts a one-directional information asymmetry: the attacker reasons about the target, but the target never reasons about the attacker. Yet any real operational security practice already operates on the assumption that your adversary has a profiling pipeline. The Tor Project didn’t wait for this paper. Whistleblower protocols at serious newsrooms didn’t wait for this paper. The people who actually need pseudonymity already treat their online personas as adversarial constructions, not their natural expressions.

Their extrapolation is purely speculative. Why? Log-linear projection from 89k to 100M candidates spans three orders of magnitude beyond their data. They fit a line to four points and project it into a range where dynamics could change entirely, because denser candidate pools mean more near-matches, which could degrade the Reason step nonlinearly.

Their multi-model stack totally obscures the mechanism. They use Grok 4.1 Fast for selection, GPT-5.2 for verification, Gemini 3 for extraction, GPT-5-mini for tournament sorting. It’s LLM salad. What’s actually doing the work? If the attack works partly because GPT-5.2 has seen these HN profiles in training data, that’s a memorization attack dressed up as a reasoning capability. They admit they can’t remove spilled ink from their water, an inability to separate reasoning from memorization. That’s a huge problem for the conclusions.

A case study looks exactly like their Anthropic Interviewer results with 9 out of 33 identified, 2 wrong, 22 refusals, manually verified with acknowledged uncertainty. That is not systematic evidence.

I mean this is a paper that claims to be about threats, yet it’s not really about threats at all. It’s about defenders who don’t defend. Someone call the fire department and paramedics, Swiss researchers noticed a few people lit their cigarettes. Fire! Cancer!

Section 3.3 admits their ground truth comes from users who voluntarily linked accounts across platforms. They’re studying the sheep population that doesn’t even care if they have a sheepdog or a fence, fraudulently crying wolf as if they are infinitely unstoppable.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.