The cURL repo maintainers aren’t mincing words about the failure of Mythos match the hype. Daniel Stenberg calls it a lot of hot air:
Stenberg wrote that the report “felt like nothing,” and that feeling was further validated by a review of Mythos’ findings.
Nothing. Mythos felt like nothing.
And here’s the real kicker:
“Once my curl security team fellows and I had poked on this short list for a number of hours and dug into the details, we had trimmed the list down and were left with one confirmed vulnerability,” Stenberg said, bringing us back to the aforementioned number.
As for the other four, three turned out to be false positives that pointed out cURL shortcomings already noted in API documentation, while the team deemed the fourth to be just a simple bug.
BOOM. Mythos didn’t even retrieve the docs in its walks of the codebase!
That is yet more validation that a human designed harness is the only real threat, and NOT the model. I’ve said it repeatedly on this blog and nothing so far has proven Mythos is something we haven’t seen before. Stenberg is at the same place so many other experts have arrived:
“I see no evidence that this setup finds issues to any particular higher or more advanced degree than the other tools have done before Mythos.”
Exactly. And yet? ZOMG the FEAR. It burns.
What is really wrong with this survey is that the security industry needs to be focusing on Post Quantum preparedness instead of AI-vendor hype about AI.
Anthropic has made the world significantly less prepared by blowing hundreds of millions into trying to scare people into a billionare-run cartel of disinformation.
As lead developer of curl I was offered access to the magic model and I graciously accepted the offer. Sure, I’d like to see what it can find in curl.
I signed the contract for getting access, but then nothing happened. Weeks went past and I was told there was a hiccup somewhere and access was delayed.
Eventually, I was instead offered that someone else, who has access to the model, could run a scan and analysis on curl for me using Mythos and send me a report. To me, the distinction isn’t that important. It’s not that I would have a lot of time to explore lots of different prompts and doing deep dive adventures anyway. Getting the tool to generate a first proper scan and analysis would be great, whoever did it. I happily accepted this offer.
That is some very hard evidence to add to the the cartel theory.
Meanwhile, Post Quantum? Hello? It’s a real threat. This cartel nonsense is destroying trust in Anthropic. Every day now I get open mouths and saucer eyes when I demonstrate free and commodity tools to CISOs that prove how Mythos “velvet rope” access gets them exactly… nothing.
How bad is this nothingburger distraction? I usually show the following vulnerability family, for example. Mythos was trained on a 2007 bug on the left and then tested it against the sister 2008 codebase on the right. That’s a retrieval of very old known bugs, NOT discovery, by any modern definition. In the cURL example above, even retrieval wasn’t done right by Mythos. If you read the Anthropic announcement right, it’s been a lot of hot air from day one.
