“Each of the results cost roughly $100,000 in API cost to develop.”
And
“Over the course of a week, one Anthropic researcher worked together with Claude to develop the HAWK attack, and another researcher built a scaffold4 that allowed Claude to fully autonomously discover the AES attack.”
Spending $100k in tokens in a week is an impressive feat even with massive parallelization. I suspect the TPS their internal folks have access to is far higher than their bulk public endpoints.
There’s a tech aristocracy rapidly emerging in our society and it’s going to tear us apart.
I can already picture the faces of national security directors everywhere.
"The attacks described in these two papers are the strongest attacks we have found to date. We are sharing them after a period of consultation with US government and industry leaders. But as we develop increasingly powerful cryptanalytic results, it would be prudent to consider how researchers should react if a language model were to discover vulnerabilities in cryptosystems where attacks do have an immediate real-world impact. We believe answering this question will require input from academia, government, and industry. We hope that our work here will help launch these conversations."
And a veiled pitch to real cryptanalysis researchers:
"Researchers at Anthropic then spent several hundred hours learning enough cryptography research to validate the model’s claim"
> The multi-agent workflow led to interesting dynamics. For example, the key idea in producing this attack was discovered by a pair of workers working together. Both started investigating the idea; the first worker prematurely rejected the idea as infeasible, but the second found a way to fully exploit it. The pair kept exchanging messages, and eventually both agreed they had found an effective attack.
this is pretty interesting. the way it is written doesn't make it sound like the collaboration actually led to the discovery, but rather just the stochastic nature of each thread in the search. it would be interesting to replay and repeat the search (possibly with prior/context pertubations) to get a sense for how often it finds or misses the known working path.
Hypothesis: the pairing / collaboration makes it much more likely to find a fruitful road previously dismissed, because... that's what happens in fiction - including books, movies, and journalism (long-form "people stories"). It's a common trope: if one character dismisses a course of action, the plot demands the other character to take it.
In a way LLMs are, after all, trained to LARP people, including fictional characters and their tropes - this was actually exploited for jailbreaking to good effect in the late pre-agentic era (read: some two years ago). C.f. Waluigi effect. Not sure if it still holds for current models, but I can't imagine why it would not.
> But as we develop increasingly powerful cryptanalytic results, it would be prudent to consider how researchers should react if a language model were to discover vulnerabilities in cryptosystems where attacks do have an immediate real-world impact.
How would they react if a human were to discover vulnerabilities in cryptosystems?
First of all we likely wouldn't know it's better to call US govt or any other govt if you have that tech, and then take that govt job and hope you can happy life... instead of annoucing it publicly only when it's a AI model where we expect it's ability to tend/scale towards infinity does it become something to tell the wider public.
Although if RSA had a vulnerability I would be very very shocked probably because I still haven't learnt post quantum encryption algorithms enough to really feel like they should be unbreable...
If there is a researcher or someone in space how should I feel about it. Is it as bad as RSA being completely broken open?
I do understand that AI will get better, and a lot actually at very easily verifiable tasks but this one I find it hard to wrap my head around because of my ignorance.
Going to guess it's more available or less overconstrained. See e.g. Fable, which is much better than Opus 4.8 and possibly than Opus 5... in the rare case of a task it doesn't punt on because of its safety guardrails.
It's awfully creepy how people are pushing us to abandon hybrid classical-PQC cipher suites and switch entirely to newfangled lattice things. I'd rather hedge my bets.
TL;DR: They marginally improved on the best known academic attack on 7-round AES-128 (which normally uses 10 rounds - you do not need to worry about AES being broken).
The attack on HAWK is perhaps more interesting - they were able to halve the effective key length. HAWK is a candidate for NIST standardisation. It has been studied academically, but isn't really deployed anywhere (because it hasn't been standardised!)
And
“Over the course of a week, one Anthropic researcher worked together with Claude to develop the HAWK attack, and another researcher built a scaffold4 that allowed Claude to fully autonomously discover the AES attack.”
Spending $100k in tokens in a week is an impressive feat even with massive parallelization. I suspect the TPS their internal folks have access to is far higher than their bulk public endpoints.
There’s a tech aristocracy rapidly emerging in our society and it’s going to tear us apart.
"The attacks described in these two papers are the strongest attacks we have found to date. We are sharing them after a period of consultation with US government and industry leaders. But as we develop increasingly powerful cryptanalytic results, it would be prudent to consider how researchers should react if a language model were to discover vulnerabilities in cryptosystems where attacks do have an immediate real-world impact. We believe answering this question will require input from academia, government, and industry. We hope that our work here will help launch these conversations."
And a veiled pitch to real cryptanalysis researchers: "Researchers at Anthropic then spent several hundred hours learning enough cryptography research to validate the model’s claim"
this is pretty interesting. the way it is written doesn't make it sound like the collaboration actually led to the discovery, but rather just the stochastic nature of each thread in the search. it would be interesting to replay and repeat the search (possibly with prior/context pertubations) to get a sense for how often it finds or misses the known working path.
In a way LLMs are, after all, trained to LARP people, including fictional characters and their tropes - this was actually exploited for jailbreaking to good effect in the late pre-agentic era (read: some two years ago). C.f. Waluigi effect. Not sure if it still holds for current models, but I can't imagine why it would not.
First, is it reproducible consistently at ~50% of workers? If not, what is the rate.
Second, are there any lessons to be learned here to increase the rate of success by changing models/weights/training?
The news by itself isn't really good news. But it could lead to good news. Maybe.
How would they react if a human were to discover vulnerabilities in cryptosystems?
Although if RSA had a vulnerability I would be very very shocked probably because I still haven't learnt post quantum encryption algorithms enough to really feel like they should be unbreable...
If there is a researcher or someone in space how should I feel about it. Is it as bad as RSA being completely broken open?
I do understand that AI will get better, and a lot actually at very easily verifiable tasks but this one I find it hard to wrap my head around because of my ignorance.
The attack on HAWK is perhaps more interesting - they were able to halve the effective key length. HAWK is a candidate for NIST standardisation. It has been studied academically, but isn't really deployed anywhere (because it hasn't been standardised!)