Mythos is good for cybersecurity simply because now executives can’t just tell people that only superhackers can break their stuff, as people wouldn’t believe them now anyways.
Infosec for decades has been 99% “hey I found some low-hanging fruit” only to get treated like a liability by the company you report it to, if you got acknowledgment at all. Because of Mythos though, now Artificial Superhumans can find these same vulns, and anyone could be running such an intelligence! Even better, the rich untouchable people operating this particular Artificial Superhuman can’t just be suppressed or ignored by the other set of rich untouchable people that have routinely not cared in the past. So long as it makes anthropic money, maybe we’ll actually see actual improvements in security!
I don't see that it makes much difference until we know the distribution of issues that Mythos finds and how reliably it discovers them? Vulns from inspection are discovered via a stochastic process of someone looking at the code, knowing about bug classes and paying sufficient attention to notice them. That's still the case.
Whats interesting about AI assisted bug hunting is that it grants particular factions more power than they used to.
If you only rely only on stecurity through obscurity (eg attackers not having the source code) you gonna have a bad time. And even if your source code is not available, you can make a good guess about their dependencies. Find a vulnerability there and chances are your software is also vulnerable.
>What if finding every vulnerability in a piece of software were just as fast and easy as finding a few of them, thanks to automation?
This presumes there is such a thing as "every" vulnerability. It is possible that ever more sophisticated, complicated, and abstract attacks become possible/discoverable as one applies more intelligence to the problem.
IF it is indeed possible to make a piece of software completely secure, then yes, more intelligent systems make the situation better, because it will always be possible to audit a system before it is ever released and make it completely safe.
That is a very big if and, as far as I am aware, remains to be seen if it's the case
-edit- They mention this possibility themselves further down, so the authors know this is a completely speculative point/article. They don't even try to make an argument about why one possibility might be more likely than the other. This article is useless.
We know about physical-layer attacks that break some of the abstractions that software relies on, allowing an attacker to use physical access or physical proximity to violate security guarantees that are enforced by software alone. (I worked on some of these a while ago!)
For a purely remote attacker (although maybe we have to get clear on what distance counts as "physical proximity" because we need to clarify what phenomena spy satellites, for example, can observe), it seems pretty straightforward to me that there is such a thing as actually secure software.
You can make a very strong model of what the software computes and then prove that it never does some undesired thing. It's not common to do this at all, and even formal verification work may not use very strong models or models that capture some important part of the behavior, but it is possible to mathematically reason about what software does and doesn't or can and can't do.
To summarize some of the problems that I partly just mentioned (in no particular order)
(1) We may not have the will, the skill, or the economic demand to make software secure in a very strong sense.
(2) Attackers may subvert our infrastructure or organizations so that we don't actually apply the processes or controls, or run the software, that we expect.
(3) Physical proximity (for active or passive attacks) might sometimes include distances that are actually attainable for attackers. Maybe there are passive or active attacks involving lasers that can be mounted from multiple kilometers away, as an example. In that case most software users might not be able to be sufficiently isolated from the attackers to be protected against those attacks.
(4) Software or hardware other than the specific software whose security we're talking about might be compromised in its supply chain in a way that people don't have a plan, or resources, to detect or mitigate.
(5) Some systems might be compositionally insecure (their pieces might be secure in some relevant model, but the pieces might interact in a way that isn't secure overall, for example related to timing and concurrency problems).
(6) Our proofs of security for cryptosystems rely on unproven hardness assumptions for various primitives, some of which might turn out to be wrong.
(7) Some security properties, especially related to communications security, might be inherently unattainable even with correct software. For example, there's an argument that Roger Dingledine (Tor lead developer) once told me about that implies that no anonymity system is perfectly secure in the long run against a very powerful active adversary, unless the system is willing to make extreme trade-offs like shutting down completely in response to any attack. So it might be that we can't actually build any useful communications system that can absolutely guarantee perfect traffic analysis resistance, essentially because of inherent architectural trade-offs.
But I don't want to lose sight of the idea that you can actually meaningfully reason about what software does and so there is such a thing as the software being correct or incorrect, relative to some specification or goal for its behavior, and correct software actually does exist (which computes correct outputs for every input).
to say that defense doesn't win in the limit is the same thing as saying there is an attack that can not be defended against.
So to re-phase the question to more clearly have an answer: does there exist an attack which no one will ever be able (for all time) to come up with a defense against? (the very existence of such an attack would end the (open) internet, wholly and completely, if the only winning move is not to play...)
There will be an exhaustion of possibilities in the end. New attacks eventually run out after each surface area is hardened against those attacks.
In the limit, defense wins.
There is only one case (that i see) where this may fail. if there is a 'predicament' with the state of security: ie, if securing against attack A requires you to be insecure against attack B and vise versa (this could be a 'whack-a-mole with many different kinds of attacks' situation). But that would be 'provable'. So if such a case exists, we will know about it. And it may be true that predicaments like this could be exercised if they even can exist, we might still be able to avoid/mitigate them.
Maybe, but they’ve got everyone scared shitless. My entire org (30k employees, not just engineers) is in sprint 2 of a remediation effort, where we are systematically fixing every high+ finding across hundreds of workloads with decades of system bloat.
I’ve never seen us so aligned on a goal! Wiz is doing pretty well for itself also…
Mythos has been a boon for "look busy" work. My global corp org has been on a bender upgrading everything, patching everything. There is a giant dashboard that shows green/red for everything we have.
I think it's a total overreaction. But the edict was passed down, and here we are go.
There will probably be congressional hearings when it turns out Lazarus Group had access, and then the USG will use it as an excuse to lock AI behind harsh KYC.
I disagree. Assuming code complexity is roughly fixed, more sophisticated code analysis will result in a smaller surface area for bugs. Bugs will still be found, but there will be less bugs to be found and less opportunities to exploit.
I'd expect the bigger shift to be toward secure-by-construction building blocks: less custom code that needs to be audited from scratch, and more hardened or verified components where common bug classes are already designed out.
Why are we worried about vulnerabilities in code when AI powered social engineering will make it fast, easy, and even fun to find vulnerabilities through human interaction, faster and more deeply than ever?
According to our software (which is, of course, imperfect), your account has repeatedly been posting AI-generated and/or AI-edited comments. If so, can you please stop? It's not allowed here, and will eventually get your account banned.
Infosec for decades has been 99% “hey I found some low-hanging fruit” only to get treated like a liability by the company you report it to, if you got acknowledgment at all. Because of Mythos though, now Artificial Superhumans can find these same vulns, and anyone could be running such an intelligence! Even better, the rich untouchable people operating this particular Artificial Superhuman can’t just be suppressed or ignored by the other set of rich untouchable people that have routinely not cared in the past. So long as it makes anthropic money, maybe we’ll actually see actual improvements in security!
Whats interesting about AI assisted bug hunting is that it grants particular factions more power than they used to.
This presumes there is such a thing as "every" vulnerability. It is possible that ever more sophisticated, complicated, and abstract attacks become possible/discoverable as one applies more intelligence to the problem.
IF it is indeed possible to make a piece of software completely secure, then yes, more intelligent systems make the situation better, because it will always be possible to audit a system before it is ever released and make it completely safe.
That is a very big if and, as far as I am aware, remains to be seen if it's the case
-edit- They mention this possibility themselves further down, so the authors know this is a completely speculative point/article. They don't even try to make an argument about why one possibility might be more likely than the other. This article is useless.
For a purely remote attacker (although maybe we have to get clear on what distance counts as "physical proximity" because we need to clarify what phenomena spy satellites, for example, can observe), it seems pretty straightforward to me that there is such a thing as actually secure software.
You can make a very strong model of what the software computes and then prove that it never does some undesired thing. It's not common to do this at all, and even formal verification work may not use very strong models or models that capture some important part of the behavior, but it is possible to mathematically reason about what software does and doesn't or can and can't do.
To summarize some of the problems that I partly just mentioned (in no particular order)
(1) We may not have the will, the skill, or the economic demand to make software secure in a very strong sense.
(2) Attackers may subvert our infrastructure or organizations so that we don't actually apply the processes or controls, or run the software, that we expect.
(3) Physical proximity (for active or passive attacks) might sometimes include distances that are actually attainable for attackers. Maybe there are passive or active attacks involving lasers that can be mounted from multiple kilometers away, as an example. In that case most software users might not be able to be sufficiently isolated from the attackers to be protected against those attacks.
(4) Software or hardware other than the specific software whose security we're talking about might be compromised in its supply chain in a way that people don't have a plan, or resources, to detect or mitigate.
(5) Some systems might be compositionally insecure (their pieces might be secure in some relevant model, but the pieces might interact in a way that isn't secure overall, for example related to timing and concurrency problems).
(6) Our proofs of security for cryptosystems rely on unproven hardness assumptions for various primitives, some of which might turn out to be wrong.
(7) Some security properties, especially related to communications security, might be inherently unattainable even with correct software. For example, there's an argument that Roger Dingledine (Tor lead developer) once told me about that implies that no anonymity system is perfectly secure in the long run against a very powerful active adversary, unless the system is willing to make extreme trade-offs like shutting down completely in response to any attack. So it might be that we can't actually build any useful communications system that can absolutely guarantee perfect traffic analysis resistance, essentially because of inherent architectural trade-offs.
But I don't want to lose sight of the idea that you can actually meaningfully reason about what software does and so there is such a thing as the software being correct or incorrect, relative to some specification or goal for its behavior, and correct software actually does exist (which computes correct outputs for every input).
So to re-phase the question to more clearly have an answer: does there exist an attack which no one will ever be able (for all time) to come up with a defense against? (the very existence of such an attack would end the (open) internet, wholly and completely, if the only winning move is not to play...)
There will be an exhaustion of possibilities in the end. New attacks eventually run out after each surface area is hardened against those attacks.
In the limit, defense wins.
There is only one case (that i see) where this may fail. if there is a 'predicament' with the state of security: ie, if securing against attack A requires you to be insecure against attack B and vise versa (this could be a 'whack-a-mole with many different kinds of attacks' situation). But that would be 'provable'. So if such a case exists, we will know about it. And it may be true that predicaments like this could be exercised if they even can exist, we might still be able to avoid/mitigate them.
So large bets on defense winning in the end.
I’ve never seen us so aligned on a goal! Wiz is doing pretty well for itself also…
I think it's a total overreaction. But the edict was passed down, and here we are go.
Granted, given that most cybersecurity news over the past decade has been grim, both could be true...
https://x.com/kevinakwok/status/2049984076141281482
In a physical sense, Anthropic is giving access to who we believe are the "defenders", aka the United States DoD and Israel.
Social engineering requires a lot more organisation from attackers.
(See https://news.ycombinator.com/newsguidelines.html#generated and https://news.ycombinator.com/item?id=47340079.)
hn@ycombinator.com