We’ve opened Pandora’s box. The capabilities that let us build faster also let attackers find and exploit weaknesses faster. Those tools are spreading, and we cannot take them back. We now have to defend a world that wasn’t built for what they can do.
Bitcoin was always going to be a canary in the coal mine. When a security failure can lead directly to money being stolen, there is an immediate incentive to find it. The Coldcard incident this summer made that painfully clear. A firmware flaw weakened the way some wallets generated their seeds, allowing attackers to recover keys and steal funds without taking possession of the devices.
Developers then reproduced the attack with help from frontier AI models. Coinkite had already been using AI to review its code. That review missed the flaw, and its subsequent tests with frontier models missed it too. That is the asymmetry we have to understand: asking AI to find a way in is a very different task from asking it to protect against every way in. One missed weakness can be enough.
An attacker doesn't need the best AI model in the world. They need one good enough to find a weakness and turn it into a way in. Increasingly, they can download that capability and run it themselves. There is no account for a provider to suspend and no central switch that takes those copies back.
That changes the protection we get from keeping the most capable models behind restrictions. Those restrictions can buy defenders time. They cannot make the capabilities stay there.
Open-weight models are still behind the leading frontier systems overall, but they are crossing thresholds that matter. GLM-5.3 can develop working exploits in controlled tests where its predecessor barely succeeded. The frontier can remain ahead while the tools available to everyone else become dramatically more dangerous.
Attackers are already using these tools to break into financial institutions and steal data. A recent campaign against South Korean financial organizations combined an open-source agent framework with models including DeepSeek and GLM.
Now put that against the software the internet runs on.
The PHP flaws disclosed this week could allow an attacker to take control of affected servers without logging in. These weaknesses sit underneath the websites and applications people use, not just in some obscure app they could choose to avoid. The NSA recommended taking affected systems offline if they couldn't be updated immediately, until they could be fully mitigated.
Attempts to exploit PHP2Shell were detected before public disclosure, with broad attempts afterward. The full impact remains unclear, but the response window is already shrinking. Amazon is telling security teams to measure their response to critical internet-facing vulnerabilities in hours, not weeks.
AI is the only way to defend at the scale and speed we are heading toward. Human teams cannot manually match an expanding supply of machines searching for weaknesses, testing them and trying again. Defenders need AI doing that work continuously too, finding problems, developing fixes and checking whether those fixes hold.
And we are still relying on people to make that happen.
Much of our digital life rests on shared open-source software maintained by teams far smaller than the number of businesses depending on them. In PHP's case, early signs of one underlying problem were found in 2024. The work stalled over difficult trade-offs, limited specialist capacity and a decision process that couldn't resolve them quickly enough.
They had to fix a foundational system without breaking the software built on top of it. Then every organization running that software had to get the repair into use.
AI is already taking on more of that repair work. It can help write patches and test what those changes might break. But people still review the changes, approve commits and get updates deployed. The model can generate the repair faster, but getting it into use still depends on someone having the time, knowledge and authority to act.
How long is that sustainable?
More funding and better tools will help, but they do not answer the long-term question. If attackers can keep increasing the number of weaknesses they investigate, can we keep protecting the internet with a process that waits for people to approve and ship each important repair? And if we give AI more authority to make those changes itself, how do we know it isn't introducing the next failure across thousands of systems at once?
That is the reckoning ahead. We need more autonomous defense, and we need to be able to trust it with systems we cannot afford to break. Neither waiting for human teams to catch up nor handing over unrestricted control is a serious answer on its own.
The question is how quickly we can move from AI that helps people find and fix bugs to systems that can carry more of that responsibility reliably. Until then, the gap between what attackers can attempt and what people can safely repair is the one we should be watching.