AI Is Breaking Cybersecurity’s Old Rules — and the Fixes Aren’t Ready

A stylized illustration of mouse pointers tearing down a fence

AI is breaking cybersecurity’s familiar playbook, and the consequences are already visible in a punishing stretch of breaches that has left defenders exhausted and exposed. Alex Stamos, chief security officer at Cognition and a longtime cybersecurity expert, describes the current moment as the start of an adversary capability takeoff—one that is automating the steps attackers use to break into systems.

“We have been at DEFCON 2 for all of October and now going into November,” Stamos said, relaying a sentiment shared by other security leaders. The past month brought major software bugs in PHP and Node, alongside a wave of breaches at AI-related companies, a massive Binance heist, and the Cursor incident. In that case, attackers reportedly stole API keys from Fireworks, used them to breach Cursor, and then pushed a malicious plugin to thousands of companies.

The kill chain, inverted

A stylized illustration of mouse pointers tearing down a fence

Stamos argues that open-weight models have reached a point where most of the cyber kill chain can be automated. Traditionally, attackers researched software, found vulnerabilities, wrote exploits, and then deployed them. Now, AI systems can hunt for bugs and write exploits in real time.

He points to ARTEX, a Chinese open-source tool available on GitHub, as an example. Powered by a capable open-weight model, it can break into a shocking number of organizations. The tool was initially linked to three South Korean banks before spreading to roughly 200 more companies this week. “It has been a long time since you’ve been able to go and just download an open-source tool and break into multi-billion dollar companies,” Stamos said. “It’s like the ’90s.”

The attackers are varied. North Koreans carried out the Binance hack, which Stamos attributes mostly to human intelligence and sophisticated social engineering. The AI-company breaches, by contrast, appear tied to a Chinese cottage access industry—small contractors who break in and sell information to larger firms.

Why the old defense economy is failing

Article image

The security industry that grew up after Microsoft’s trustworthy computing memo in the early 2000s relied on a hierarchy of skills. Elite researchers were few, and operational attack teams required a conspiracy of people. Bitcoin weakened one assumption by letting technically skilled people launder money. AI is demolishing another: the skills gap.

Stamos says a single person with a Mac Studio and a quantized open-weight model can now reverse engineer patches and write stable exploit code. It is not fully automatic yet, but it is close. Meanwhile, the cost of attack is falling toward zero while the cost of defense remains high. One colleague at a large security platform company said a single investigation using a frontier cyber model costs $900 on their product.

That economics problem is central. Stamos argues the industry must lower the cost of defense, not just focus on finding and fixing bugs. Software security is only about a quarter of a CISO’s job; the rest—watching logs, investigating alerts, and responding without shutting down the company—has been helped less by AI and in some cases made worse.

Companies will need to automate first-tier security operations with cheaper classifiers and smaller models, escalating to larger models only when necessary. Stamos is working on that at Cognition, but he says much more needs to be done across the industry.

A dark age, not doomsday

Stamos is neither a doomer nor a techno-optimist. He expects a difficult period—a “dark age” for cybersecurity—before conditions improve. The goal for defenders is to shorten it.

One reason for optimism, he says, is that median code will increasingly be written by LLMs in memory-safe and type-safe languages. But decades of human-written code in unsafe languages remain. He argues that scanning and patching is not enough. If a codebase has thousands of bugs, rewriting it may now be cheaper.

His most ambitious suggestion: Google or Amazon should rewrite the Linux kernel in Rust and release it as a fork for production servers. He estimates the cost at $100 million, a bargain compared with repeatedly patching and rebooting Linux machines worldwide. “The technology industry has not been able to get together and make these hard decisions,” he said.

In the near term, Stamos urges companies to red-team themselves with AI within the next 90 days. If they do not, he warns, they will be breached. He acknowledges the legal and insurance hurdles but insists the risk is necessary. “You can just do things,” he said, describing a lesson from working at an AI startup.

He is skeptical of “swarm” rhetoric, noting that the Hugging Face incident involved hundreds to more than a thousand agents and enormous hardware costs—an inefficient attack unlikely to be replicated by most adversaries. Defensive agents can help, but only with deterministic limits and human oversight.

Verifying what's real

The conversation also turned to AI-generated content. Stamos argues that detecting fake media is a losing battle compared with verifying authentic content. He suggests platforms like Instagram could restrict content to camera-originated media and in-app edits, creating a verifiable chain of custody.

He also criticizes AI products designed to seem like friends. “They should not seem to have souls,” he said. For Stamos, the broader challenge is human uplift risk—people using AI to harm others—and the industry needs more agency to tackle it.

“We are entering a dark age,” he said. “Our responsibility is to help shorten that dark age.”

Leave a comment

Your email address will not be published. Required fields are marked *