AI Proves Its Own Code Is Safe, Cutting Years of Manual Checks

For decades, the question of whether software is truly safe has been a massive financial hurdle for developers. Proving that code runs without error under every possible condition was so expensive that it was reserved for life-critical systems like airplane avionics. Now, that barrier is crumbling. Anthropic has demonstrated that its Claude model can automatically generate the mathematical proofs required to verify existing software, a process that previously required armies of human specialists.

The scale of the recent effort is staggering. In just a few weeks, Claude produced approximately 180 million lines of mathematical proof code. This work focused on 123 open-source libraries from Google’s OSS-Fuzz security project. While humans once took years to verify even small components, this AI-driven approach compressed the timeline into a matter of weeks, fundamentally changing the economics of software security.

The project was a collaboration between Anthropic’s Frontier Red Team and its partner, Theorem. Instead of writing new code from scratch, Claude took on the role of a verifier. It translated crash tests into mathematical problems, analyzed compiled machine code, and generated extensive proofs of correctness. An independent program called Rocq then automatically checked these proofs. According to Anthropic, the process requires almost no human reviewers, a state the researchers describe as "fully autonomous" proof generation, even though the underlying formal foundations were built by humans at Theorem.

The practical impact of this automation was immediately visible in the bugs it uncovered. While generating proofs, Claude identified hundreds of flaws that conventional testing had failed to catch. Many of these errors only manifested under specific, rare conditions, such as unusually large input files or particular sequences of data access. Because they were hidden from standard tests, they had remained undetected for long periods.

One example highlights the speed of detection. During the verification of a widely used PDF reader, the team found as many security bugs in a single week as are typically reported for that software in an entire year. In another instance, Claude caught a logical error that had persisted for more than a decade. Anthropic notes that even when a complete proof fails, the attempt itself often surfaces real, critical bugs.

Despite the success, the team acknowledges strict limitations. "Formally verified" does not mean the software is perfect in every environment. The proofs hold only under specific conditions, including a particular processor architecture, single-threaded programs, and inputs smaller than 64 MB. In the real world, these libraries often run on different hardware configurations, meaning the verification covers a specific slice of reality rather than the whole.

The cost of this precision is high. Individual proofs can consume hundreds of millions of tokens, and Anthropic has not disclosed the total project costs. However, efficiency is improving. Smaller test runs showed that proofs are reusable, and newer models are becoming more efficient. Specifically, Opus 5.5 solved half the proofs using about 40 percent fewer tokens than its predecessor, Opus 5.

The stakes extend beyond just fixing bugs. If AI can write code and simultaneously guarantee its correctness under defined conditions, developers can trust AI-generated software more. This shifts the industry focus from "Can AI program?" to "Can AI prove that the code it writes is safe?" The potential for reducing review times and overall development costs is significant.

Anthropic expects formal verification with AI to become as routine as standard testing in the near future. Theorem’s roadmap already targets critical infrastructure components like glibc, OpenSSL, and cURL. If successful, this shift could lead to a major leap in software automation, making the entire coding process faster and more reliable.

As these tools mature, the dynamic between human oversight and machine verification will continue to evolve. The ability to catch bugs that have slipped through for years suggests a future where software safety is not just tested, but mathematically assured, at least within defined parameters.