180 Million Lines: Claude's New Method for Proving Code Safety

Anthropic’s Claude model recently generated approximately 180 million lines of mathematical proof code to verify the correctness of existing software. This achievement marks a significant shift in how developers might approach quality assurance, moving beyond traditional testing methods toward formal mathematical validation. The effort was conducted over a span of just a few weeks by Anthropic’s Frontier Red Team in partnership with a firm named Theorem. By applying this advanced method to 123 open-source libraries, the system not only produced extensive proofs but also identified hundreds of bugs that standard testing protocols had previously missed.

Formal verification has long been considered too costly for widespread use. Historically, proving that software runs without errors under all conditions required specialized teams and significant time. For instance, a team of experts previously needed 20 person-years to formally verify the small operating system kernel known as seL4. Because of this high barrier to entry, the method was largely reserved for critical applications such as aviation software. The collaboration between Anthropic and Theorem aims to lower that barrier by letting an AI model handle the heavy lifting of creating these proofs.

The process did not involve Claude writing new software from scratch. Instead, the model focused on verifying existing code. It achieved this by translating crash tests into mathematical problems and analyzing the compiled machine code. The model then produced extensive proofs for these libraries, which are part of Google’s OSS-Fuzz security project. An independent verification program called Rocq automatically checks these proofs for validity. According to Anthropic, this automated workflow requires very little human review, a state the researchers describe as fully autonomous proof generation. It is important to note, however, that the underlying tools and formal foundations were still built by human engineers at Theorem.

During the verification process, Claude encountered hundreds of bugs and was permitted to fix them directly. Many of these defects would have remained hidden because they only appear under highly specific conditions, such as when processing unusually large input files or when a particular sequence of accesses occurs. The impact of this capability was evident in a specific case involving a widely used PDF reader. In that instance, the team discovered as many security bugs in a single week as are typically reported for that software in an entire year. In another application, the model identified a logical error that had gone undetected for more than a decade. Anthropic notes that even when a complete proof eventually fails, the attempt still surfaces real issues.

Despite the impressive results, Anthropic acknowledges that formal verification is not a guarantee of being error-free. The proofs hold only under strict conditions, including a specific processor architecture, single-threaded programs, and inputs under 64 megabytes. In practical use, these libraries often run on different hardware and in varied configurations. Additionally, the computational cost is significant, with individual proofs requiring up to hundreds of millions of tokens. While Anthropic did not report the total project costs, the potential for saving time and money on the entire coding process remains a key driver for the technology.

A smaller test run also demonstrated that these proofs are reusable. Newer models are also showing improved efficiency. For example, the Opus 5.5 model solved half of the proofs using about 40 percent fewer tokens than its predecessor, Opus 5. These efficiency gains suggest that the method could become more viable for broader application as models continue to improve in performance and cost-effectiveness.

This project strengthens the argument for greater software automation. If AI can write code and guarantee its correctness under defined conditions, the industry’s focus may shift from asking whether AI can program to whether it can prove that its code is safe. Anthropic expects that formal verification with AI will soon become as practical as standard testing. Theorem’s roadmap already targets critical infrastructure components such as glibc, OpenSSL, and cURL, indicating a clear path for future implementation in essential software systems.

The ability to automatically prove code safety could have consequences that extend well beyond current coding tools. By integrating mathematical proof into the development cycle, developers may be able to trust AI-generated software more confidently. This could lead to reduced time spent on manual reviews and a more robust foundation for the next generation of automated software development.