OpenAI claims breakthrough in COBOL migration, but experts urge caution

OpenAI announced on October 19th that its latest AI models have achieved substantial improvements in migrating legacy software, specifically targeting aging languages like COBOL. The company claims these advancements could help organizations maintain and modernize critical infrastructure without disrupting existing operations, marking a potential shift in how industries handle decades-old codebases.

What happened

The announcement highlights the persistent challenge of legacy systems in global infrastructure. COBOL, developed in 1959, still underpins more than 40 percent of online banking systems, supports 80 percent of in-person credit card transactions, and handles 95 percent of ATM transactions according to IBM. It also powers essential government services, including Social Security and the IRS. Despite efforts to replace it, the language remains entrenched due to the high risk of breaking critical systems and the scarcity of developers who understand it.

OpenAI reported that its GPT-6.1 Sol model achieved 87 percent behavior preservation accuracy at low reasoning effort when migrating COBOL to modern languages such as C++, Java, Kotlin, Python, and Rust. This figure climbed to 98 percent at higher reasoning levels. For context, earlier models performed significantly worse, with GPT-5.6 Sol scoring only 5 percent under similar conditions. OpenAI emphasized that a migration is considered successful only if all tests pass, ensuring the new code behaves identically to the original.

Competitor Anthropic has also entered this space, publishing a code modernization playbook in February and promoting its Claude Code agent for similar tasks. While Anthropic did not provide specific benchmarks, it stated that the economics of COBOL modernization have shifted. This news reportedly caused shares in IBM, which generates revenue from maintaining legacy systems, to drop. However, experts remain skeptical about the immediate practicality of fully automated replacements.

How it works

The core mechanism involves using large language models to translate source code from one programming language to another while preserving its functional behavior. OpenAI’s approach relies on increasing the computational steps, or reasoning effort, the model spends analyzing the problem before generating an answer. This deeper analysis allows the model to better understand the logic and structure of the legacy code, resulting in higher accuracy rates.

Crucially, the process is not just about syntax translation. The models are tested to ensure that the output code passes the same test suites as the original software. This behavior preservation is vital for critical systems where even minor deviations can cause failures. The technology aims to automate the tedious and error-prone task of manual rewriting, potentially reducing the cost and time required for modernization projects.

Key details

• OpenAI’s GPT-6.1 Sol reached 98 percent behavior preservation accuracy at high reasoning effort.
• COBOL supports 95 percent of ATM transactions and 80 percent of in-person credit card transactions.
• The estimated annual cost of COBOL inefficiencies to the US GDP is around $105 billion.
• Experts note that rewriting code is distinct from replacing entire complex systems.
• Anthropic released a code modernization playbook in February but provided no performance benchmarks.
• Formal verification and human review remain essential for ensuring safety in critical migrations.

Why it matters

For software engineers and technical leaders, this development signals a potential reduction in the technical debt associated with legacy systems. The ability to automatically translate COBOL into modern languages like Python or Rust could lower the barrier to entry for maintaining these systems, which currently rely on a shrinking pool of specialized developers. This could lead to significant cost savings and improved security for banks, governments, and other institutions dependent on outdated infrastructure.

However, the hype must be tempered by the reality of system complexity. Experts warn that AI tools currently struggle with the broader context of legacy systems, which often involve multiple languages, diverse data formats, and intricate integrations. As Vadim Zaytsev from the University of Twente noted, breaking up a complex legacy system into smaller, replaceable units is the key difficulty that current evaluations do not address. Relying solely on AI for such tasks without robust testing and human oversight poses significant risks.

What you can do

• Evaluate AI coding tools for assisting with documentation and understanding of existing legacy code rather than full replacement.
• Implement rigorous testing frameworks to compare the behavior of legacy and modernized code side-by-side.
• Invest in formal verification methods to mathematically prove that translated code meets specified requirements.
• Maintain human oversight by having developers familiar with legacy languages review AI-generated translations.
• Start with non-critical modules to test the reliability of AI migration tools before applying them to core systems.
• Stay updated on benchmark results from multiple providers to assess the real-world readiness of these technologies.