OpenAI has published new coding evaluations showing major improvements in how its latest models handle older and less commonly supported programming languages. Among the headline results, GPT-6 Astra achieved an average 87.4% test pass rate across MultiLangBench, a benchmark covering 20 programming languages including COBOL, Fortran, Visual Basic and OCaml.
The benchmark was developed in response to feedback from developers and enterprise customers who said existing coding models often struggle with languages that remain deeply embedded in critical business systems. OpenAI has also released the MultiLangBench evaluation setup and accompanying test suite, allowing results to be examined on a language-by-language basis.
Legacy Languages Still Power Critical Infrastructure
Although languages such as COBOL and Fortran may appear outdated compared with modern development environments, they continue supporting some of the world's most important systems. COBOL alone is estimated to support more than 40% of online banking platforms, around 80% of in-person credit card transactions and approximately 95% of ATM transactions.
Fortran continues to play an important role in scientific and engineering workloads, including weather forecasting and nuclear reactor safety analysis. These are systems that organisations cannot simply replace overnight, particularly when decades of business logic and technical knowledge are embedded within the existing code.
Maintaining this software can also be extremely difficult. Documentation may be incomplete, original developers may have retired or left the organisation, and important business rules may exist only inside the source code itself.
MultiLangBench Tests More Than Simple Code Generation
MultiLangBench is designed to evaluate whether AI models can handle a substantial programming project rather than merely solve isolated coding exercises. The benchmark asks the model to build a SQLite-compatible database engine from scratch using a specified programming language.
Each candidate program receives an initial database together with an SQL query and must produce a modified database. The output is then compared against the reference SQLite implementation to determine whether the resulting database contents match.
Tests were adapted from SQLite's existing test suite, meaning the model must manage a larger codebase, preserve state, handle errors and implement additional functionality without breaking existing behaviour. Results also include estimated API costs alongside the percentage of tests successfully passed.
GPT-6 Astra Performs Strongly Across Almost Every Language
At maximum reasoning effort, GPT-6 Astra reportedly scores above 83% in 19 of the 20 languages included in the benchmark. COBOL remains the most difficult language, but even there the model achieves a significant improvement over previous-generation systems.
Several of the biggest gains compared with GPT-5.6 Sol include:
• COBOL improved from 10.9% to 72.1%.
• Fortran improved from 21.4% to 83.5%.
• Visual Basic improved from 27.5% to 89.4%.
• OCaml improved from 25.6% to 85.8%.
These results suggest newer coding models are becoming substantially more capable outside the programming languages that dominate modern development and public code repositories.
COBOL Remains The Most Difficult Language
Despite the overall performance gains, COBOL continues to present the greatest challenge. GPT-6 Astra reaches around 72%, which is dramatically better than the 10.9% achieved by GPT-5.6 Sol but still noticeably lower than its performance across most of the other languages.
Part of the difficulty comes from COBOL's age and the way it is commonly used. Enterprise COBOL applications can contain decades of accumulated business rules, unusual data structures and behaviour that differs significantly from modern programming conventions.
An AI model therefore needs to understand much more than syntax. It must preserve subtle application behaviour that may have significant operational or financial consequences if changed incorrectly.
OpenAI Also Tested Full COBOL Migration
A separate evaluation focuses on another major enterprise problem: migrating existing COBOL applications into more modern languages. The models were asked to translate COBOL programs into C++, Java, Kotlin, Python or Rust while preserving the original behaviour exactly.
Hidden tests require byte-for-byte parity on observable outputs, and a migration only counts as successful if every test passes. That makes the benchmark much harder than simply generating code that looks logically equivalent.
Even tiny differences in language semantics can cause a migration to fail. COBOL's fixed-point arithmetic, for example, discards excess decimal digits unless rounding is specifically requested, meaning a seemingly reasonable translation could still produce different financial results.
GPT-6.1 Sol Reaches 86.6% At Low Reasoning Effort
On the COBOL migration benchmark, GPT-6.1 Sol reportedly reaches 86.6% accuracy at low reasoning effort. Under the same conditions, GPT-5.6 Sol managed only 5.4%.
The newer model also achieves the stronger result while costing less. OpenAI reports that GPT-6.1 Sol requires approximately 61% of the API cost of GPT-5.6 Sol, representing a reduction of around 39%.
That combination of better accuracy and lower cost is particularly relevant for enterprises considering large-scale legacy modernisation, where thousands or millions of lines of old code may need to be analysed or migrated.
Higher Reasoning Pushes Accuracy To 97.9%
When additional reasoning effort is applied, GPT-6.1 Sol reaches 97.9% accuracy on the migration evaluation. OpenAI describes this as bringing the benchmark close to saturation.
That figure is impressive, but it does not mean COBOL migrations have suddenly become fully automated. Production applications frequently contain dependencies, undocumented business logic and edge cases that no benchmark can reproduce completely.
The result instead suggests that AI could become increasingly useful as an accelerator for experienced engineering teams working through large modernisation projects.
Exact Behaviour Matters More Than Similar-Looking Code
Legacy migrations are difficult because producing syntactically correct code is only part of the challenge. The new application must behave exactly like the system it replaces.
A translation can look cleaner, use more modern programming patterns and compile successfully while still producing slightly different results in certain situations. In a banking system or safety-critical scientific application, those differences may be unacceptable.
That is why byte-for-byte behavioural parity makes the migration benchmark particularly demanding. The model is not merely being judged on whether the converted code appears sensible; it needs to preserve what the original software actually does.
GPT-6 Luna Takes A More Cost-Focused Approach
GPT-6 Luna occupies a different position in the results. Rather than matching Astra's overall accuracy, Luna appears designed to provide significantly lower operating costs while still improving on GPT-5.6 Sol across every tested language.
At maximum reasoning effort, GPT-6 Luna achieves an average 40.3% MultiLangBench pass rate. Selected results include COBOL at 17.1%, Fortran at 31.4%, OCaml at 34.1% and Visual Basic at 41.6%.
The estimated API cost averages around US$0.77 per sample, compared with US$3.48 for GPT-5.6 Sol. That represents a reduction of roughly 78%.
Cost Is Becoming As Important As Accuracy
AI coding models are increasingly evaluated not only on capability but also on how much it costs to use them. A small price difference does not matter much when generating one short function, but the economics change dramatically when models are applied across large repositories.
Enterprise modernisation projects may involve enormous quantities of source code. Organisations could use AI to analyse programs, generate documentation, identify dependencies, produce migration candidates and run repeated validation cycles.
At that scale, the difference between less than a dollar and several dollars per sample can become substantial. A cheaper model with slightly lower accuracy may therefore make more sense for routine or high-volume tasks.
Legacy Modernisation Could Become A Major AI Use Case
Many large organisations still operate software created decades ago because replacing it is difficult, risky and expensive. The systems may continue performing essential work perfectly well, but finding developers capable of maintaining them becomes harder every year.
AI could help bridge some of that knowledge gap. Coding models can potentially explain unfamiliar applications, identify relationships across files and translate old logic into more modern languages.
Rather than completely replacing legacy specialists, AI may allow those specialists to work more effectively. One experienced COBOL engineer supported by capable AI tools could potentially review and modernise far more code than would previously have been practical.
AI Could Help Recover Undocumented Business Logic
One of the biggest problems with old applications is that the original design documentation may no longer exist. Developers who understood why certain decisions were made may have retired decades ago, leaving the code as the only surviving record of the system's behaviour.
AI models capable of understanding older languages could help analyse that code and explain its logic in more accessible language. This could help modern engineering teams determine which rules need to be preserved before attempting a migration.
In some projects, understanding the old application may ultimately be more valuable than automatically translating it. Once organisations know exactly what the system does, they can make better decisions about how it should eventually be replaced.
Language Coverage Is Becoming More Important
Most AI coding benchmarks naturally concentrate on popular languages such as Python, JavaScript, Java and C++. Those languages have enormous quantities of publicly available examples and remain central to modern development.
Enterprise environments are much more diverse. Banks may rely on COBOL, scientific organisations on Fortran and internal business software on Visual Basic or other technologies that receive relatively little attention from modern coding tools.
A coding model intended for enterprise use therefore needs to support more than the latest technology stacks. MultiLangBench attempts to measure that broader capability by deliberately including languages where previous models have historically struggled.
AI Coding Is Moving Beyond Greenfield Development
Early AI coding assistants were particularly effective when developers wanted to build something new. They could generate functions, create boilerplate code, write tests and explain unfamiliar APIs.
The more difficult challenge is working with software that already exists. Legacy repositories can contain decades of architectural decisions, unusual conventions and hidden dependencies.
The latest evaluations suggest AI coding is increasingly moving into that territory. Understanding existing systems, preserving behaviour and performing migrations may ultimately become more valuable to large enterprises than simply generating another new application from scratch.
Production Systems Still Require Expert Review
OpenAI emphasises that strong benchmark performance does not mean organisations should deploy AI-generated code directly into production without human validation. MultiLangBench measures behaviour against defined tests, not readiness for a live banking or industrial environment.
Real applications contain external integrations, infrastructure dependencies, security requirements and business rules that may not appear in the benchmark. A model can therefore score extremely well while still making mistakes that would matter in production.
Expert review, comprehensive testing and staged deployment remain essential for critical systems.
97.9% Is Impressive, But Critical Systems Need More
A 97.9% success rate sounds extremely strong, but the acceptable error rate depends heavily on the application. A migration that works correctly almost every time may still be unsuitable if the remaining failures affect financial calculations or safety-critical behaviour.
Banks cannot accept a system that occasionally produces a slightly different transaction result simply because the migration benchmark score looks impressive. Scientific and engineering environments may have similarly strict requirements.
AI can reduce the amount of manual work required, but human teams remain responsible for proving that the migrated system behaves correctly under every important condition.
Benchmark Performance Is Not Deployment Performance
Another important distinction is that published benchmark figures describe controlled evaluations. They measure pass rates and API costs under defined conditions.
Real-world deployments introduce additional complications such as enormous repositories, proprietary frameworks, inconsistent documentation and dependencies that were never designed to be analysed by AI.
The gap between benchmark success and production readiness is therefore still significant. Organisations should treat these results as indicators of capability rather than guarantees about what will happen inside their own systems.
The Results Still Represent A Major Shift
Even with those limitations, the improvements are difficult to ignore. Moving COBOL performance from around 11% to more than 72% within one model generation represents a substantial jump.
The improvements in Fortran, Visual Basic and OCaml are similarly significant. These are precisely the types of languages where organisations may have previously found AI coding assistants too unreliable to be useful.
As models continue improving, tasks that once required highly specialised manual work could increasingly become collaborative workflows between experienced engineers and AI.
Final Thoughts
OpenAI's latest coding evaluations suggest that its models are becoming substantially better at working with the programming languages that continue powering much of the world's critical legacy infrastructure. GPT-6 Astra's 87.4% average MultiLangBench pass rate is particularly notable because the benchmark spans 20 languages rather than concentrating only on modern development favourites.
The improvements are dramatic in several difficult areas, with COBOL climbing from 10.9% to 72.1%, Fortran reaching 83.5%, Visual Basic hitting 89.4% and OCaml reaching 85.8%. Meanwhile, GPT-6.1 Sol's COBOL migration performance rises as high as 97.9% when additional reasoning is applied.
GPT-6 Luna shows another important trend by significantly reducing API costs while still improving over GPT-5.6 Sol across every tested language. This suggests coding models are becoming not only more capable but also increasingly practical for high-volume enterprise workloads.
None of these numbers mean organisations should hand decades-old banking or industrial applications to an AI and expect flawless modernisation. Legacy software still requires expert engineers, careful validation and extensive testing.
What the results do suggest is that AI coding tools are finally becoming useful for something potentially far more valuable than generating another new application: helping organisations understand, maintain and modernise the software they cannot afford to replace.
Comments 0