OpenAI has published a new series of coding evaluations showing substantial improvements in how its latest models handle older and less commonly supported programming languages. Among the highlights is GPT-6 Astra, which achieved an average 87.4% test pass rate across MultiLangBench, a 20-language benchmark designed to measure how effectively coding models can work outside today's most popular development ecosystems.
The benchmark includes languages such as COBOL, Fortran, Visual Basic and OCaml, alongside 16 others. OpenAI says the selection was influenced by feedback from developers and enterprise customers who reported that existing coding models often struggle when dealing with these languages, particularly within large legacy systems that organisations still depend on.
Legacy Languages Still Power Critical Infrastructure
The focus on older programming languages is particularly relevant because many continue to support essential systems despite receiving far less attention from modern AI coding tools. COBOL, for example, remains deeply embedded within banking and financial infrastructure, while Fortran is still widely associated with scientific and engineering workloads.
OpenAI highlighted estimates suggesting that COBOL continues to support more than 40% of online banking systems, around 80% of in-person credit card transactions and approximately 95% of ATM transactions. Fortran, meanwhile, remains important in areas such as weather forecasting and nuclear reactor safety analysis.
Replacing these systems is rarely straightforward. Some applications have been running for decades, documentation may be incomplete and the original developers may have retired or left the organisation long ago. In many environments, the business rules required to keep critical systems operating exist primarily inside the source code itself.
MultiLangBench Rebuilds SQLite In 20 Languages
MultiLangBench attempts to evaluate coding ability through a demanding practical task rather than relying only on short programming exercises. Models are asked to build a SQLite-compatible database engine from scratch using the specified programming language.
Database files serve as the shared interface for evaluation. The candidate program receives an initial database together with a SQL query, processes the request and produces a modified database. The resulting contents are then compared with the output generated by the reference SQLite implementation.
Tests were adapted from SQLite's existing test suite, with results measuring both the proportion of tests successfully passed and the estimated API cost required to produce each sample. This forces the model to handle much more than isolated code generation, including maintaining a larger codebase, managing state, handling errors and adding functionality without breaking previously working behaviour.
GPT-6 Astra Performs Strongly Across Nearly Every Language
At maximum reasoning effort, GPT-6 Astra reportedly scores above 83% in 19 of the 20 languages included in MultiLangBench. COBOL remains the most challenging language in the benchmark, although Astra still achieves a considerably stronger result than earlier models.
The improvement compared with GPT-5.6 Sol is particularly significant across several of the benchmark's hardest languages:
• COBOL improved from 10.9% to 72.1%.
• Fortran improved from 21.4% to 83.5%.
• Visual Basic improved from 27.5% to 89.4%.
• OCaml improved from 25.6% to 85.8%.
These results suggest that newer coding models are becoming substantially more capable of dealing with languages that previously produced much weaker performance.
COBOL Remains The Hardest Language
Despite the overall gains, COBOL continues to stand out as the most difficult language within MultiLangBench. GPT-6 Astra reaches roughly 72%, which is a dramatic improvement over the 10.9% recorded by GPT-5.6 Sol but still noticeably behind performance in most of the other languages.
That difficulty is understandable. COBOL has very different conventions from modern programming languages and frequently appears inside enormous systems containing business rules accumulated across decades.
An AI model working with such software therefore needs to understand not only syntax but also subtle behaviour that may have significant financial consequences if changed incorrectly.
OpenAI Also Tested Full COBOL Migration
A second evaluation moves beyond generating new code and instead tests whether AI can safely migrate existing COBOL programs into modern languages.
Models were asked to translate existing COBOL applications into C++, Java, Kotlin, Python or Rust while preserving the original behaviour exactly. Hidden tests compare observable outputs byte-for-byte, and a migration only counts as successful when every test passes.
This makes the task significantly more demanding than simply producing code that looks equivalent.
Small differences in language semantics can easily change program behaviour. COBOL's handling of fixed-point arithmetic is one example: excess decimal digits are discarded unless explicit rounding is requested, meaning a seemingly reasonable translation into another language could still produce different financial results.
GPT-6.1 Sol Reaches 97.9% On COBOL Migration
OpenAI reports another major performance jump with GPT-6.1 Sol on the COBOL migration benchmark.
At low reasoning effort, GPT-6.1 Sol reaches 86.6% accuracy, compared with just 5.4% for GPT-5.6 Sol under the same setting. At the same time, the newer model reportedly costs around 39% less, requiring approximately 61% of the previous model's API expenditure.
When reasoning effort is increased, GPT-6.1 Sol reaches 97.9% accuracy. OpenAI describes this as pushing the benchmark close to saturation, meaning there is relatively little remaining room for improvement within the current evaluation.
That does not mean COBOL migration has suddenly become a fully automated process, but the results show how quickly model capability is improving on highly specialised programming tasks.
Exact Behaviour Matters More Than Similar-Looking Code
Migrating legacy applications is particularly difficult because producing syntactically correct code is not enough. The new application needs to behave exactly like the old one.
This can become challenging when the original system contains decades of business rules, unusual numeric behaviour or assumptions that were never properly documented.
An AI-generated translation might look cleaner and more modern while still producing slightly different output under certain conditions. In banking, government or industrial systems, a tiny difference can become a serious operational problem.
That is why OpenAI's benchmark requires exact output parity rather than judging code primarily on appearance or readability.
GPT-6 Luna Focuses More On Cost Efficiency
GPT-6 Luna takes a different position in the results. Rather than attempting to match Astra's overall accuracy, Luna appears designed to offer significantly lower operating costs while still improving over older models.
At maximum reasoning effort, GPT-6 Luna achieves an average 40.3% MultiLangBench pass rate and reportedly outperforms GPT-5.6 Sol across every tested language.
Results for several of the more difficult languages include:
• COBOL – 17.1%
• Fortran – 31.4%
• OCaml – 34.1%
• Visual Basic – 41.6%
The estimated average API cost is approximately US$0.77 per sample, compared with US$3.48 for GPT-5.6 Sol. That represents a reduction of around 78%.
Cost Is Becoming An Important Coding Model Metric
Model evaluations have traditionally focused heavily on accuracy, but API cost is becoming increasingly relevant as AI coding systems move into large-scale enterprise deployments.
A small difference in price may not matter when generating one function. It becomes much more significant when an organisation is analysing millions of lines of source code, performing automated migrations or running coding agents continuously across large repositories.
This means developers increasingly need to evaluate models based on both capability and economics.
A slightly weaker model may still be the better choice for high-volume routine tasks if it can complete them at a fraction of the cost.
Legacy Modernisation Is A Huge Enterprise Opportunity
These improvements could become especially important for organisations maintaining decades-old systems.
Legacy modernisation projects are traditionally expensive because teams need engineers who understand both the old technology and the target environment. Those specialists are becoming increasingly difficult to find as languages such as COBOL become less common among younger developers.
AI coding systems could potentially help analyse undocumented applications, explain unfamiliar code, identify dependencies and produce initial migrations.
Rather than replacing experienced engineers, the more realistic near-term role may be helping those engineers work through enormous legacy systems faster.
AI Could Help Recover Undocumented Business Logic
One of the hardest challenges in old enterprise applications is understanding why the software behaves the way it does.
The original design documents may no longer exist. Comments may be incomplete. Employees who understood the system may have left decades ago.
The code effectively becomes the documentation.
Modern AI models could potentially help organisations analyse those applications and explain the logic in more understandable terms. That could make it easier for current teams to determine which behaviours must be preserved during migration.
This may ultimately become just as valuable as automatically translating the code itself.
Production Systems Still Require Expert Review
OpenAI is clear that benchmark performance should not be interpreted as proof that AI-generated code can be deployed directly into production without human oversight.
The evaluations measure behaviour against defined tests. They do not demonstrate that a migrated banking application, weather system or industrial platform is automatically ready for live deployment.
Real production environments contain business rules, integrations, infrastructure dependencies and unusual edge cases that benchmarks cannot fully reproduce.
Expert review therefore remains essential.
Passing Tests Does Not Guarantee Operational Safety
A model achieving 97.9% on a benchmark sounds extremely impressive, but critical software demands a different standard.
A migration that works correctly 97 times out of 100 may still be unacceptable if one of those failures affects financial calculations, safety systems or customer records.
Organisations will therefore still need extensive testing, validation and staged deployment.
AI may dramatically reduce the amount of manual work required, but responsibility for confirming that the migrated system behaves correctly remains with the development and engineering teams.
The Biggest Improvement May Be Language Coverage
One of the more interesting aspects of MultiLangBench is that it moves AI coding evaluation beyond the languages that dominate modern benchmarks.
Most coding models are naturally strongest in languages such as Python, JavaScript and other ecosystems with enormous quantities of publicly available source code.
Enterprise environments are much more diverse.
Banks may still depend on COBOL. Scientific institutions may rely on Fortran. Internal applications can contain Visual Basic or other technologies that disappeared from mainstream development years ago.
A genuinely useful enterprise coding model needs to operate across all of these environments rather than only the newest technology stacks.
AI Coding Is Moving Beyond Greenfield Development
Early coding assistants were particularly useful when developers wanted to create something new.
Generate a function.
Write a unit test.
Build a simple application.
The newer evaluations show that AI coding is increasingly moving towards the much harder challenge of working with software that already exists.
That includes understanding legacy code, maintaining compatibility and performing migrations without breaking established behaviour.
For large organisations, those capabilities may ultimately be more valuable than generating another new application from scratch.
Final Thoughts
OpenAI's latest coding evaluations suggest that AI models are becoming dramatically better at working with programming languages that have historically been difficult for coding assistants. GPT-6 Astra's average 87.4% MultiLangBench pass rate, combined with major improvements in COBOL, Fortran, Visual Basic and OCaml, demonstrates how quickly that capability is advancing.
The COBOL migration results are perhaps even more interesting. GPT-6.1 Sol reportedly reaches 86.6% accuracy at low reasoning effort while costing less than GPT-5.6 Sol, and climbs to 97.9% when additional reasoning is applied.
GPT-6 Luna takes another approach by offering considerably lower costs while still improving over the previous generation across all tested languages.
None of this means organisations can suddenly hand decades-old banking or industrial systems to an AI and migrate them automatically. Critical production software still requires experienced engineers, extensive testing and careful validation.
But these results point towards something potentially very significant for enterprise IT: AI coding tools are no longer becoming useful only for the languages developers are writing today. They are increasingly becoming capable of understanding and modernising the languages that much of the world's critical infrastructure was built with decades ago.
Comments 0