OpenAI reveals low-resource language coding performance… COBOL pass rate 10.9%→72.1%

OpenAI announced on October 19, 2026, that it improved AI coding performance in low-resource programming languages such as COBOL, Fortran, and Visual Basic. Since the evaluation was conducted directly by the company, the figures below should be read as values reported by OpenAI.

1. What was announced

These languages have less code and documentation available for training than Python or JavaScript. Nevertheless, OpenAI explained that COBOL supports more than 40% of online banking systems, 80% of in-person credit card payments, and 95% of ATM transactions. These figures are based on IBM material cited by OpenAI. OpenAI released two evaluation tools. MultiLangBench is a task to build a SQLite-compatible database engine from scratch in 20 languages. The COBOL migration evaluation examines whether original behavior is preserved when existing programs are converted to one of C++, Java, Kotlin, Python, or Rust.

2. New engine implementation evaluation: 26.8% to 87.4% on average

At the maximum reasoning intensity, GPT-6 Astra achieved an average pass rate of 87.4% on MultiLangBench. The previous model, GPT-5.6 Sol, achieved 26.8%. COBOL rose from 10.9% to 72.1%, OCaml from 25.6% to 85.8%, Fortran from 21.4% to 83.5%, and Visual Basic from 27.5% to 89.4%. Nineteen of the 20 languages scored at least 83%, while COBOL was the lowest at 72%.

GPT-6 Luna, which reduces costs, recorded an average of 40.3%. OpenAI estimates that it costs about $0.77 per sample, approximately 78% less than GPT-5.6 Sol's $3.48. The evaluation setup and test set are available in the MultiLangBench public repository ↗.

3. COBOL migration: 5.4% to 86.6% at low reasoning intensity

In the COBOL migration evaluation, GPT-6.1 Sol recorded 86.6% accuracy at low reasoning intensity. GPT-5.6 Sol under the same setting scored 5.4%. OpenAI stated that the cost was 61% of GPT-5.6 Sol's. Increasing the reasoning intensity raises the score to 97.9%.

OpenAI also pointed out a specific pitfall. COBOL discards digits after the decimal point unless this is explicitly specified. If it is incorrectly migrated to round numbers, the program may run without errors but produce different financial results. Therefore, tests that verify such business rules are essential for migration work.

4. What practitioners should check

First, these figures come from OpenAI's own evaluation. As of the time of writing, I have not confirmed replication results from an independent institution. Second, the task design may not be identical to real-world legacy systems. Third, when migrating financial and public-sector systems, testing and business-rule review must follow the model's output.

Source: OpenAI official announcement (October 19, 2026), MultiLangBench public repository https://github.com/openai/MultiLangBench

This English edition was written and translated with AI assistance, with sources provided. For errors or additional information, please request a correction. Editorial policy