OpenAI is targeting a blind spot in LLM coding performance: low-resource programming languages. ChatGPT now receives focused improvements for languages like COBOL and SQLite, where training data is scarce and model performance has historically degraded. The company built new evaluation benchmarks specifically for software engineering tasks in these environments, a methodological choice that matters more than the headline result.
The COBOL migration work is the detail worth reading for. COBOL runs an estimated 95 billion lines of legacy enterprise and government infrastructure. Improving LLM performance there is not an academic exercise, it is an infrastructure problem with direct economic stakes. The SQLite evaluation track adds a second concrete axis, testing code generation and debugging against a database engine with strict, well-defined correctness criteria.
What makes this worth the full read is the evaluation design, not the model update. OpenAI is publishing how it measures progress in data-sparse language environments, which sets a reproducible standard other labs can challenge or adopt. If the benchmarks hold up to scrutiny, they matter more long-term than any single ChatGPT improvement shipped alongside them.
[READ ORIGINAL →]