Will open-weight LLMs catch closed models by December 2026? Depends how you measure

A viral chart claims to predict the moment open-weight LLMs catch up to proprietary frontier models. The method measures the ‘catch-up gap’: take the best open model’s score on Artificial Analysis’s headline Intelligence Index, then find how many months earlier a closed model first hit that level. That gap began shrinking in mid-2024, and a naive line of best fit hits zero around December 3rd, 2026 — roughly six months out, prompting the author’s tongue-in-cheek advice to cash out your pension and find an island.

The punchline is that the single-benchmark story falls apart under scrutiny. Repeating the analysis across all 18 benchmarks Artificial Analysis tracks tells a very different tale: the average gap has stayed almost perfectly flat at just under five months for the entire period, and may even be widening for some tasks. Nearly all the dramatic convergence comes from coding, where open models closed from about 15 months behind to one or two. On most other benchmarks the gap holds steady or grows modestly.

The real takeaway is about measurement, not prophecy. Pick the Intelligence Index alone and you forecast open-source parity by Christmas; aggregate the full benchmark suite and you conclude open models are durably trailing by five months. The same underlying data supports opposite narratives depending on which yardstick you trust — a reminder that headline LLM ‘capability’ numbers are far softer than the confident extrapolations drawn from them.