The most misunderstood graph in AI just got a major update

Glass Epoch DeskOriginal digestSource: MIT Technology Review

AI-assisted digest, checked against the original source before publishing. How we work

METR's refreshed time horizon plot confirms four-month doubling on verifiable tasks, while judgment-heavy work — the kind that matters most for automating R&D — still lags human performance.
The most misunderstood graph in AI just got a major update
Photo: Markus Winkler · free stock license

The AI evaluation nonprofit METR has released an update to its time horizon plot, the widely shared and frequently misinterpreted chart that tracks how long a task frontier models can successfully complete. The new version keeps the headline trend intact: the length of tasks that frontier systems can finish has been doubling roughly every four months. But it also draws a sharper line between what models do well and where they still fall short.

The plot rates models against a suite of tasks that humans have also completed, with each task's difficulty scored by how long human testers needed to finish it. The updated suite adds work estimated to take people far longer than the originals, plus more open-ended assignments. That matters because the original chart left out exactly the fuzzy, judgment-laden tasks now folded in — the kind that will matter most if AI systems are to automate research and development.

The report's central contrast is between verifiable tasks and tasks requiring judgment. Reinforcement learning works well when success can be checked automatically: researchers let a system attempt a task many times, strengthening neural pathways tied to success and weakening those tied to failure. That recipe has made models strong at coding and math. It is far harder to apply when success is ambiguous or slow to confirm — choosing a new research direction, for example — which is why the first time horizon plot omitted such tasks entirely.

Concrete results from the new suite show the gap. In one task, models were asked to design AI evaluations that METR could use in its own research, with human METR employees scoring the output. Even though leading models now outperform nearly every human at software engineering, not one AI-designed evaluation was good enough for METR staff to consider useful. In another task, models scored philosophical arguments. Researchers do report movement: a handful of newer models now land borderline above the bar on evaluation design, up from clearly below a year ago, though they caution the improvements are too sparse to support predictions.

The update lands amid unsettled sentiment in the field. Earlier this month, a separate, heavily redacted METR report claimed AI might be accelerating capabilities progress at Anthropic by a factor of two, stoking fears of full recursive self-improvement — AI systems designing their successors without human involvement. Anthropic, which recently completed a blockbuster IPO, sits at the center of that debate.

What to watch is whether the sparse gains on judgment tasks coalesce into a measurable trend line, and whether the verifiable-task doubling rate holds as the suite moves toward longer, more ambiguous work. Both answers shape how seriously the industry can take claims about AI-driven R&D acceleration.

Key facts

  • • METR's updated plot adds longer and more open-ended tasks to its suite.
  • • Frontier models still double completed-task length roughly every four months.
  • • No AI-designed evaluation met METR researchers' usefulness threshold.
  • • A prior redacted METR report claimed a 2x AI progress speedup at Anthropic.
  • • Anthropic recently completed a blockbuster IPO, per the report.

Why it matters

Builders should stop reading the time horizon curve as a proxy for general capability: the same systems that dominate coding still can't produce research tooling METR staff would actually use. Expect evaluation design and other judgment-heavy benchmarks to become the differentiator in capability claims, and expect vendors to keep citing the four-month doubling figure while quietly omitting where i

Spotted an error? Send a correction.

Get the day's AI news in one email

The Daily Epoch · 60-second read · unsubscribe anytime

Related