OpenAI’s chief scientist has thoughts on how to limit AI development

Glass Epoch DeskOriginal digestSource: TechCrunch AI

AI-assisted digest, checked against the original source before publishing. How we work

OpenAI chief scientist Jakub Pachocki proposes perplexity as the shared metric for a possible AI speed limit.
OpenAI’s chief scientist has thoughts on how to limit AI development
Photo: Jonathan Kemper · free stock license

OpenAI chief scientist Jakub Pachocki published a blog post arguing that the industry could anchor limits on AI development to perplexity, a measure of how accurately a model predicts the next token on average, where lower values mean better predictions. The post arrives as calls for slowing frontier development have moved from researcher statements into formal company documents, and it positions a technical metric as the missing prerequisite for any enforceable pace.

The timing reflects a broader shift in the debate. A former OpenAI and Anthropic researcher said the companies were "gambling with our lives," and Anthropic's IPO filing itself suggested the technology could pose "catastrophic or existential risks to society." Anthropic CEO Dario Amodei subsequently published a plan to "pace the frontier," which appears to have drawn endorsement from OpenAI CEO Sam Altman and SpaceX CEO Elon Musk. Amodei's proposal asked companies to agree to "common safety standards as well as limits on the rate of unchecked AI progress." Altman described Pachocki's post as "the most important thing OpenAI has published in a while."

Pachocki's core argument is that conventional benchmarks lose value quickly. He contends that evaluations focused on AI's "downstream capabilities" become outdated as the technology advances, whereas perplexity has remained "informative" across model generations. The post includes a chart tracking falling perplexity across OpenAI's models since GPT-3.5, framing the metric as a durable yardstick rather than a snapshot.

He attaches specific readings to that trajectory. "GPT‑4, at ~0.91, was proficient at utilizing chain-of-thought reasoning productively, which motivated the subsequent targeted elicitation of this capability and the creation of modern reasoning models," he wrote, adding that losses below 0.8 have coincided with "the emergence of transformative capabilities in fields like cybersecurity and mathematics." From there he proposes perplexity as the basis for "a shared benchmark for general intelligence" that companies could use to set "common limits," agreeing not to cross a set threshold "until our ability to understand and monitor AI behavior achieves a certain concrete milestone."

What remains unresolved is whether rivals would accept OpenAI's chosen yardstick. The proposal requires agreement on both a numeric threshold and the monitoring milestone that would unlock further progress, neither of which the post specifies. Watch for responses from Anthropic and other labs, whether any regulator picks up the framing, and whether a metric that has tracked capability gains could plausibly function as a brake rather than a scoreboard.

Key facts

  • • Pachocki's proposal follows Amodei's "pace the frontier" plan and Altman's endorsement.
  • • Perplexity scores fell from about 0.91 at GPT-4 to below 0.8 recently.
  • • Benchmarks targeting downstream capabilities degrade too quickly to measure progress.
  • • Anthropic's IPO filing warned of catastrophic or existential societal risks.

Why it matters

Builders should expect lab commitments to be framed around a single predictive-loss number rather than capability benchmarks, which shifts compliance work toward eval and monitoring infrastructure. If a threshold-based pause gains traction, release schedules and compute planning could be gated by monitoring milestones that no lab has yet defined.

Spotted an error? Send a correction.

Get the day's AI news in one email

The Daily Epoch · 60-second read · unsubscribe anytime

Related