OpenAI Chief Scientist Warns AI Is Racing Toward Recursive Self-Improvement

In an essay titled “An Alien Mind,” OpenAI chief scientist Jakub Pachocki argues that today’s frontier systems are the product of compute scaling far more than deliberate engineering — models that are, in his framing, “grown more than designed.” That distinction is the core of his worry: capabilities emerge from repeated optimization over enormous compute that even the builders do not fully understand, producing an intelligence whose inner workings are genuinely foreign. He traces the shift to mid-2023, when scaling let models generate their own reasoning chains; within three years those systems can operate computers and contribute to scientific research, and he sees reasoning models now accelerating toward recursive self-improvement.

Pachocki’s central safety claim is blunt: no lab, including OpenAI, has solved alignment and monitoring well enough to keep scaling at maximum speed responsibly for much longer. He separates goal alignment (following instructions) from value alignment (holding to principles without supervision), and warns that a key safeguard — chain-of-thought monitoring — is eroding as models blend reasoning with tool use, making their intent harder to inspect. Beyond technical alignment, he flags governance and power-concentration risks, including the unresolved question of who builds and audits defensive AI when offense and defense originate in the same labs, and the need to preserve human agency and control over the transition.

The piece functions as both a warning and a signal of intent from inside a leading lab. Pachocki says OpenAI will “withhold scaling unilaterally as needed” and calls for stronger safeguards and international coordination — while notably stopping short of naming concrete triggers, benchmarks, or thresholds that would actually trigger a pause. Coming from the person running OpenAI’s research, the essay is significant less for any new technical disclosure than for how candidly it concedes that current control techniques may not keep pace with the systems they are meant to govern.