Panic on a schedule
OpenAI called GPT-2's 1.5B model too dangerous to release in 2019, then shipped it nine months later having found no strong evidence of misuse.
In February 2019 OpenAI described GPT-2 and then declined to ship it. The largest configuration had 1.5 billion parameters, ten times GPT-1, trained on 40GB of web text, and the company released only a much smaller model plus a technical paper, citing concern about malicious applications. It called this an experiment in responsible disclosure. Nine months later, on November 5, 2019, OpenAI released the full 1.5B model with code and weights, reporting that it had seen no strong evidence of misuse.
The hold-back drew attention partly because GPT-1 had shipped the year before with no such ceremony, which left people to assume GPT-2 must be dramatically more capable. Architecturally it was not a new thing. GPT-2 is a direct scale-up of GPT-1: the same transformer decoder stack, more parameters, more and more varied training text. The biggest version stacks 48 decoder blocks with an embedding dimension of 1600. The original transformer used six blocks at 512. The capability people worried about came from scale and data, with the GPT-1 work having already shown that a pre-trained model carries enough task knowledge in its weights to perform without supervised fine-tuning.
So the thing OpenAI held back was a bigger version of a model it had already published, and the danger it named was the quality of the output rather than any specific exploit.
Over the nine months, OpenAI ran the staged release and reported what it learned:
- People find GPT-2 output convincing.
- The model can be fine-tuned for misuse.
- Detection is hard. A RoBERTa-based detector caught about 95% of text from the 1.5B model.
- No strong evidence of actual misuse appeared.
- The field needs standards for studying bias.
Ninety-five percent reads like a pass until you run it against real volume: one in twenty generated passages slips through, and that is against a known model with a detector tuned for it. OpenAI still called detection challenging, which is the honest reading. The fine-tuning point is the one that aged well. A general model that behaves on release can be adapted toward a narrow abusive task by whoever downloads the weights, and that holds for open weights in general, not for GPT-2 alone.
What the episode produced was a template. OpenAI framed it explicitly as a test case for developers of future powerful models and for publication norms in AI: announce, release a small version, watch, release the rest when the evidence supports it. The staged schedule gave outside researchers something to study and gave the lab a window to look for abuse before the full weights went public.
Some of it carried into later systems. OpenAI applied lessons from GPT-2 to ChatGPT, for example refusing to impersonate specific people. Writing in December 2022, with ChatGPT already in front of everyone, the author of this account judged GPT-2 mild by comparison. The staged release did not solve the harder problems. Students getting a model to do their homework is the obvious one, and detection against it is getting harder as the models improve.
Keep Reading
AI safetyOpenAI Safety Firings Raise Control Questions
OpenAI's firing of three safety researchers shows how unclear research-handling rules can weaken AI safety review and accountability.
AI safetyAI labs racing to look the most dangerous
A satire imagines AI labs racing to look the most dangerous, and the real point under it: the vendor telling you how dangerous a model is, is selling it.
WindowsMicrosoft's November update broke SLMGR activation, fixed Friday
Microsoft shipped out-of-band updates on Friday to fix Windows activation failures its November 2026 security updates caused for SLMGR users.
Latest on the Wire
Full wire →- AI Coding Agents Boost Output But Code Review Bottlenecks IncreaseHacker News
- AI Overviews Cut Organic Traffic by 40%Hacker News
- Apache Patches Critical Roller RCE VulnerabilityBleepingComputer
- Complete Web Browser Built from Scratch in PythonHacker News
New signal daily · RSS
Stay in the loop
New writing delivered when it's ready. No schedule, no spam.