Can Open Models Keep Up With OpenAI and Anthropic? | Nathan Lambert
Nathan Lambert argues that open models can fast-follow the closed frontier, driven by compute, data and talent, while fears of runaway takeoff are overblown and nearer-term societal risks are underprepared.
Watch on YouTube ↗Charlie's summary
- ▶ 2:01
Model progress comes down to three forms of capital: compute, data, and talent, and org charts decide who uses them well.
Nathan thinks closed labs have the most of all three, and that making the org chart work for building models is very hard. That is why companies like Meta do complete resets from the bottom up. OpenAI and Anthropic led because they made the earliest, most ambitious bets: OpenAI on reasoning and inference scaling, Anthropic on coding. His advice for anyone wanting to work at the frontier is to go to one of those two.
- ▶ 3:32
Chinese labs release open models as a practical way to get noticed, and fast-following will last longer than expected.
Nathan says that if Moonshot AI or Zhipu sold only an API from China, “nobody would know about them.” Open weights get them into the conversation. He expects tiers: the best closed models command top margins, while near-frontier models are open. Data companies resell what OpenAI and Anthropic paid for a year later, so open labs get similar data. He thought consolidation would come two years ago, and open labs have lasted much longer than he predicted.
- ▶ 8:34
Chinese labs match US talent but lack compute, and Nathan doubts distillation explains much of their progress.
Nathan says China is at least equal on talent, but a top Chinese pre-training researcher is stunned that an average OpenAI researcher has access to 5,000 H100 equivalents. Distillation is a way to spend money in place of compute. Yet open post-training recipes are stuck on an outdated Qwen model, because supervised fine-tuning on strong-model reasoning traces does not obviously work for modern RL-heavy reasoning. He estimates distillation helps by only a few months, with big error bars.
- ▶ 13:42
Cutting off distillation through policy would do more harm than good, so labs should secure their own products.
Nathan notes reasoning models expose thinking tokens that can be scraped, and as long as APIs are offered, distillation can't be stopped. Labs can restrict access to enterprise products if they want security. But he doesn't see why government should be involved in a terms-of-service violation, which you can only answer by cancelling access. He worries that punishing Chinese firms would harm the US-China dialogue that AI safety needs, and they won't stop anyway.
Give your email to unlock the rest of the key points — for every episode.
Do more with Charlie
Free — sign in with your email, no password.