Is AI safety solvable in the time we have left? | Decoder
Helen Toner argues that AI labs are really racing toward recursive self-improvement and superintelligence, that alignment is unsolved and may not be solvable in time, and that policy should buy time without relying on one trustworthy person.
Watch on YouTube ↗Charlie's summary
- ▶ 13:07
Frontier labs are not consumer tech companies: they are trying to build superintelligence through recursive self-improvement
Helen says OpenAI and Anthropic were founded to build machines smarter than people, and the chatbot apps are a side effect. She describes recursive self-improvement (RSI) as AI taking over parts of AI R&D. Labs now use AI for more of their R&D pipeline and report seeing acceleration. Helen sees value in explaining plainly that this is the goal, even if it might not work.
- ▶ 17:41
AGI is no longer a useful milestone; we are already inside the fuzzy cloud of definitions
Helen says AGI was a useful label ten years ago, when systems could do only one thing and general ability was far off. Today some definitions are already met and others remain far away. Arguing over whether a given model counts as AGI wastes effort. She suggests dropping the term and naming the specific capabilities people mean.
- ▶ 23:52
Don't assume traits like sentience, autonomy and brittleness travel together in AI
Helen says building AI keeps showing that qualities we assumed were linked are not. Chess engines did not learn deep strategy, and 'it's just math' did not stop models adapting on the fly once trained on long-horizon problems. She warns against conflating sentience with the risk of causing damage. She also says it is unsolved whether Claude is conscious, and that guessing wrong in either direction would be a disaster.
- ▶ 27:28
'Alignment' means several incompatible things, and the most coherent one, doing what the user wanted, is not on track to be solved
Helen says alignment is used to mean company guardrails, doing exactly what the user wants, or being morally good in a broad sense, and these are wildly different things to build. She thinks the most coherent is getting the AI to do what the user intended, and as far as she can tell that is not on track to be solved. Sandboxing is no fix, because people want to give agents more autonomy and permissions.
Give your email to unlock the rest of the key points — for every episode.
Do more with Charlie
Free — sign in with your email, no password.