Why Nick Bostrom Thinks Superintelligence Could End Aging
Bostrom argues for flexibly pacing the AI frontier rather than stopping it, since safe superintelligence could end aging while a permanent stagnation would itself be an existential catastrophe.
Watch on YouTube ↗Charlie's summary
- ▶ 1:00
We know more puzzle pieces than before, but one unknown still leaves us in the dark
Bostrom says we now understand more of the strategic picture than decades ago, but the pieces don't add up to a reliable opinion. His analogy: you must head north if the sum of ten binary digits is even, south if odd. Decades ago we knew three digits, now maybe seven. Until you know all of them, you are “completely in the dark.”
- ▶ 8:04
Pacing the frontier: allow short pauses, but don't build machinery that makes AI stagnation permanent
Bostrom isn't enthusiastic about stopping or pausing AI, but likes “pacing the frontier”: a time-limited pause for safety work, or coordinated slowing if alignment trails. A lab that can't slow for more than a few weeks without becoming irrelevant is a bad setup. But if stigma and a global regulatory apparatus became entrenched, the result could be a permanent failure to develop superintelligence, which he would count as an existential catastrophe.
- ▶ 13:13
Alignment gets easier and harder at once as models get smarter, and on balance harder
Smarter AIs grasp the gist of what we mean. But behavioral testing breaks down once models are situationally aware: Bostrom notes they can detect tests and act differently, and current frontier systems already strategize in their chains of thought. They might also shape their own training or reason invisibly. His main hope is to quasi-align slightly superhuman systems, then have them help design more aligned successors, reaching a self-reinforcing attractor.
- ▶ 16:15
If nobody builds superintelligence, everyone dies too: the optimal delay is likely months or a few years
Responding to “if anyone builds it, everyone dies,” Bostrom notes that 95% of everyone who ever lived is already dead. Without aging, life expectancy might be 1,200 to 1,400 years. His model, focused only on the life expectancy of existing people, weighs safety gains from delay against roughly 150,000 deaths a day. Delay pays off only if safety improves at an intermediate rate. For a wide range of parameters, the optimal delay is short. He stresses this is one consideration among many.
Give your email to unlock the rest of the key points — for every episode.
Do more with Charlie
Free — sign in with your email, no password.