How do AI models actually work? | The scientist looking inside Claude's mind
Emanuel, an Anthropic interpretability researcher, argues that language models are grown rather than built, that looking inside them reveals concepts and planning, and that studying them is essential to alignment.
Watch on YouTube ↗Charlie's summary
- ▶ 1:31
Models are grown, not built: to make them is not to understand them
Emanuel says engineers build a scaffold, pass huge amounts of data through it, and the model grows “in inscrutable ways.” The part that is grown is the math: formulas the model learns, which nobody wrote or understands. His team's job is to decode what those formulas and their intermediate results mean, closer to biology than to software engineering.
- ▶ 7:37
Next-word prediction is deceptively hard, so it forces models to understand the world
Emanuel says people picture iPhone autocomplete, but once models predict easy words well, only hard ones remain, like a physics answer or a thesis's final paragraph. His example: to say the capital of the state containing Dallas is Austin, you need Dallas is in Texas, and states have capitals. Inside the model, a Texas concept lights up and combines with a capital concept. He sees the simple objective as hiding real structure.
- ▶ 14:46
Models plan ahead even though they output one word at a time
Emanuel explains that intermediate results from earlier words can be reused by all later words. In poetry, the model picks a rhyme target like rabbit in its intermediate results, and later words look back at that plan, without saying it aloud. Next-word prediction and planning are not in conflict: predicting well means anticipating what comes later.
- ▶ 19:20
Interpretability works like brain scanning: record internal numbers, find patterns, then intervene
Inside a model is just lists of numbers. Emanuel's team runs lots of text through it, records the intermediate numbers, and looks for patterns that appear only for English, love, or eyes. This is the equivalent of scanning a brain while showing images. Then they test guesses by secretly changing results mid-computation and checking that behavior shifts as predicted. Golden Gate Claude came from turning one such concept on permanently to confirm it was decoded correctly.
Give your email to unlock the rest of the key points — for every episode.
Do more with Charlie
Free — sign in with your email, no password.