CharlieKey takeaways from your favorite podcasts
← 10 October 2026The Cognitive Revolution

How Agents Decide: Goodfire's Eric Bigelow on Critical Tokens, Phase Shifts, & In-Context Learning

Eric Bigelow argues that LLM decisions largely come from sampling at critical tokens, with in-context learning shaping behavior, and that interpretability is badly underfunded relative to its importance as chain of thought becomes less readable.

Watch on YouTube ↗

Word for word

Reasoning is almost like a misnomer for what reasoning models are doing. I think traditional chain of thought reasoning really was like human reasoning in some way where it's like let's break it down step by step and here's a set of propositions A therefore B therefore C therefore D but reasoning model will just enumerate A B C D A E FGA all these different it's like a it's almost like a soup of tokens they create and then they just go back and skim out something out of that soup
★ In the digest▶ 44:55
I think when people say in context learning historically it's meant this sort of traditional few shot give a few shots of input output examples and see how the model behavior changes and you can do these like scaling curves for example where you like increase the number of shots that you give and see how model import performance improves. moves as a function of this. But then I think the way I think of in context learning and I think there's a few there's a few other people that have echoed this and I one paper that I like around this is by Andrew Lamin and others on the I think it's called the broader spectrum of in context learning where I think what I think of in context learning is capturing is everything the model is doing to adapt its behavior that's not happening with in weights learning.

Do more with Charlie

Free — sign in with your email, no password.