Individual neurons in MLPS in neural networks in general are more interpretable than the field has thought to this point. They are very difficult to interpret. They're complicated objects. They were very confusing in the initial like days of interp and because of that I think it became not very common to study them in detail. there always has been a group of people like trying to do some interp on individual neurons but yeah I think it just was seen as a very difficult thing and that this representations would be distributed against across many neurons rather than localized in individual ones. So our message is that we can understand them that you often don't need very many active at once and that even when underon has more than one meaning which can be very confusing. It's usually not like an unbounded number of hundreds and hundreds of meanings that are just like very difficult to understand. Rather it's like a compact set of meanings.
← 11 November 2026The Cognitive Revolution
The Neuron Basis: Goodfire's Curt Tigges on Agentic Interp, Monosemanticity & Reverse Engineering
Curt argues individual MLP neurons are far more interpretable than assumed: sparse, mostly single-function, scalable to label with agents, and a promising base for monitoring and eventually ambitious reverse engineering.
Watch on YouTube ↗Word for word
Things that push the model towards more sparity will be useful. I also think localization is important too. there's one paper I think it's from AE Studios that talks about like localizing for example potentially harmful biological knowledge or W and D knowledge in like parts of the model so that you can just like not have those working or just not even have those be part of the public release of the model. so if with our knowledge of neurons and expert specialization we can sort of incentivize the model to put buckets of knowledge in a particular place then we can just kind of ablate that when necessary.
Do more with Charlie
Analyze any YouTube videoPaste a link — get its key points and best quotes in minutes. 10 videos a day.Paste a YouTube link →Your own podcast feedAdd up to 10 YouTube channels you follow. Charlie reads their new episodes for you.Add your channels →
Free — sign in with your email, no password.