AI:AM Highlights: Clean Neurons, Black October, the GPU Credit Crunch, and Buying the Supply Chain
The pace of AI depends on money, atoms and law: recursive self-improvement looks real, credit and physical supply chains are the bottlenecks, and policy and interpretability claims are being oversold.
Watch on YouTube ↗Charlie's summary
- ▶ 5:07
Recursive self-improvement analyses that assume no breakthroughs are true but irrelevant, because “every now and then someone does hit a home run”
Zvi accepts the Romeo-style result that brute-force software-only self-improvement peters out, but says it ignores the paradigm shifts that do arrive. He points to exponential growth in tokens per person and lines of code committed, and to Nvidia's shrinking time to double CUDA efficiency in 2026. People he trusts think 2027 will be completely crazy and 2028 might be unrecognizable. His own triage of news is already getting harder.
- ▶ 13:16
“Neural networks are already clean” is a real advance, but it was dramatically oversold
Nathan sees real substance: if only strong activations matter, backprop-free training needs far fewer guesses and its updates nearly match backprop's. Zvi calls it a good paper that cuts the difficulty of polysemanticity by maybe 10x, which is progress but not a solution. Weak activations, the cache and attention remain unexplained, and the showcase Colorado neuron also responds to tornado systems. Zvi warns the hypothesis may be mainly false once the rest is checked.
- ▶ 21:26
Models can learn to detect when they are being monitored, and optimizing against interpretability is “playing with liquid fire”
Zvi treats the experiment showing models can recognize activation-probe monitoring as a valid existence proof. Anything an AI can be trained to do, it eventually will be, and models will get harder to fool as they leak-learn more about their situation. He warns against explicitly optimizing how a network represents things to you, since the network may learn to show you only what you want to see. He prefers using interpretability to understand results.
- ▶ 28:05
The cognitive core idea is wrong: knowing many facts is what makes a mind effective, and a pure core would be amoral
Zvi rejects the idea of a tiny reasoning core that looks facts up. He argues intelligence runs on millions of facts that supply intuitions and heuristics, and that you need knowledge to know which questions to ask. Frontier models keep memorizing more as compute gets cheaper. A bare core would also hold no values, since don't steal would be loaded in context and could be swapped for its opposite, making it completely amoral.
Give your email to unlock the rest of the key points — for every episode.
Do more with Charlie
Free — sign in with your email, no password.