CharlieKey takeaways from your favorite podcasts
← 2 October 2026Latent Space

Recursive Language Models — Alex Zhang, MIT PhD

Alex Zhang argues that harnesses are the underexplored lever: recursive language models, which offload context into code, generalize better, and academics should take big bets on simple ideas that look trivial.

Watch on YouTube ↗

Charlie's summary

  1. Human expertise still matters because a knowledgeable person can erase a trillion tokens of search

    On GPU kernels, Alex notes that almost every top leaderboard solution is AI generated. The one from a veteran kernel writer who steered the AI was the only top-10 entry stable end to end, and it had far fewer lines of code. Throwing compute at a problem explores it, but “someone who knows something about the problem” can uncover something that erases “one trillion token spent.” Efficiency still matters.

    ▶ 7:22
  2. A PhD student's edge is taking big bets on simple ideas that look trivial and that industry ignores

    Alex thinks the best academic work targets problems industry isn't looking at. Early reactions of “what is even the purpose” or “this is just sub agents” are “almost a good sign.” He cites SWE-bench, which nobody cared about until Devin, plus STaR, chain of thought and RLMs. Academia can't out-resource labs, so the advantage is freedom from bureaucracy. Expect most bets to fail.

    ▶ 17:26
  3. A language model doesn't have to be an autoregressive text-to-text decoder

    Alex says the new Jev model shows a new thing to tune: the output space. Calling a big model for a binary classification is a 400x cost that is silly. He thinks we will see models beyond “the bog standard frontier model,” and that labs won't take risky bets while scaling works, which leaves room for neolabs. He also argues that RLMs matter most if trained well.

    ▶ 21:13
  4. Harnesses can improve generalization: trained RLMs transfer strategies across tasks and lengths

    Alex says most harnesses (Claude Code, Codex, Pi) are essentially the same. In an RLM, sub-agents handle the varied local subtasks, so the top-level strategy looks the same across different tasks. Models trained on short tasks generalized to tasks 8 to 30 times longer, and across different task types. The argument is inductive: competitive programming and GPU optimization can share one strategy. Alex wants teams to test new harness designs with post-training.

    ▶ 32:59

Give your email to unlock the rest of the key points — for every episode.

Do more with Charlie

Free — sign in with your email, no password.