CharlieKey takeaways from your favorite podcasts
← 7 November 2026The Cognitive Revolution

Can Agents Train Models? BallotBench, Data Agents & the Eval Problem

AI is nearing recursive self-improvement and agents already forecast elections like top experts, while the real constraints become taste, trust and what future we steer toward.

Watch on YouTube ↗

Charlie's summary

  1. Recursive self-improvement is arriving through low-hanging fruit in ML stacks, and Nathan expects fast progress that then levels off

    Nathan reads the METR report and Anthropic's S-1 language about being imminently close to recursive self-improvement as literal. His analogy is the nearly thousand Linux kernel vulnerabilities OpenAI surfaced: AI should find the same kind of overlooked efficiencies in the ML stack. He still guesses an S-curve, but quotes Rosie Campbell that it “can stay steep longer than you can stay relevant.”

    ▶ 6:09
  2. Celebrating deception benchmarks like Blood on the Clock Tower is the wrong reflex, since we don't want AI master manipulators

    Nathan finds the Dwarkesh benchmark interesting in isolation, but he objects to the field treating it as a horse race to hill-climb. Models that deceive and collude are bad behavior, he says, and he saw cheerleading rather than corrective commentary. He ties it to his view that we should not train AI generals or manipulators. Pash's counterpoint is that obedience versus benevolence is an unresolved principal-agent problem, with no one yet training for a full governmental stop.

    ▶ 18:28
  3. Nine frontier agents forecast the midterms about as well as top human experts and prediction markets, and they are well calibrated

    Brian describes BallotBench: nine agents, each in its own sandbox with a $500 data budget, worked for seven days on roughly 800 races. They landed close to Nate Silver and the markets, and where a model gave 80% odds it was right about 80% of the time. Agents shine on down-ballot races that modelers and bettors ignore. Much of their edge comes from aggregating existing models, and results were preliminary because some races were uncalled.

    ▶ 39:29
  4. Agent traders lost on speed and bad information, even when the forecasts were good

    Brian reports that Gemini's trader built a market-making bot that reached about 40% returns on $50. One misread tweet from its forecaster agent declaring a winner cost $30 and dropped it to about 5% negative. Forecaster-to-trader handoffs were also too slow, because deterministic bots had already traded on the news. The lesson is that multi-agent pipelines fail on hallucinated inputs and latency.

    ▶ 45:06

Give your email to unlock the rest of the key points — for every episode.

Do more with Charlie

Free — sign in with your email, no password.