Jev Creator on Will US Open Models Catch Chinese | The Cost of Intelligence & Tokens | Diogo Almeida
Diogo argues that unreliable intelligence, not capability, blocks AI automation, that the right training task and data matter most, and that benchmarks, lab spending and enterprise model-ownership narratives are largely misleading.
Watch on YouTube ↗Charlie's summary
- ▶ 1:37
Reliability of intelligence, not capability, is the bottleneck to automation: you get Einstein 95% of the time and Mr. Bean the other 5%
Diogo says models solve Millennium Prize-level math yet fail at accounting, because their output is extremely high variance. By expected value, companies can't automate on that basis. He says this isn't a diffusion problem. He doesn't mean uptime, but non-jaggedness of intelligence. Running in the background needs “nines of intelligent reliability.”
- ▶ 4:42
AI's bitterest lesson: the right task matters most, then data, then compute, then algorithms
Diogo extends Sutton's bitter lesson. He thinks scaling plateaued at GPT 4.5 and that data matters far more than compute. The two existing tasks, RLHF (please humans) and RLVR (score high on benchmarks), don't produce software that runs in the background. You get what you optimize for. Choosing the right task is also what unlocks the data, since that data usually doesn't exist anywhere until you define and build it.
- ▶ 8:47
Valuable training data isn't sitting in corporate data lakes; it has to be created for a specific task
When Diogo's team made RLHF, human preference data comparing LLM outputs existed nowhere, so they had to define it with labelers. He says good data depends on how strong the current models are, so data makers need frontier-model access. He doubts enterprise data lakes are shaped correctly. Synthetic data is a tool, not an idol: its value depends on how the team uses it.
- ▶ 15:01
Most neolabs are incredibly wasteful, and the AI money furnace isn't justified by expected value
Diogo describes the neolab model as hiring ex-lab people, buying GPUs, having fun and getting acquired. He thinks that wastes resources, though pouring money into RL still makes models smarter. He wouldn't invest his own money that way and says the expected-value math doesn't justify it. He allows that VC math could work if a 1% moonshot chance pays off.
Give your email to unlock the rest of the key points — for every episode.
Do more with Charlie
Free — sign in with your email, no password.