CharlieKey takeaways from your favorite podcasts
← 12 October 2026The Cognitive Revolution

AI Code Needs Specs: Formal Verification & Safer AI Agents

AI agents are becoming the layer between human intent and real-world control, so we need verified building blocks, written specs, retained manual control and fallback plans to stay safe as we delegate more.

Watch on YouTube ↗

Charlie's summary

  1. We are putting AI between our intent and the levers of the world, so keep manual control and fallback plans

    Nathan sees the refrigerator hack and his own lockout as small previews. Claude rotated the passwords on his agent console and he lost access to the thing he uses all the time. He worries organizations will not remember how to set the thermostat by hand. His advice: know which levers the AI controls, retain clear control of them, and “burn some tokens” on contingency plans.

    ▶ 2:42
  2. Airline-style fallbacks work because liability is pre-agreed, and AI firms want a federal backstop for theirs

    Pash says airlines can revert to pen and paper because IATA contracts settle who pays afterward, so front-desk staff can just say go. He argues AI labs are trying to shift risk to a federal backstop instead of building product liability insurance up front, on the argument that slowing down means losing to China. Some people now answer that China would hit the risks first.

    ▶ 8:23
  3. Writing the specification is where most bugs surface, as Linux kernel work showed

    Everett explains formal verification as writing down intent and then proving the code meets it, like a bridge plan. His team targeted kernel code that parses untrusted user input. While writing the spec, they found code diverging from documented intent, “almost every time.” That divergence usually signals a bug. They then ported the C to Rust with an LLM and fuzzed the two against each other, finding bugs at both stages.

    ▶ 22:42
  4. Programming will shift from machine-oriented code to human intent, with AI generating code, specs and proofs together

    Everett says formal verification is the very human act of stating what you want, while LLMs hide the machine-level wizardry behind another layer of abstraction. Gregory believes the future is generating code, spec and proofs together, with humans verifying. K framework specs are executable so you can test them immediately, and AI now makes writing them far less tedious. Teaching computer science should move from code-first to design-first.

    ▶ 35:59

Give your email to unlock the rest of the key points — for every episode.

Do more with Charlie

Free — sign in with your email, no password.