CharlieKey takeaways from your favorite podcasts
← 22 October 2026Latent Space

Why AI Compute Comes in Rectangles — Erik Bernhardsson, Modal

Erik Bernhardsson argues AI infrastructure needs flexible, aggregated compute rather than rigid multi-year GPU blocks, and explains how Modal's products, capacity strategy and company-building choices follow from that.

Watch on YouTube ↗

Charlie's summary

  1. New computing eras need new tools: GPU workloads outgrew Kubernetes and Docker, so Modal rebuilt the stack

    Erik's thesis is that every technology era hits the limits of old tooling. Kubernetes and Docker were built for steady, low-utilization CPU backends. GPU workloads have expensive hardware, large models loaded over the network, and bursty demand. Modal built its own file system, CPU and GPU snapshotting, autoscaler and scheduler. Erik thinks you “can't build like a wrapper on top of the old stuff” for this generation of software. Stable Diffusion became the early killer app: revenue went from $100 a month to $30,000 in two months.

    ▶ 6:42
  2. Treat capacity as one global pool: going multi-region early was a fateful decision

    Erik says Modal made capacity management a single worldwide pool rather than separate clusters per region, and went multi-region very early. That choice let it aggregate spot capacity from hundreds of regions. It worked until the cloud market tightened and Modal grew fast. Modal now works with almost 25 clouds, often on three-plus-year deals with large upfront payments. It built a procurement desk of four people and a ten-person bare-metal engineering team.

    ▶ 12:24
  3. Rent rather than buy GPUs, and fear collapsing prices more than rising ones

    Erik prefers staying a lean software company and renting, citing capital efficiency and the operational and financial complexity of owning. Rising prices don't worry him, because Modal's reservations leave it effectively “long GPU prices.” Collapsing prices would hurt anyone locked into expensive supply. Hardware prices eventually fall like gravity. He sees the Hunt brothers' silver squeeze as a warning against cornering the market, so Modal ramps aggressively but only to match demand growth.

    ▶ 17:31
  4. Smaller, riskier neoclouds are Modal's opportunity, because the big labs buy from the tried and true

    Erik speculates that Anthropic and OpenAI prefer established vendors and large blocks for training runs. That leaves plenty of smaller capacity, such as 32 nodes in Armenia or 128 in Indonesia, at good prices. Modal vets vendors with a spec sheet covering IOPS, CPU and bandwidth, runs acceptance tests, and keeps an internal score for reliability and support responsiveness. He also expects 2027 might finally be the year of non-Nvidia accelerators.

    ▶ 21:34

Give your email to unlock the rest of the key points — for every episode.

Do more with Charlie

Free — sign in with your email, no password.