CharlieKey takeaways from your favorite podcasts
← 24 October 202620VC with Harry Stebbings

Inference Will Be $10TRN Market: What No One Sees | Software Investors are F***ed | Naveen Rao

Naveen Rao argues energy is AI's binding constraint, that efficiency gains now come from co-designing models and hardware, and that intelligence demand is effectively unbounded, making hardware the place to invest over software.

Watch on YouTube ↗

Charlie's summary

  1. Energy is the real bottleneck: a gigawatt costs about $50 billion to build, and spare grid capacity is running out

    Naveen says people throw around gigawatt as if it were easy, yet industry figures put one at about $50 billion, more than a $20-30 billion chip fab. Recent data center growth has soaked up excess grid capacity (about 900 GW average output against a 1.2-1.3 TW peak). He expects that to run out within a few years, and thinks claims of building 40 GW in a year are impossible; 10 GW (half a trillion dollars) would be amazing.

    ▶ 1:35
  2. Data centers already pay a premium for power, so behind-the-meter energy deals are the early sign of scarcity

    Naveen sees two-tier pricing emerging: bespoke behind-the-meter deals, like Bloom Energy's multi-gigawatt project in Arizona, cost far more per kilowatt hour than the standard grid. Compute can sell for roughly 5x the cost of its electricity, so buyers can tolerate paying 30-40% more. He calls the current moment a blip, not a bubble bursting, because demand for intelligence is so strong.

    ▶ 7:16
  3. Physical efficiency gains have stalled; the remaining gains are in co-designing the model with the hardware

    Naveen splits computing into three layers. Transistor scaling has largely stopped delivering gains. Recent gains came from smaller data types (32-bit to 4-bit). The untapped layer is enforcing locality and cutting data movement at the model level. Unconventional is energy first and doesn't treat the model as sacrosanct. Computers were historically built to be faster, a lineage going back to ENIAC. Brains co-evolved intelligence and hardware, and he wants to do the same.

    ▶ 9:19
  4. Intelligence has infinite demand, so chips won't commoditize: hardware is the business of converting energy into intelligence

    Naveen argues the universe is adversarial, so intelligence is the fuel of every contest, from bacteria to nations. Making it requires energy, and the conversion of energy to intelligence is the whole hardware stack: chips, systems, cloud, networks. The better a company does that conversion, the more value it accrues. He therefore doesn't expect commoditization anytime soon, and says the hardware is the place to be over the next 10 years.

    ▶ 20:37

Give your email to unlock the rest of the key points — for every episode.

Do more with Charlie

Free — sign in with your email, no password.