Frontier Chips for Frontier AI Labs, with Walter Goodwin, Founder/CEO of Fractile
Walter Goodwin argues frontier AI inference needs chips with very high memory bandwidth and affordable capacity, built by lean full-stack teams that iterate fast, since frontier labs cannot safely bet alone on proprietary silicon.
Watch on YouTube ↗Charlie's summary
- ▶ 2:39
Most AI chips are near-identical because a few ASIC houses deliver them; few teams build the full silicon stack.
Walter notes that Google's TPU, Meta's MTIA, Microsoft's Maya and OpenAI's chip are built with a small number of back-end houses like Broadcom. All share HBM, tensor cores and TSMC advanced packaging. Few teams go from architecture down to physical design and foundry work, so truly new capabilities are rare.
- ▶ 7:18
Being full-stack with a skinny team creates an agile closed loop that a handoff to partners cannot match.
Fractile has about 150 people, spread thinly across front-end design, physical design, back-end implementation and packaging. Walter says this lets it place bets and strike fast, instead of being “at the mercy” of a partner after a handoff. In AI chips you need skill, luck and speed.
- ▶ 10:52
The fast-inference bet: fast chips are useless without capacity, so Fractile moved from SRAM to high-bandwidth DRAM.
Fractile began with an SRAM design like Groq or Cerebras, then worried about growing parameters and context length. Walter says fast chips today have tiny capacity, so long-context attention still flips back to a GPU. Fractile now works with memory vendors on much higher DRAM bandwidth, because data-center economics collapse down to cost per gigabyte. Fast chatbots are the faster horses; the real prize is long-running agents.
- ▶ 18:47
Don't ship a new chip every few weeks; compress design time to get more shots on goal.
Workloads churn fast, but fab turnaround takes three to five months and chips need a 3 to 5 year amortization window. Walter doesn't believe in constantly shipping new chips. Shorter design cycles instead let you keep a rolling frontier of bets and trigger the right ramp, as Nvidia's six to nine custom chips per system show.
Give your email to unlock the rest of the key points — for every episode.
Do more with Charlie
Free — sign in with your email, no password.