CharlieKey takeaways from your favorite podcasts
← 17 November 2026The Generalist

Baseten’s CEO: Inference Is AI’s Final Market

Tuhin argues inference is AI's final market, far harder than it looks, and that companies will move from closed models to owned, post-trained model portfolios while compute stays scarce.

Watch on YouTube ↗

Word for word

Inference in a lot of ways I think is maybe like one of the most like misunderstood markets in the world cuz people think you just take VLM, which is open source framework, you slap it on some GPUs and you call it a day. And supply can trade market operating at large scale with what I would call quite flaky technology, which is GPUs fail all the time. chips fail all the time. and running at very high utilization in a market where the demand is growing exponentially. There's so much complexity at every layer of this.
There is capex involved, whether that means building data centers yourself or whether that means… If you are looking at a couple of billion dollars of compute expenditure over a given period, in today's market that probably requires about a five-year term and a 30% prepayment.

Do more with Charlie

Free — sign in with your email, no password.