Inference in a lot of ways I think is maybe like one of the most like misunderstood markets in the world cuz people think you just take VLM, which is open source framework, you slap it on some GPUs and you call it a day. And supply can trade market operating at large scale with what I would call quite flaky technology, which is GPUs fail all the time. chips fail all the time. and running at very high utilization in a market where the demand is growing exponentially. There's so much complexity at every layer of this.
← 17 November 2026The Generalist
Baseten’s CEO: Inference Is AI’s Final Market
Tuhin argues inference is AI's final market, far harder than it looks, and that companies will move from closed models to owned, post-trained model portfolios while compute stays scarce.
Watch on YouTube ↗Word for word
There is capex involved, whether that means building data centers yourself or whether that means… If you are looking at a couple of billion dollars of compute expenditure over a given period, in today's market that probably requires about a five-year term and a 30% prepayment.
Do more with Charlie
Analyze any YouTube videoPaste a link — get its key points and best quotes in minutes. 10 videos a day.Paste a YouTube link →Your own podcast feedAdd up to 10 YouTube channels you follow. Charlie reads their new episodes for you.Add your channels →
Free — sign in with your email, no password.