CharlieKey takeaways from your favorite podcasts
← 14 October 2026The Pragmatic Engineer

Uber’s AI infra, from the inside

Uber's AI-native engineering works because a decade of developer platform investment (gateways, cloud dev environments, tracing-based testing) could be retrofitted for agents; the harness is the easy part.

Watch on YouTube ↗

Charlie's summary

  1. Uber's years of pre-AI developer platform work is what made agents practical to retrofit

    Udai describes dev pods, cloud developer boxes built around 2017-18 for the monorepos, as an early bet that paid off. Builds and tests are standardized, so a top-level agents.md works for most teams. Agents get the same warm, pre-built workspaces as humans, with Claude Code, Codex and others preinstalled. He also built a mega dev pod with all repositories so agents can open PRs across repos. Uber's lesson: the harness is easy; the infrastructure around it is hard.

    ▶ 16:18
  2. Put one model gateway in front of every model so guardrails, spend and audit are enforced in one place

    Uber exposes a single OpenAI- or Anthropic-compatible API over all vendors, open or closed weight, built on the Michelangelo ML platform. Udai says every model request at Uber goes through it. The first addition was AI Guard, fine-tuned safety models that run on every request and response. Later came per-user attribution, spend tiers, audit logging and session traces. Because the landscape changes quickly, Udai's rule is to invest in building blocks that are harness- and model-agnostic.

    ▶ 19:25
  3. Wrap every coding harness in one layer (AIX) so engineers can switch tools without losing settings or skills

    AIX wraps Claude Code, Codex and other harnesses, and comes preinstalled on dev pods. Personas bring default plugins and skills, and settings are managed centrally and synced across environments. A status line shows tips and spend within the session. Udai says the point is to let engineers switch harnesses without carrying over configuration manually. Spend is handled with tiers: moving up needs only a manager conversation about ROI, since Uber wants to maximize gain, not restrict use.

    ▶ 25:03
  4. Auto-generate an MCP gateway from existing service IDLs instead of hand-building tool integrations

    Starting around April 2025, Uber built one gateway for all tools. Because every Thrift or protobuf service already sits in a single IDL registry, MCP specs are generated automatically, with LLM-optimized descriptions (some tools ran 3,000-4,000 tokens). Teams enable a server with a config and a security-policy change, no code. Vendor tools like Slack and Google go through it too, and MCPs are projected as CLIs. Udai says the same team that built the API gateway built it quickly.

    ▶ 29:39

Give your email to unlock the rest of the key points — for every episode.

Do more with Charlie

Free — sign in with your email, no password.