CharlieKey takeaways from your favorite podcasts
← 18 November 2026a16z

What It Takes to Build an AI That Actually Knows You

Ken Liu argues that a single, secure personal context layer built from your computer's own event stream can let any agent know you and act proactively, instead of each agent relearning you.

Watch on YouTube ↗

Charlie's summary

  1. Portable context: one user model that every agent can query instead of re-asking you

    Ken says agents today force you through rounds of questions about seat preferences or arrival times. Second builds a model of you in the background, covering priorities, preferences, goals and commitments, and shares it with agents such as Grockbot or Codex instantly. Ken's Alfred and Batman analogy: you trust the butler because he has a good model of you and just gets things done.

    ▶ 0:59
  2. People underestimate how much your computer already knows

    Ken notes that the OS already exposes an endless stream of events: app switches, pauses, typing, Wi-Fi state, across Apple, Microsoft and Linux. Second runs agents on this raw stream around the clock, first interpreting single events, then hierarchically building up an understanding of what you are doing. Together with connectors, this gives a rich picture that is “sitting there you know waiting to be tapped.”

    ▶ 4:32
  3. Reactive versus reflective: the harder, more valuable AI connects dots across months

    Ken separates reactive productivity (an email came in, so ping me) from reflective productivity, which links events over long horizons: what you said months ago, your current goals, your to-do list, your preferences. Users Martin and Joel describe unprompted texts, such as weather updates for event planning. Ken thinks interfaces should intervene without being asked: “You're not supposed to ask things” of a superintelligence.

    ▶ 6:34
  4. You want one Second, not many: a single trusted vault with ephemeral sub-agents

    Ken thinks having many personal agents is transitory, and agents will specialize, in shopping for example. Second is meant to be the single consistent store of your life's context, “like a vault,” that guards your data and aligns with your interests. You can spin up temporary sub-agents, but the persistent store is where the value lies.

    ▶ 9:04

Give your email to unlock the rest of the key points — for every episode.

Do more with Charlie

Free — sign in with your email, no password.