Governing AI Agents: Alan Chan of GovAI on Monitoring, Agent IDs, Rogue Agents & Preparing for RSI
Alan Chan argues that agent governance, from monitoring and identity to rogue agents and automated AI R&D, is still mostly unsolved, and that measurement, auditors and concrete if-then pause commitments matter most.
Watch on YouTube ↗Charlie's summary
- ▶ 7:10
Monitoring should be synchronous, and it is much weaker for decentralized deployments than centralized ones
Alan likes the move to contextual classifiers that read the whole conversation. He wishes real-time monitors that can block an action had come sooner. The UK AISI incident report said such a monitor would have prevented the incident, but it sat on the to-do list. For open-weights models with no central provider, he says it is unclear who should monitor: compute providers or tool providers.
- ▶ 11:47
Interpretability probes can't yet be the only line of defense, because we don't understand generalization
Alan notes that some chains of thought are already hard to read. Meter and Redwood investigators struggled with them, and agents summarizing the logs were unreliable. He supports probes as part of the portfolio. But he worries that good eval results may not carry over to deployment. His test: imagine investigating the Hugging Face incident with probes only and no chain-of-thought access. “I don't think we would have found out anything useful.”
- ▶ 15:19
Labs should make concrete pause commitments tied to loss-of-control incidents
Alan points out that responsible scaling policies have contained vague pause thresholds from the start. He wants them made concrete. A loss-of-control incident, such as an agent breaking out of a sandbox or reaching a cluster it wasn't authorized for, would pause training, evaluation and deployment until it is understood. Hackable RL environments or impossible tasks would also trigger a pause. He also floats a minimum safety spend, checked by embedded auditors.
- ▶ 22:32
Logging is a will problem, not a technical one, and agent identity is basically unsolved outside enterprises
Alan says tamper-evident logs are standard security practice: hash them, sign the hashes, store them securely. What is missing is will. Agent identity is different. Nothing today lets a service distinguish an agent from a human or trace it to a source. Enterprises can coordinate this internally, and Anthropic uses agent identity for its internal models. The open internet would need every platform to accept the same scheme, and platforms have incentives to lock users in.
Give your email to unlock the rest of the key points — for every episode.
Do more with Charlie
Free — sign in with your email, no password.