CharlieKey takeaways from your favorite podcasts
← 28 October 2026How I AI

I tested 4 AI agents on 9 real tasks. Here’s who won.

Claire's first personal agent benchmark ran four agents on nine real tasks and found no single winner: Grockbot is best for work jobs, Muse for life jobs, while Dot and Claude Duet lag outside coding.

Watch on YouTube ↗

Charlie's summary

  1. Build a benchmark from the tasks your agents already do, and verify they all have the same access first

    Claire asked each agent for its top 10 tasks, then had Claude pick the most common, cross-cutting ones. She wrote nine one-line prompts in her own voice, not long AI-style prompts. She also had Codex use browser use to check that every agent had roughly the same connectors and skills. Even so, she forgot to connect Muse to business tools like Slack and GitHub, and that cost it tasks. Check access parity before you compare agents.

    ▶ 2:01
  2. Her earlier impressions held up: Muse is consistently fastest, Claude Duet slowest and most timid

    Muse was fastest on four of nine tasks, even though it needed the most steering. Claude Duet was the slowest overall, and it asked for permission four times where the other agents asked once. Claire says this backed up her earlier reviews, and she jokes it is “tremendously accurate data science.” She sees the permission habit as a safety-style difference that affects how usable an agent feels.

    ▶ 8:09
  3. Muse surprised her on short-form video editing, while Claude Duet could not download the video at all

    Given only a Dropbox raw video and a transcript, Dot, Grockbot and Muse each cut a clip with no touching. Claude Duet would not download the video and failed. Claire preferred Muse: simple branding, her caption layout, and punchy cuts. Grockbot's overlays were ugly, though its zooms were better, and it was also the slowest. Dot was a second choice. Claire had never tried Muse on this task and may now use it for creating social posts.

    ▶ 11:47
  4. For fixing bugs, the OpenAI and Anthropic tools do best, and Grockbot loosened validation instead of fixing the bug

    On a Sentry bug, all agents took about 10 minutes. Claude Opus graded the fixes. Dot's fix was closest to the real one. Claude Duet found a second bug and noticed Dot had already fixed the first. Grockbot loosened validation, which dodges the bug. Muse had no GitHub access and mistook the agent Devon for a person. Claire says she would still use the underlying Codex and Claude tools for coding, not the agents.

    ▶ 14:50

Give your email to unlock the rest of the key points — for every episode.

Do more with Charlie

Free — sign in with your email, no password.