CharlieKey takeaways from your favorite podcasts
← 19 November 2026AI & I

Retesting GPT-6.2 Astra & Sol

Every's retest finds GPT-6.2 Astra and Soul essentially unchanged: fast, concise and actionable, but pushed so far that they fall short of frontier rivals on writing, coding and creative work.

Watch on YouTube ↗

Charlie's summary

  1. Retesting after the model changed barely moved the scores: Astra and Soul 6.2 are essentially the same model reviewed

    Dan says Every was never told the model had changed before release, so it reran its tests. Mike's benchmark scores “basically didn't budge” between pre-release and released versions, with only run-to-run variability. Dan's writing-feedback task showed minor improvements, but the same issues remain. Tests take hours and cost a lot of tokens, so the retest was provisional.

    ▶ 0:28
  2. OpenAI's strengths, concision and speed, were dialed too far: the model compresses where the task wants depth

    Dan asked for editorial feedback plus a rewrite on an article. Astra 6.2 put suggested copy at the top and compressed everything, while Argon gave meaty suggestions. It also writes backwards sentences like “triple the runner-up's token count”. Dan says labs turn up a trait like instruction-following until it becomes a flaw, then overcorrect. Good defaults become inappropriate when a task needs thinking with the model.

    ▶ 3:01
  3. Astra jumps on the first fix and doesn't reason about why, which loses on architecture

    Kieran ran an optimization hill-climb. Astra won on one of 12 items by going very hard on the one slow thing it found, while the Google model improved everything and won overall. Astra even conceded it started too narrow. Kieran thinks Anthropic and Google models reason about why they did something; OpenAI's just do it. That's fine for specific instructions, but weak for fuzzy, long-running work. Jack adds it asks questions for questions' sake and makes it hard to stay out of the loop.

    ▶ 8:12
  4. Astra got better at junior-engineer tasks but still isn't enough to displace Fable 5.2

    Mike's junior engineer benchmark gives the model hundreds of backlog tasks and asks it to pick the ones it can do and build PRs. Released Astra beat pre-release by up to 29 points on some tickets. Its cost curve looks great. But it's still below Fable 5.2 on average, and Mike enjoys working with Fable. With so much competition, a new model must be significantly better in a couple of dimensions to enter a daily-driver rotation.

    ▶ 16:52

Give your email to unlock the rest of the key points — for every episode.

Do more with Charlie

Free — sign in with your email, no password.