SHELFCRITTER AI Digest

MODEL

Gemini 2.5 Pro

modeltopic-notegoogle

Overview

Gemini 2.5 Pro is Google’s frontier-class reasoning model, the successor to Gemini 2.0 Pro. It appears on the Terminal-Bench 2.0 agentic-coding leaderboard as a reference point for scaffold-sensitivity analysis: the same model scores materially differently depending on whether it is paired with the Gemini CLI or the Terminus 2 scaffold, illustrating that benchmark results for agentic-coding tasks are heavily harness-dependent.

Timeline

  • 2026-05-17-AI-Digest — Benchmarked on Terminal-Bench 2.0 with two scaffolds: 19.6% via Gemini CLI and 32.6% via Terminus 2 — a 13-point swing from scaffold choice alone. Cited as the reference point for the digest’s “scaffold matters as much as model” takeaway, framing the Qwen3.6-35B-A3B 24.6% result in context.
  • 2026-05-26-AI-Digest — Holds the only non-gpt-5 slot on the Aider polyglot top-5 (gemini-2.5-pro-preview-06-05 32k think at 83.1%, fourth overall behind three GPT-5 effort tiers and o3-pro). The takeaway in the digest’s Key Takeaways framing: integrated agentic-coding leaderboards remain a frontier-closed game, and Gemini 2.5 Pro is the lone non-OpenAI presence on this canonical practitioner benchmark.
  • 2026-05-27-AI-Digest — Still holds the only non-gpt-5 slot on the Aider polyglot top-5 (gemini-2.5-pro-preview-06-05 32k think at 83.1%, fourth overall behind gpt-5 high, gpt-5 medium, and o3-pro). Board is identical to yesterday’s snapshot; the durability of the GPT-5 dominance against the absence of any newer Gemini entry is itself the data point.
  • 2026-05-29-AI-Digest — Holds #4 (the only non-gpt-5 slot) on the Aider polyglot top-5 at the 2026-05-29 fetch — gemini-2.5-pro-preview-06-05 (32k think) at 83.1%, behind gpt-5 high/medium and o3-pro and ahead of gpt-5 low. The board is unchanged from prior days; Gemini 2.5 Pro remains the lone non-OpenAI presence in the frontier-quality tier of this canonical practitioner benchmark.
  • 2026-06-06-AI-Digest — Holds #4 on the Aider polyglot top-5 at the 2026-06-06 fetch — gemini-2.5-pro-preview-06-05 (32k think) at 83.1%, the lone non-gpt-5 slot in the frontier-quality tier. Board unchanged from prior fetches; today’s Key Takeaways frame the closed-reasoning sweep as the ceiling the on-device substrate (Gemma 4 QAT E2B at ~1 GB) is being compared against — “1 GB multimodal on a phone, not open-weights caught up.”
  • 2026-06-12-AI-Digest — gemini-2.5-pro-preview-06-05 (32k think) holds #4 on the Aider polyglot top-5 at 83.1% — the lone non-GPT-5 slot, unchanged from yesterday. Same digest: DeepMind broadens the Deep Think variant rollout via the consumer Gemini app, the chain-of-thought-heavy reasoning mode in the Gemini 2.5 family being lifted off advanced-user gating into broader availability.
  • 2026-06-07-AI-Digest — Holds #4 on the Aider polyglot top-5 at the 2026-06-07 fetch — gemini-2.5-pro-preview-06-05 (32k think) at 83.1%, the lone non-gpt-5 slot. Board identical to the prior snapshot; the public board has now been frozen at the 2025-11-20 refresh for over six months. Today’s Key Takeaways frame the public top-5 as a reference floor, with unbenchmarked Jun 2026 open-weight releases (DeepSeek V4-Pro, Qwen3-Coder-Next, MiniMax M3) plus a community-mirror DeepSeek-V3.2-Exp at ~74% flagged as the live edge.
  • 2026-06-13-AI-Digest — Holds #4 on the Aider polyglot top-5 at the 2026-06-13 fetch — gemini-2.5-pro-preview-06-05 (32k think) at 83.1%, the lone non-gpt-5 slot. Board identical to prior fetches. Lands the same day SWE-Bench Verified shows Claude Mythos 5 / Claude Fable 5 / Claude Opus 4.8 sweeping the top three — the cross-benchmark divergence is the corpus’s read on coding-race attribution today, not a Gemini 2.5 Pro shift.
  • 2026-07-17-AI-Digest — Holds #4 on the Aider polyglot top-5 at the 2026-07-17 fetch — gemini-2.5-pro-preview-06-05 (32k think) at 83.1%, the lone non-gpt-5 slot. Board unchanged from yesterday’s print; the digest carries the disciplined framing that Aider is a snapshot benchmark materially trailing the open-weight release cycle (Kimi K3 shipping today not yet scored). Gemini 2.5 Pro remains the sole non-OpenAI presence in the frontier-quality tier of this canonical practitioner benchmark.

Key Developments

  1. Scaffold Sensitivity on Terminal-Bench 2.0: 13-point gap (19.6% vs 32.6%) between Gemini CLI and Terminus 2 scaffolds on the same model is the clearest single-day illustration of how benchmark ranking on agentic-coding tasks depends on harness architecture, not just model capability.

See also: Gemini, MOC - Agentic Coding, MOC - Open Source Models