head-to-head
| Metric | GPT-5.6 Sol | Claude Opus 4.8 |
|---|---|---|
| SWE-bench Verified | 96.2% | 88.6% |
| SWE-bench Pro | — | 69.2% |
| Terminal-Bench | 88.8% | ~82.7% (TB2.1) |
| Input $ / 1M | $5 | $5 |
| Output $ / 1M | $30 | $25 |
| Context | — | 1M |
| Open weights | No | No |
| Access | API · Codex (public since Jul 9 2026) | API · Claude Code · claude.ai (Max) |
| Maker | OpenAI | Anthropic |
what do the benchmarks actually say?
On SWE-bench Verified — real, human-validated GitHub issues resolved end-to-end — GPT-5.6 Sol posts 96.2% against 88.6% for Claude Opus 4.8, a 7.6-point gap. Verified is the closest public proxy for "can it fix a real bug in a real repo without help", which is why it anchors our ranking.
A few points either way is real but not decisive: within that band, the agent scaffolding around the model — how it retrieves files, runs tests, and retries — often matters as much as the base model. Treat the gap as a lean, not a verdict.
which is cheaper to run?
Both list at $5 per 1M input tokens, so the sticker price is a wash — output rates ($30 vs $25) and token efficiency decide the real bill. Coding workloads are output-heavy — agents write diffs, tests and retries — so weight the output rate more than the input rate when you estimate a monthly bill.
when to pick each
The highest independently measured coding score of any model, and it holds up on the long tasks: 98% on the 1-to-4-hour tier where most models fall apart.
The hardest agentic refactors and long, autonomous multi-file tasks where every point of accuracy saves a human review cycle.
how were these scores verified?
We only print a number once it's confirmed against a primary source or an independent evaluation, and each row on our leaderboard records which kind it is:
- GPT-5.6 Sol: Independent (vals.ai, Jul 14 2026, mini-swe-agent bash-only harness): SWE-bench Verified 96.20% ±0.86 — the top score on the board. Verified Jul 17, 2026; it had been unranked since Jun 26 because OpenAI published no SWE-bench number of its own, and it still has not. Read the #1 with care: the 1.2-point lead over Claude Fable 5 (95.00% ±0.98) is inside the combined margin of error (~0.9 sigma, not significant), so the two are a statistical tie and we rank Sol first only because it scored higher. Where it does separate is task length — 98% on 1-4 hour tasks vs 93% for Fable 5. OpenAI's own Terminal-Bench 2.1 claim is 88.8% (Sol) / 91.9% (Sol Ultra). No SWE-bench Pro score published. Pricing $5/$30 per 1M.
- Claude Opus 4.8: Independent (vals.ai, Jul 14 2026, mini-swe-agent bash-only harness): SWE-bench Verified 88.6% ±1.42. Corrected Jul 17, 2026: we previously printed Anthropic's own "~86%" and claimed independent evals tracked it within ~1 point, which was wrong — the independent number is 2.6 points higher, and our methodology is to prefer the independent one. Run through the Claude Code harness instead of the bare bash agent, vals.ai measures 85.8%, a reminder that the harness moves these numbers as much as the model does. SWE-bench Pro 69.2% is Anthropic-reported.
Full reviewsGPT-5.6 Sol, decoded
Ranked on our AI Coding Leaderboard, updated 2026-07-17. Scores are confirmed against primary sources; prices are per 1M input tokens and can change.
- OpenAIvals.ai — SWE-bench Verified (independent) — Independent (vals.ai, Jul 14 2026, mini-swe-agent bash-only harness): SWE-bench Verified 96.20% ±0.86 — the top score on the board. Verified Jul 17, 2026; it had been unranked since Jun 26 because OpenAI published no SWE-bench number of its own, and it still has not. Read the #1 with care: the 1.2-point lead over Claude Fable 5 (95.00% ±0.98) is inside the combined margin of error (~0.9 sigma, not significant), so the two are a statistical tie and we rank Sol first only because it scored higher. Where it does separate is task length — 98% on 1-4 hour tasks vs 93% for Fable 5. OpenAI's own Terminal-Bench 2.1 claim is 88.8% (Sol) / 91.9% (Sol Ultra). No SWE-bench Pro score published. Pricing $5/$30 per 1M.
- Anthropicvals.ai — SWE-bench Verified (independent) — Independent (vals.ai, Jul 14 2026, mini-swe-agent bash-only harness): SWE-bench Verified 88.6% ±1.42. Corrected Jul 17, 2026: we previously printed Anthropic's own "~86%" and claimed independent evals tracked it within ~1 point, which was wrong — the independent number is 2.6 points higher, and our methodology is to prefer the independent one. Run through the Claude Code harness instead of the bare bash agent, vals.ai measures 85.8%, a reminder that the harness moves these numbers as much as the model does. SWE-bench Pro 69.2% is Anthropic-reported.
- BenchmarkSWE-bench — the real-GitHub-issue benchmark