head-to-head
| Metric | Claude Opus 5 | Kimi K3 |
|---|---|---|
| SWE-bench Verified | 97.0% | 93.4% |
| SWE-bench Pro | — | — |
| Terminal-Bench | — | 88.3% |
| Input $ / 1M | $5 | $3 |
| Output $ / 1M | $25 | $15 |
| Context | — | 1M |
| Open weights | No | No |
| Access | API (claude-opus-5) · Claude Code · Claude Cowork · claude.ai (Pro, Max) | API · app · open weights promised Jul 27, 2026 |
| Maker | Anthropic | Moonshot AI |
what do the benchmarks actually say?
On SWE-bench Verified — real, human-validated GitHub issues resolved end-to-end — Claude Opus 5 posts 97.0% against 93.4% for Kimi K3, a 3.6-point gap. Verified is the closest public proxy for "can it fix a real bug in a real repo without help", which is why it anchors our ranking.
A few points either way is real but not decisive: within that band, the agent scaffolding around the model — how it retrieves files, runs tests, and retries — often matters as much as the base model. Treat the gap as a lean, not a verdict.
which is cheaper to run?
Kimi K3 is the cheaper model: $3 per 1M input tokens ($15 output) versus $5 ($25 output) for Claude Opus 5 — roughly 1.7× less on input. Coding workloads are output-heavy — agents write diffs, tests and retries — so weight the output rate more than the input rate when you estimate a monthly bill.
when to pick each
Pick Claude Opus 5 if
The highest independently measured coding score on the board at 97.0%, at half the price of Fable 5. Strongest on short and medium tasks, though GPT-5.6 Sol still edges it on multi-hour work.
Pick Kimi K3 if
Third-best coding score of any model and by far the cheapest way to buy a 90%+ result, at $3 per 1M input. Caveat that matters: despite the "open" billing, the weights are not downloadable yet, so today this is an API model.
how were these scores verified?
We only print a number once it's confirmed against a primary source or an independent evaluation, and each row on our leaderboard records which kind it is:
- Claude Opus 5: Independent (vals.ai, observed Jul 25 2026, mini-swe-agent bash-only harness): SWE-bench Verified 97.00% ±0.76, the highest score on the board and 1st of the 75 systems vals.ai has run. Entered ranked Jul 25, 2026 after one day unranked: Anthropic published no SWE-bench Verified number at launch and still has not, so this is vals.ai's own measurement rather than a vendor claim. Read the #1 as a three-way tie, not a win — GPT-5.6 Sol is at 96.20% ±0.86 (a 0.8-point gap, ~0.7 sigma) and Claude Fable 5 at 95.00% ±0.98 (2.0 points, ~1.6 sigma), both inside the combined margin of error. We rank Opus 5 first only because it scored highest. Where the top two genuinely separate is task length, and not in Opus 5's favour: on the 1-to-4-hour tier Sol solves 98% against Opus 5's 90%, while Opus 5 leads on shorter work (98% under 15 minutes and 97% on 15-minute-to-1-hour tasks, vs 97% and 95% for Sol). Released Jul 24, 2026 at $5/$25 per 1M, the same price as Opus 4.8 and half of Fable 5. Anthropic's launch claims stay unreproducible (Frontier-Bench v0.1, CursorBench 3.2 and Zapier AutomationBench are proprietary), so this is the first externally checkable score the model has. Fast mode runs about 2.5x default speed at 2x base price.
- Kimi K3: Independent (vals.ai, mini-swe-agent bash-only harness): SWE-bench Verified 93.40% ±1.11. Verified Jul 18, 2026 — vals.ai had not run K3 at our Jul 17 check and now ranks it third overall (above GPT-5.6 Luna at 93.0% and Claude Opus 4.8 at 88.6%), making it the highest-scoring open-weight coder we track. Moonshot published no SWE-bench Verified score of its own, only Terminal-Bench 2.1 88.3% on its KimiCode harness, so we rank on the independent number per our standing rule. Corroborated by a second independent evaluator: Artificial Analysis, which runs its own tests, has K3 4th on GDPval-AA v2 (Elo 1684), 2nd on AA-Briefcase (Elo 1545) and 4th of 187 on its Intelligence Index v4.1 (57.1). It also classifies K3 as proprietary outright. AA measured Terminal-Bench 2.1 at 85.02% against Moonshot's own claim of 88.3%, a 3.3-point vendor overstatement, which is why we print the independent figure. Blind developer voting on Arena WebDev ranks it 1st of 99. Price confirmed on Moonshot's own pricing page: $3/$15 per 1M, cache-hit input $0.30 (platform.kimi.ai, checked Jul 19, 2026). **open: false, corrected Jul 19, 2026** — we had this flagged as open weights, which put it top of our "best open-source coding model" page as a model nobody can download. Checked Jul 19: no K3 repo under huggingface.co/moonshotai (18 repos, none K3) and github.com/MoonshotAI/Kimi-K3 returns 404. Moonshot's own blog still says "The full model weights will be released by July 27, 2026." Until they actually ship, open is a roadmap item, not a fact, and this row is an API model. Flip back to open: true when the weights land.
Full reviewsClaude Opus 5, decodedKimi K3, decoded
Ranked on our AI Coding Leaderboard, updated 2026-07-25. Scores are confirmed against primary sources; prices are per 1M input tokens and can change.
Primary sources
- Anthropicvals.ai — SWE-bench Verified (independent) — Independent (vals.ai, observed Jul 25 2026, mini-swe-agent bash-only harness): SWE-bench Verified 97.00% ±0.76, the highest score on the board and 1st of the 75 systems vals.ai has run. Entered ranked Jul 25, 2026 after one day unranked: Anthropic published no SWE-bench Verified number at launch and still has not, so this is vals.ai's own measurement rather than a vendor claim. Read the #1 as a three-way tie, not a win — GPT-5.6 Sol is at 96.20% ±0.86 (a 0.8-point gap, ~0.7 sigma) and Claude Fable 5 at 95.00% ±0.98 (2.0 points, ~1.6 sigma), both inside the combined margin of error. We rank Opus 5 first only because it scored highest. Where the top two genuinely separate is task length, and not in Opus 5's favour: on the 1-to-4-hour tier Sol solves 98% against Opus 5's 90%, while Opus 5 leads on shorter work (98% under 15 minutes and 97% on 15-minute-to-1-hour tasks, vs 97% and 95% for Sol). Released Jul 24, 2026 at $5/$25 per 1M, the same price as Opus 4.8 and half of Fable 5. Anthropic's launch claims stay unreproducible (Frontier-Bench v0.1, CursorBench 3.2 and Zapier AutomationBench are proprietary), so this is the first externally checkable score the model has. Fast mode runs about 2.5x default speed at 2x base price.
- Moonshot AIvals.ai — SWE-bench Verified (independent) — Independent (vals.ai, mini-swe-agent bash-only harness): SWE-bench Verified 93.40% ±1.11. Verified Jul 18, 2026 — vals.ai had not run K3 at our Jul 17 check and now ranks it third overall (above GPT-5.6 Luna at 93.0% and Claude Opus 4.8 at 88.6%), making it the highest-scoring open-weight coder we track. Moonshot published no SWE-bench Verified score of its own, only Terminal-Bench 2.1 88.3% on its KimiCode harness, so we rank on the independent number per our standing rule. Corroborated by a second independent evaluator: Artificial Analysis, which runs its own tests, has K3 4th on GDPval-AA v2 (Elo 1684), 2nd on AA-Briefcase (Elo 1545) and 4th of 187 on its Intelligence Index v4.1 (57.1). It also classifies K3 as proprietary outright. AA measured Terminal-Bench 2.1 at 85.02% against Moonshot's own claim of 88.3%, a 3.3-point vendor overstatement, which is why we print the independent figure. Blind developer voting on Arena WebDev ranks it 1st of 99. Price confirmed on Moonshot's own pricing page: $3/$15 per 1M, cache-hit input $0.30 (platform.kimi.ai, checked Jul 19, 2026). **open: false, corrected Jul 19, 2026** — we had this flagged as open weights, which put it top of our "best open-source coding model" page as a model nobody can download. Checked Jul 19: no K3 repo under huggingface.co/moonshotai (18 repos, none K3) and github.com/MoonshotAI/Kimi-K3 returns 404. Moonshot's own blog still says "The full model weights will be released by July 27, 2026." Until they actually ship, open is a roadmap item, not a fact, and this row is an API model. Flip back to open: true when the weights land.
- BenchmarkSWE-bench — the real-GitHub-issue benchmark