head-to-head
| Metric | Kimi K3 | GPT-5.6 Luna |
|---|---|---|
| SWE-bench Verified | 93.4% | 93.0% |
| SWE-bench Pro | — | — |
| Terminal-Bench | 88.3% | — |
| Input $ / 1M | $3 | — |
| Output $ / 1M | $15 | — |
| Context | 1M | — |
| Open weights | No | No |
| Access | API · app · open weights promised Jul 27, 2026 | API |
| Maker | Moonshot AI | OpenAI |
what do the benchmarks actually say?
On SWE-bench Verified — real, human-validated GitHub issues resolved end-to-end — Kimi K3 posts 93.4% against 93.0% for GPT-5.6 Luna, a 0.4-point gap. Verified is the closest public proxy for "can it fix a real bug in a real repo without help", which is why it anchors our ranking.
A few points either way is real but not decisive: within that band, the agent scaffolding around the model — how it retrieves files, runs tests, and retries — often matters as much as the base model. Treat the gap as a lean, not a verdict.
which is cheaper to run?
Public per-token pricing isn't confirmed for both models, so we don't print a cost comparison yet.
when to pick each
Pick Kimi K3 if
Third-best coding score of any model and by far the cheapest way to buy a 90%+ result, at $3 per 1M input. Caveat that matters: despite the "open" billing, the weights are not downloadable yet, so today this is an API model.
Pick GPT-5.6 Luna if
The cost outlier of the leaders: within striking distance of the top scores while costing a fraction per solved task, which makes it the obvious candidate for high-volume agent runs.
how were these scores verified?
We only print a number once it's confirmed against a primary source or an independent evaluation, and each row on our leaderboard records which kind it is:
- Kimi K3: Independent (vals.ai, mini-swe-agent bash-only harness): SWE-bench Verified 93.40% ±1.11. Verified Jul 18, 2026 — vals.ai had not run K3 at our Jul 17 check and now ranks it third overall (above GPT-5.6 Luna at 93.0% and Claude Opus 4.8 at 88.6%), making it the highest-scoring open-weight coder we track. Moonshot published no SWE-bench Verified score of its own, only Terminal-Bench 2.1 88.3% on its KimiCode harness, so we rank on the independent number per our standing rule. Corroborated by a second independent evaluator: Artificial Analysis, which runs its own tests, has K3 4th on GDPval-AA v2 (Elo 1684), 2nd on AA-Briefcase (Elo 1545) and 4th of 187 on its Intelligence Index v4.1 (57.1). It also classifies K3 as proprietary outright. AA measured Terminal-Bench 2.1 at 85.02% against Moonshot's own claim of 88.3%, a 3.3-point vendor overstatement, which is why we print the independent figure. Blind developer voting on Arena WebDev ranks it 1st of 99. Price confirmed on Moonshot's own pricing page: $3/$15 per 1M, cache-hit input $0.30 (platform.kimi.ai, checked Jul 19, 2026). **open: false, corrected Jul 19, 2026** — we had this flagged as open weights, which put it top of our "best open-source coding model" page as a model nobody can download. Checked Jul 19: no K3 repo under huggingface.co/moonshotai (18 repos, none K3) and github.com/MoonshotAI/Kimi-K3 returns 404. Moonshot's own blog still says "The full model weights will be released by July 27, 2026." Until they actually ship, open is a roadmap item, not a fact, and this row is an API model. Flip back to open: true when the weights land.
- GPT-5.6 Luna: Independent (vals.ai, eval listed Jul 17 2026, mini-swe-agent bash-only harness): SWE-bench Verified 93.00% ±1.14. Added Jul 21, 2026 — this row was missing from the board even though vals.ai had already evaluated Luna, and our Kimi K3 note referenced its 93.0% score without ever listing it; adding it moves every row below it down one rank. Treat 3rd and 4th as a tie: Kimi K3's 93.40% ±1.11 is 0.4 points higher, well inside the combined margin of error (~0.25 sigma), so the ordering between them is not significant. Like the rest of the GPT-5.6 family, OpenAI has published no SWE-bench Verified figure of its own, so we rank on the independent number per our standing rule. The striking number is cost: vals.ai measured $0.21 per test against $1.15 for GPT-5.6 Sol, $1.92 for Claude Opus 4.8 and $2.05 for Claude Fable 5, at 201s median latency. Per-token list pricing not confirmed against OpenAI's own pricing page, so inPrice/outPrice stay blank rather than estimated.
Full reviewsKimi K3, decoded
Ranked on our AI Coding Leaderboard, updated 2026-07-21. Scores are confirmed against primary sources; prices are per 1M input tokens and can change.
Primary sources
- Moonshot AIvals.ai — SWE-bench Verified (independent) — Independent (vals.ai, mini-swe-agent bash-only harness): SWE-bench Verified 93.40% ±1.11. Verified Jul 18, 2026 — vals.ai had not run K3 at our Jul 17 check and now ranks it third overall (above GPT-5.6 Luna at 93.0% and Claude Opus 4.8 at 88.6%), making it the highest-scoring open-weight coder we track. Moonshot published no SWE-bench Verified score of its own, only Terminal-Bench 2.1 88.3% on its KimiCode harness, so we rank on the independent number per our standing rule. Corroborated by a second independent evaluator: Artificial Analysis, which runs its own tests, has K3 4th on GDPval-AA v2 (Elo 1684), 2nd on AA-Briefcase (Elo 1545) and 4th of 187 on its Intelligence Index v4.1 (57.1). It also classifies K3 as proprietary outright. AA measured Terminal-Bench 2.1 at 85.02% against Moonshot's own claim of 88.3%, a 3.3-point vendor overstatement, which is why we print the independent figure. Blind developer voting on Arena WebDev ranks it 1st of 99. Price confirmed on Moonshot's own pricing page: $3/$15 per 1M, cache-hit input $0.30 (platform.kimi.ai, checked Jul 19, 2026). **open: false, corrected Jul 19, 2026** — we had this flagged as open weights, which put it top of our "best open-source coding model" page as a model nobody can download. Checked Jul 19: no K3 repo under huggingface.co/moonshotai (18 repos, none K3) and github.com/MoonshotAI/Kimi-K3 returns 404. Moonshot's own blog still says "The full model weights will be released by July 27, 2026." Until they actually ship, open is a roadmap item, not a fact, and this row is an API model. Flip back to open: true when the weights land.
- OpenAIvals.ai — SWE-bench Verified (independent) — Independent (vals.ai, eval listed Jul 17 2026, mini-swe-agent bash-only harness): SWE-bench Verified 93.00% ±1.14. Added Jul 21, 2026 — this row was missing from the board even though vals.ai had already evaluated Luna, and our Kimi K3 note referenced its 93.0% score without ever listing it; adding it moves every row below it down one rank. Treat 3rd and 4th as a tie: Kimi K3's 93.40% ±1.11 is 0.4 points higher, well inside the combined margin of error (~0.25 sigma), so the ordering between them is not significant. Like the rest of the GPT-5.6 family, OpenAI has published no SWE-bench Verified figure of its own, so we rank on the independent number per our standing rule. The striking number is cost: vals.ai measured $0.21 per test against $1.15 for GPT-5.6 Sol, $1.92 for Claude Opus 4.8 and $2.05 for Claude Fable 5, at 201s median latency. Per-token list pricing not confirmed against OpenAI's own pricing page, so inPrice/outPrice stay blank rather than estimated.
- BenchmarkSWE-bench — the real-GitHub-issue benchmark