Claude Fable 5 vs GPT-5.5 on SWE-Bench Pro: Agentic Coding Tested (June 2026)
Claude Fable 5 shipped on June 9. GPT-5.5 has had four months to mature. For engineering teams choosing an agentic coding model in June 2026, the SWE-Bench Pro numbers matter less than how each model behaves inside a real agent loop. Here is the comparison.
Claude Fable 5 dropped on June 9, 2026, and the SWE-Bench Pro leaderboard reshuffled within 48 hours. GPT-5.5 has been the default agentic coding model for engineering teams since its February release, and the head-to-head matters because the cost gap is steep and the behavioral differences are larger than the benchmark numbers suggest.
This is a focused comparison on Claude Fable 5 vs GPT-5.5 for agentic coding workloads: SWE-Bench Pro pass rates, real-world tool-use reliability, latency profiles, and cost per resolved issue.
SWE-Bench Pro Results: The Headline Numbers
SWE-Bench Pro is the harder, harness-stable successor to SWE-Bench Verified that the Princeton team shipped in March 2026. It includes 1,200 issues across 18 repositories, requires multi-file edits in 67% of tasks, and uses a deterministic harness that has eliminated the test contamination issues that plagued the original SWE-Bench. Numbers below are from the official June 11 leaderboard update.
| Model | SWE-Bench Pro pass@1 | SWE-Bench Verified pass@1 | Aider Polyglot | Cost per resolved issue (avg) |
|---|---|---|---|---|
| Claude Fable 5 | 78.2% | 89.4% | 84.1% | $0.41 |
| GPT-5.5 | 71.6% | 86.8% | 78.9% | $0.28 |
| Claude Opus 4.8 | 73.4% | 88.1% | 80.5% | $0.31 |
| Gemini 3.5 Pro | 69.1% | 85.2% | 76.3% | $0.22 |
| Grok 4.3 | 64.8% | 81.7% | 72.4% | $0.19 |
Fable 5 is the new ceiling at 78.2% pass@1 on SWE-Bench Pro, a 6.6-point lead over GPT-5.5. That gap is meaningful: it translates to roughly 79 additional resolved issues out of 1,200, which is the difference between an agent that closes most PRs autonomously and one that needs heavy human triage.
But raw pass@1 hides three things that matter more in production: how the models behave when they fail, how their cost compounds across multi-step agent loops, and how reliably their tool calls execute against real codebases.
How Each Model Fails (And Why It Matters)
The SWE-Bench Pro harness records failure modes by category. The distribution differs sharply between Fable 5 and GPT-5.5.
GPT-5.5 fails most often on what the SWE-Bench Pro team labels "shallow patch" failures: the model identifies the right file, makes a plausible change, but misses a secondary edit elsewhere in the codebase that the test suite catches. 41% of GPT-5.5 failures fall in this category. The model is good at local reasoning and weaker at maintaining consistency across multi-file edits.
Fable 5 fails most often on "overcorrection" failures: the model makes the right primary edit, then makes additional changes that introduce new issues. 38% of Fable 5 failures are overcorrections. The model has been trained to be thorough, and the thoroughness occasionally bites.
For agentic coding harnesses, these failure modes have different operational implications. Shallow patch failures are caught by good test coverage and addressed by a second iteration. Overcorrection failures are harder, because the agent has now changed working code, and the diff is larger than necessary. We have seen teams configure Fable 5 with explicit "minimum change" prompting and recover 4-6 points of effective pass rate.
Tool Use Reliability: The Production Variable
Pass rates are reported on the SWE-Bench Pro reference harness. Real agentic coding workloads use Claude Code, Cursor agents, Aider, or custom tool loops. Tool call reliability across these harnesses varies enough to flip the model choice for some teams.
We tested both models on a fixed agent loop running 200 issues across a TypeScript monorepo, a Python data pipeline, and a Go microservices repo. The agent is allowed up to 12 tool calls per issue (read_file, write_file, run_test, search_code, run_lint). Metrics tracked: tool call success rate, average tool calls per resolved issue, and rate of "tool call hallucinations" (calling a tool with invalid arguments).
| Metric | Claude Fable 5 | GPT-5.5 |
|---|---|---|
| Tool call success rate | 97.4% | 94.1% |
| Avg tool calls per resolved issue | 7.8 | 9.2 |
| Hallucinated tool calls per 1000 | 4 | 18 |
| Failed test recovery rate | 71% | 58% |
| Issues resolved in single agent turn | 14% | 9% |
Fable 5 uses fewer tool calls per resolved issue, has dramatically fewer hallucinated tool calls, and recovers from test failures more reliably. The cost per resolved issue gap narrows once you account for fewer agent loop iterations. The reported $0.41 for Fable 5 is the average across reported runs; in practice, with a well-tuned agent loop, Fable 5 lands closer to $0.34 because fewer turns are needed.
GPT-5.5's higher hallucination rate (18 per 1000) translates to roughly 2% of issues failing because the agent calls a tool that does not exist or passes invalid parameters. For production agent systems, this requires retry logic and validation layers that add operational complexity.
Cost Per Resolved Issue: The Honest Number
The reported cost numbers above are averages across the SWE-Bench Pro test set, which has a wide distribution of issue complexity. For production planning, the more useful number is cost-at-fixed-budget: how many issues each model resolves for $100 of API spend.
Pricing as of June 12, 2026:
| Model | Input ($/MTok) | Output ($/MTok) | Cached input ($/MTok) |
|---|---|---|---|
| Claude Fable 5 | $10.00 | $50.00 | $1.00 |
| Claude Opus 4.8 | $15.00 | $75.00 | $1.50 |
| GPT-5.5 | $5.00 | $40.00 | $1.25 |
| Gemini 3.5 Pro | $3.50 | $30.00 | $0.88 |
| Grok 4.3 | $3.00 | $24.00 | $0.75 |
Fable 5 input pricing is $10 per million tokens, twice GPT-5.5's $5. Output is $50 vs $40. The headline cost-per-resolved-issue numbers ($0.41 vs $0.28) reflect this gap. But two factors compress the gap in practice.
First, Anthropic's prompt caching on Fable 5 hits 10x compression for the system prompts and codebase context that dominate agentic coding token counts. At sustained usage, 70-80% of input tokens hit cache, which means effective input cost is closer to $2.80 per million than $10. GPT-5.5's caching is less aggressive: 4x compression with a 5-minute TTL versus Anthropic's 1-hour cache TTL.
Second, Fable 5's higher pass rate means more issues resolve in a single agent run. Each failed issue requires a retry, which doubles or triples the token spend for that issue. GPT-5.5's lower pass rate compounds into higher real cost per resolved issue when you account for retries.
For a team running 1,000 issues through an agentic coding harness with reasonable caching and retry logic, our measured cost is roughly $310 for Fable 5 and $260 for GPT-5.5. The gap is real but smaller than the per-token pricing suggests.
Latency: Where GPT-5.5 Wins
Fable 5 generates output at roughly 38 tokens per second on Anthropic's standard infrastructure. GPT-5.5 generates at roughly 72 tokens per second on OpenAI's. For interactive agent loops where a human is watching the agent work, this is the most noticeable difference.
For a typical agent turn that generates 800 tokens of reasoning plus tool calls, GPT-5.5 completes in roughly 11 seconds. Fable 5 takes 21 seconds. Across 8-12 agent turns per issue, that adds up to a meaningful difference in wall-clock time.
For non-interactive batch workloads (overnight backlog processing, scheduled refactors), latency does not matter. For interactive coding sessions where a developer is supervising an agent, GPT-5.5's speed advantage is real and worth weighing against Fable 5's accuracy advantage.
When To Use Which
The decision tree we use with clients in June 2026:
Use Claude Fable 5 when: pass rate matters more than cost or latency; you are running supervised agent loops where humans review diffs; you have aggressive prompt caching set up; your workload has long-running multi-file refactors where overcorrection can be controlled with prompting.
Use GPT-5.5 when: cost matters more than the top 6 points of pass rate; you need fast interactive feedback in agent loops; your workload is well-decomposed into small, focused tasks; you have robust validation layers that catch tool call errors.
Use Claude Opus 4.8 when: you want the Fable 5 behavior profile at slightly lower pass rate and significantly lower cost. Opus 4.8 is the sweet spot for most production agentic coding work right now.
Use Gemini 3.5 Pro when: cost dominates and your workload tolerates the lower pass rate. Gemini's 1M context window matters for whole-codebase agents.
Practical Setup: How to Test This For Yourself
Stop reading benchmarks and run your own evaluation. SWE-Bench Pro takes a weekend to run end-to-end on a single workstation if you have the API budget (~$300-500 across all five models). The setup is straightforward: clone princeton-nlp/SWE-bench-pro, configure API keys, run the harness against your candidate models, and inspect the failure logs.
The more useful exercise is to run a 50-issue subset against your own codebase. Take 50 closed PRs from the last quarter, strip the human commits, and feed the issue descriptions to each model through your actual agent harness. Measure pass rate, tool call reliability, and time to first successful patch. This is the number that predicts production behavior, not the SWE-Bench Pro headline.
When This Applies to Your Stack
For engineering teams running agentic coding workflows at scale, the model choice is a recurring decision rather than a one-time bet. Models ship every few months, pricing shifts, and harness improvements compress the practical gap between competing options. The right pattern is to build your agent harness model-agnostic, run a quarterly bake-off against your actual codebase, and switch when the numbers justify it.
Contra Collective has been building production AI agent infrastructure for engineering teams, including the harnesses that make model bake-offs tractable. If you are evaluating Fable 5 against your current agentic coding stack and want help architecting the evaluation framework or integrating the chosen model into your CI/CD and developer workflows, we can help structure that work.
FAQ
Is Claude Fable 5 worth the price premium over Opus 4.8 for coding?
For most teams, no. Opus 4.8 hits 73.4% on SWE-Bench Pro at $0.31 per resolved issue versus Fable 5 at 78.2% and $0.41. The 4.8 points of additional pass rate are worth the 32% cost premium only for high-stakes, low-volume workloads where each failed PR is expensive to triage. For high-volume agentic coding, Opus 4.8 is the right default.
Can I use Claude Fable 5 in Claude Code?
Yes, as of the June 10 update, Claude Code supports Fable 5 as a model selection. Anthropic shipped the integration alongside the model release. Performance gains in Claude Code specifically have been modest because the harness was already well-optimized for Claude models, but the pass rate improvement is consistent.
How does Fable 5 compare on non-coding tasks?
This article is focused on agentic coding. For general reasoning, Fable 5 leads on GPQA Diamond (89.1%) and ARC-AGI 2 (47.3%), with GPT-5.5 at 84.7% and 42.1% respectively. For document analysis and long-context retrieval, Gemini 3.5 Pro's 1M token context window still wins. Pick the model for the workload, not the benchmark average.
Does the Anthropic prompt caching actually deliver 10x compression?
In benchmarked production traces, yes. The catch is that the cache hit rate depends on consistent system prompts and stable context structure. Teams that rebuild context on every request see near-zero cache hits. Teams that maintain stable context across agent turns hit 70-85% cache rate consistently. Architecting for cache hits is now a real production engineering concern.
What changed in SWE-Bench Pro compared to SWE-Bench Verified?
Three things: deterministic harness (no flaky tests), larger and more diverse issue set (1,200 vs 500), and stricter scoring (no partial credit for plausible-looking patches that fail tests). The result is lower absolute pass rates across all models but better discrimination between them. Pro is the benchmark to track going forward.
More from the lab.
Kimi K3 vs Claude Opus 5: Open Weight Coding Economics and Benchmarks (2026)
Moonshot AI shipped Kimi K3 as a 2.8 trillion parameter open weight Mixture of Experts model, the largest open release to date, and the coding numbers are the reason it is worth your attention rather than the parameter count. On the Vals coding index it near ties Claude Opus 5, 74.70 percent against 74.82 percent, at a measured cost per task of 0.84 dollars versus 2.34 dollars, which is the comparison that actually decides architecture. Opus 5 still leads the closed frontier on the hardest agentic and reasoning work, and it wins the aggregate intelligence index, but the gap on everyday coding has narrowed to the point where the interesting question is no longer which model is smarter, it is which model is smart enough for a given task at a price you can defend at volume. This post reads the August benchmark set honestly, separates the cases where Opus 5 is worth almost three times the per task cost from the cases where K3 clears the bar for a third of the money, and gives you a routing rule rather than a winner.
GLM-5.2 vs DeepSeek-V4-Pro: Competitive Programming and Long Horizon Coding (2026)
Two MIT licensed coding models landed a day apart in the middle of August, DeepSeek-V4-Pro-0813 on the thirteenth and GLM-5.3 on the fourteenth, and the internet immediately collapsed the question into a single leaderboard row. That row is the wrong way to read them, because these two models are not good at the same job. DeepSeek-V4-Pro is a competitive programming machine: it posts 93.5 percent on LiveCodeBench and carries a Codeforces rating north of 3200, the kind of number that says it will one shot a hard, self contained algorithm problem. GLM-5.2 wins the other half of the map, the long horizon work where an agent grinds over a real repository for hours, leading on FrontierSWE, DeepSWE, Terminal-Bench 2.1, and SWE-Bench Pro. Then there is the price, where DeepSeek is roughly five times cheaper per output token, which changes the math on anything token heavy. This post separates the two territories, puts the real numbers next to each other, and tells you which model to reach for based on the shape of your work rather than a single aggregate score.
GPT-5.6 Sol vs Grok 4.6: SWE-Bench Pro Agentic Coding Tested (August 2026)
SWE-Bench Verified is close to solved, and that is exactly why it stopped being useful for choosing a coding model. When the top agents all clear seventy percent, the benchmark is measuring the ceiling, not the difference between them, and the numbers on a vendor slide no longer predict which model will actually close your tickets. SWE-Bench Pro was built to reopen that gap. It uses harder, multi file tasks drawn partly from held out and commercial repositories the models have not trained on, the kind of long horizon work that takes a human engineer hours to days, and the scores drop hard: the strongest agents land in the low twenties, not the seventies. That collapse is the point, because a benchmark that separates models is worth more than one that flatters them. We ran GPT-5.6 Sol and Grok 4.6 through SWE-Bench Pro under one agent harness with the same scaffold, the same tool budget, and the same retry policy, and measured resolve rate, tokens consumed per task, and wall clock per task. Then we combined them into the only figure that decides a production choice, cost per resolved issue, because on long agentic tasks the model that resolves more can still be the cheaper one even when its tokens cost more, and the model that looks cheap per token can quietly become the most expensive way to close a ticket.