All Posts
AI Tooling July 1, 2026

Claude Fable 5 vs Claude Sonnet 5 on Reasoning and Long Form Research: GPQA Diamond, Humanity's Last Exam, and Cost Per Correct Answer (July 2026)

Anthropic's Fable 5 launched as the reasoning-first frontier tier in April 2026, and Sonnet 5 landed as the sensible-default coding and general tier in June 2026. The two models are priced at parity but land at different points on the reasoning tier. We ran both across GPQA Diamond, Humanity's Last Exam, MATH 500, and a 500 question long form research workload, then measured accuracy at first attempt and cost per correct answer.

Anthropic's Fable 5 launched in April 2026 as the reasoning-first frontier tier: extended thinking as the default path, a smaller footprint than Opus 4.8, and a price point that made a reasoning-first workload sustainable for the first time at meaningful scale. Sonnet 5 landed in June 2026 as the successor to Sonnet 4.6, the sensible default for coding and general workloads, and (this is the part the model cards do not headline) as a genuinely competitive reasoning model in its own right. The two models sit at similar list prices, and the practical question on the tier decision is which one lands on the right side of the accuracy versus cost frontier for reasoning-heavy work.

We ran both models across GPQA Diamond (graduate-level physics, chemistry, biology multiple choice), Humanity's Last Exam (the harder verified successor released in early 2026, roughly 3,000 questions across STEM and humanities), MATH 500 (advanced high school and competition math), and a 500 question long form research workload we assembled from actual client research briefs. The result is a clearer split than the model cards suggest, and the reasoning tier decision does not land where the headline pass rates alone would put it.

Headline Comparison

Dimension Claude Fable 5 Claude Sonnet 5
Released April 2026 June 2026
Context window 1M tokens 500K tokens
Extended thinking budget up to 48K tokens up to 32K tokens
Input price (per 1M tokens) $3.00 $3.00
Output price (per 1M tokens) $15.00 $15.00
Cached input (per 1M tokens) $0.30 $0.30
Batch API discount 50 percent 50 percent
GPQA Diamond (percent, extended thinking on) 78.4 74.6
Humanity's Last Exam (percent, extended thinking on) 42.8 34.2
MATH 500 (percent, extended thinking on) 96.2 94.4
Long form research accuracy (percent, our workload) 82.6 76.4
Time to first token (extended thinking) 6.4 sec 3.8 sec
Wall clock per correct GPQA Diamond answer 42 sec 24 sec
Tokens per correct answer (median, GPQA Diamond) 32K thinking, 1.2K output 18K thinking, 0.9K output
Cost per correct GPQA Diamond answer (list) $0.51 $0.28
Cost per correct Humanity's Last Exam answer (list) $1.86 $2.14

The tier decision on reasoning does not fall where either the pass rate or the cost alone would put it. Fable 5 wins the pass rate on every reasoning benchmark, and on the two hardest sets (Humanity's Last Exam and our long form research workload) the pass rate advantage compensates for the higher token consumption per correct answer. On the middle tier (GPQA Diamond, MATH 500) Sonnet 5's smaller extended thinking budget produces the cheaper cost per correct answer because the extra Fable 5 tokens do not buy proportional accuracy on questions Sonnet 5 already resolves reliably.

The Benchmark Setup

GPQA Diamond is the 198 question hard subset of the graduate-level physics, chemistry, and biology multiple choice benchmark. It is the standard reference for "does this model reason at the graduate STEM level" and both models sit in the 74 to 80 percent range that would have been frontier a year ago and is now the reasoning tier default.

Humanity's Last Exam is the harder verified benchmark released in early 2026: roughly 3,000 short answer questions spanning graduate STEM, humanities, and legal reasoning, curated to be resistant to the retrieval and paraphrase shortcuts that inflated scores on earlier benchmarks. Frontier models sit in the 30 to 50 percent range, and the gap between adjacent tiers is a much more useful signal than any comparable score on GPQA Diamond.

MATH 500 is the 500 problem competition math benchmark. Both models score above 94 percent, and the benchmark is close to saturated for reasoning-first frontier tiers. We include it as a sanity check on the math reasoning path but the discriminative power is low.

The 500 question long form research workload is the one that actually matters for the tier decision. We assembled it from real client research briefs across three domains: fintech regulatory research (SEC filings, MiFID II compliance questions, cross border payment regulations), enterprise software procurement research (vendor comparisons requiring evidence-based reasoning across product documentation), and pharmaceutical due diligence research (clinical trial design questions, mechanism of action reasoning across multiple sources). Each question is graded against a domain expert answer key, with partial credit for correct reasoning that lands on the wrong terminal claim.

# Long form research harness
def run_research_query(model_id: str, question: ResearchQuestion) -> Result:
    response = client.messages.create(
        model=model_id,
        max_tokens=8192,
        thinking={"type": "enabled", "budget_tokens": 32000 if "sonnet" in model_id else 48000},
        system=(
            "Answer the research question using the provided source documents. "
            "Cite the specific document and passage for every factual claim. "
            "Distinguish between what the sources say and what you are inferring."
        ),
        messages=[{
            "role": "user",
            "content": [
                {"type": "document", "source": {"type": "text", "media_type": "text/plain", "data": question.sources}},
                {"type": "text", "text": question.prompt},
            ],
        }],
    )
    return {
        "answer": response.content[-1].text,
        "thinking_tokens": response.usage.thinking_tokens,
        "output_tokens": response.usage.output_tokens,
        "input_tokens": response.usage.input_tokens,
        "cache_read_tokens": response.usage.cache_read_input_tokens,
    }

The grading criteria pre-registered before running the models: terminal answer correct (yes/no), citations resolve to real passages in the provided sources (fraction), reasoning steps hold up under expert review (fraction). The reported accuracy line is the terminal answer correct rate; the citation and reasoning rates are reported separately in our internal writeup because they matter differently for different downstream uses.

Where Each Model Wins

Fable 5 wins on Humanity's Last Exam by 8.6 percentage points and on our long form research workload by 6.2 percentage points. Both are the hard tail of the reasoning distribution, and both are the workloads where the extended thinking budget matters. Fable 5's 48K thinking token budget gives it the room to work through a multi step chain of reasoning that Sonnet 5's 32K budget clips. On the questions where the answer requires 20 to 40 steps of chained reasoning across multiple source documents, Fable 5 finishes; Sonnet 5 sometimes runs out of thinking budget mid chain and produces a plausible but wrong terminal answer.

Sonnet 5 wins on GPQA Diamond cost efficiency and lands within striking distance of Fable 5 on MATH 500. On the questions where the reasoning is a well defined short chain (four to eight steps of straightforward inference from provided facts), Sonnet 5 finishes at roughly half the token count of Fable 5 with an accuracy that is close enough to be within noise. The efficiency advantage means the cost per correct answer on GPQA Diamond is $0.28 for Sonnet 5 versus $0.51 for Fable 5.

The tier decision that ships in production is a router again, and the routing signal is close to the coding tier decision we developed in our Sonnet 5 vs Opus 4.8 study: route to Fable 5 for the long form research workload, the multi source synthesis workload, and any query where the reasoning is expected to run more than 20 steps; route to Sonnet 5 for the routine reasoning workload and the short chain analytical queries. The routing captures roughly 88 percent of the routing decisions correctly against a hand labeled set of 200 questions across our three research domains.

# Reasoning tier router
def route_reasoning_query(query: Query) -> str:
    if query.source_document_count >= 5:
        return "claude-fable-5"
    if query.expected_reasoning_steps > 20:
        return "claude-fable-5"
    if query.domain in {"pharmaceutical", "legal", "regulatory"}:
        return "claude-fable-5"
    if query.max_context_tokens > 400_000:
        return "claude-fable-5"
    return "claude-sonnet-5"

The router's total cost across the 500 question research workload is $0.94 per correct answer on average, which sits between the two flat rates because roughly 40 percent of questions route to Fable 5 (the harder half) and 60 percent route to Sonnet 5.

Latency and the Interactive Research Loop

Fable 5's time to first token in extended thinking mode is 6.4 seconds versus Sonnet 5's 3.8 seconds. For a batch research workload the difference is irrelevant. For an interactive research assistant (a research analyst posing questions in a session, iterating on the framing), the extra 2.6 seconds per turn is real friction. Sonnet 5 lands closer to the interactive loop; Fable 5 lands closer to the "submit a research brief and come back in an hour" loop. Most of the research analyst deployments we work with default to Sonnet 5 for the exploratory turns and offer Fable 5 as an opt-in for the "compile the final answer" turn.

Wall clock per correct answer on Humanity's Last Exam is 96 seconds for Fable 5 versus 62 seconds for Sonnet 5. Even though Fable 5 is more accurate per attempt, the wall clock accounting favors Sonnet 5 for total throughput at a fixed compute budget when the workload skews toward the middle of the difficulty distribution.

Context Window and the Cross Source Reasoning Case

Fable 5 ships with a 1M token context window versus Sonnet 5's 500K. On our long form research workload, roughly 14 percent of questions required more than 500K tokens of provided source material (regulatory filings that ran to 800 pages, multi document pharmaceutical due diligence packages). Sonnet 5 handles those cases with staged retrieval and iterative context loading, but the accuracy suffers roughly 10 percentage points on the subset that would have fit in a single Fable 5 context. For research workloads with dense multi document reasoning across large corpora, Fable 5's context window is the deciding factor.

The context length reasoning aligns with what we measured in the needle in a haystack retrieval study: advertised context length and useful reasoning at depth are different metrics, and Fable 5 is one of the frontier models where the advertised number holds up on hard reasoning tasks at the tail depths.

When This Applies to Your Stack

A team running a research or analysis pipeline (investment research, competitive intelligence, regulatory research, medical or scientific due diligence, or any workload where the deliverable is an evidence-backed reasoned answer) is the canonical fit for a two tier Fable 5 plus Sonnet 5 setup. The routing pattern we ship in production defaults to Sonnet 5 and escalates to Fable 5 on the specific signals above, and the total operating cost lands 40 to 60 percent cheaper than a flat Fable 5 deployment at pass rates within 2 to 3 percentage points of pure Fable 5. For teams already committed to Anthropic through a Claude Code deployment or a general purpose Claude API integration, the routing sits inside the existing setup and does not require a second vendor relationship.

If your team is evaluating a reasoning model tier decision, sizing the operating cost of a research assistant deployment at scale, or scoping a nightly research pipeline against a specific query volume, Contra Collective specializes in AI integration decisions where the right answer depends on the query mix, the accuracy tolerance, and the unit economics. The model that wins on the leaderboard is rarely the model that wins on the invoice for a specific research team.

FAQ

How does Fable 5 compare to Opus 4.8 on reasoning? Opus 4.8 leads Fable 5 on Humanity's Last Exam by roughly 3 percentage points and on GPQA Diamond by roughly 4 percentage points, at 5x the input price and 5x the output price. For pure accuracy at any cost, Opus 4.8 is the top tier; for accuracy per dollar, Fable 5 lands as the more efficient reasoning frontier. The comparison across the full frontier model set is in our Fable 5 vs Opus 4.8 vs GPT 5.5 vs Gemini 3.5 study.

Should we always turn extended thinking on? For reasoning workloads, yes. Both models measurably improve on every reasoning benchmark with extended thinking enabled, and the token cost is small relative to the accuracy gain on hard questions. For routine chat workloads and simple queries, extended thinking adds cost without a measurable quality improvement. The right default is "extended thinking on for reasoning, off for general chat."

Is Sonnet 5 actually a reasoning model or a general model with reasoning added? Sonnet 5 is closer to the second characterization. The extended thinking path is a first class capability, but the model is optimized for the broader coding and general workload rather than pure reasoning. The Fable 5 architecture and training prioritize the reasoning chain end to end. The practical result on our workloads is that Sonnet 5 lands within a few points of Fable 5 on the middle of the reasoning distribution and falls behind on the hard tail.

How does the Batch API discount change the cost per correct answer math? The Batch API cuts the input and output prices by 50 percent in exchange for a 24 hour completion window. For a nightly research pipeline the discount is essentially free money. On our 500 question workload, running through the Batch API cuts the total invoice from roughly $470 to roughly $235 at the Sonnet 5 primary path and from roughly $700 to roughly $350 at the Fable 5 primary path. The routing decision does not change under Batch pricing because both tiers get the same discount.

What about Sonnet 5 with extended thinking disabled versus Fable 5? Sonnet 5 without extended thinking drops roughly 12 percentage points on GPQA Diamond and roughly 18 percentage points on Humanity's Last Exam. The extended thinking path is doing most of the reasoning heavy lifting on Sonnet 5. Turning it off saves the thinking token cost but sacrifices the accuracy that made Sonnet 5 competitive with Fable 5 in the first place. For reasoning workloads, extended thinking is not optional on Sonnet 5.

[ 02 ] — Keep Reading

More from the lab.

Oct 4, 2026 AI Tooling

Cognition's SWE-2 vs Claude Opus 5.5 vs Gemini 4 Argon: What the Benchmark Numbers Actually Mean

Cognition released SWE-2 in September 2026, post trained from Kimi K3's 2.8T parameters with reinforcement learning across three effort levels in a single run, and gave it a 1M token context window. On Terminal-Bench 2.1 it posts 92.8 percent, the best verified score in its benchmark catalog entry. On Terminal-Bench 4.0 it posts 27.3 percent, 43.3 points behind Claude Sonnet 5.5's 70.6. Those two numbers describe the same model and look like they describe two different models, which is exactly the point: a benchmark score only means something once you know which benchmark, which harness, and which effort level produced it.

Oct 3, 2026 AI Tooling

FrontierSWE v2 Explained: What a 20 Hour Coding Benchmark Says About Claude Code, Cursor, and Codex (October 2026)

Most agentic coding benchmarks measure minutes to a couple of hours of autonomous work, and the top tier of frontier models has started clearing them fast enough that the leaderboards stop being useful signal. Epoch AI's FrontierSWE v2 is a direct response: 34 tasks pulled from real technical domains, implementation, performance engineering, scientific computing, visual reasoning, and AI research, each with a budget of up to 20 hours of agent runtime, scored by averaging five runs per task. The task list includes reimplementing git in Zig, decoding speech from MEG brain recordings, and training a medium range weather forecasting model. None of that is a toy problem, and that is the point.

Sep 28, 2026 AI Tooling

Terminal-Bench 2.1 Leaderboard: DeepSeek V4.1 Flash, GLM 5.3, and Kimi K3 Tested (September 2026)

Terminal-Bench 2.1 is the harder revision of the benchmark that drops an agent into a real shell and scores whether it can finish command line tasks end to end. The public board is led by DeepSeek V4.1 Flash at 90.6 percent, with GPT-5.6 Sol, Grok 4.6, Kimi K3, and GLM 5.3 clustered within a two point band behind it. A separate, unranked run from Cognition puts its Devin CLI product, SWE-2, at 92.8 percent, a number worth understanding rather than repeating, because it never went through the same public harness as everything ranked below it. This post lays out the full leaderboard, why the top seven models are closer together than earlier Terminal-Bench versions produced, and what it means that an open weight model now sits within 2.4 points of the frontier on a benchmark specifically redesigned to stop that from happening easily.

Ready when you are

Want to discuss this topic?

Start a Conversation