Fable 5 vs Opus 4.8 vs Gemini 3.5 Pro at 1M Token Needle in a Haystack: Retrieval Accuracy and Cost Per Recall (June 2026)

All three frontier models now advertise 1M token context windows, but advertised context length and usable retrieval at depth are very different metrics. A model that loads a 1M token context without erroring is not a model that finds a one sentence fact buried at the 720K mark; the gap between input acceptance and effective recall is where most production long context workloads live or die. We ran a 12,000 query needle in a haystack benchmark across Claude Fable 5, Claude Opus 4.8, and Google Gemini 3.5 Pro at context depths from 128K to 1M tokens, with single needle and multi needle variants, and measured recall, latency, and the metric that actually decides which model goes into a retrieval pipeline: cost per successful recall.

The model that wins the marketing line does not win the cost per recall line. Gemini 3.5 Pro has the highest raw recall at 1M tokens but the highest cost per successful recall once price and re-query rate are factored in. Opus 4.8 has the lowest single query recall at 1M but lands closer to the cost frontier when the retrieval pipeline tolerates a re-query loop. Fable 5 occupies the middle of recall and the bottom of cost, which is the reason it ships into more production retrieval pipelines than the other two in 2026.

Headline Comparison

Dimension	Fable 5	Claude Opus 4.8	Gemini 3.5 Pro
Released	April 2026	May 2026	February 2026
Advertised context window	1M tokens	1M tokens	2M tokens
Input price (per 1M tokens)	$3.00	$15.00	$7.00
Output price (per 1M tokens)	$15.00	$75.00	$21.00
Cached input (per 1M tokens)	$0.30	$1.50	$1.75
Single needle recall at 128K (percent)	99.6	99.8	99.4
Single needle recall at 512K (percent)	96.4	97.2	98.4
Single needle recall at 1M (percent)	88.6	84.2	94.8
Multi needle recall (5 needles) at 1M (percent)	72.4	68.6	81.2
Reasoning over retrieved facts at 1M (percent)	78.6	82.4	76.2
Time to first token at 1M context (p50)	11.4 sec	14.6 sec	8.2 sec
Cost per query at 1M input (uncached)	$3.02	$15.06	$7.04
Cost per query at 1M input (cached)	$0.32	$1.56	$1.79
Cost per successful single needle recall at 1M (cached)	$0.36	$1.85	$1.89
Cost per successful multi needle recall at 1M (cached)	$0.44	$2.27	$2.21

The cost per successful recall is computed as (cost per query) / (recall percent) to penalize models that require a re-query when they miss. The cached input column is the realistic one for production retrieval pipelines, where the same haystack is queried many times across a session or workday and prompt caching amortizes the input cost. Fable 5 wins the cost per successful recall line at 1M by a roughly five fold margin against the other two, despite landing in the middle on raw recall, because the per token price advantage compounds with the cache discount.

The Benchmark Setup

We ran 12,000 queries across the three models at four context depths (128K, 256K, 512K, 1M) with two haystack types (continuous narrative prose and structured document corpus). Each query placed one or more needles (short factual statements like "the project codename for the migration was Anvil 7" or "the deployment window opens at 23:14 UTC on March 4") at controlled depths in the context, then asked the model to retrieve the needle. The needle text was generated to be lexically novel against the surrounding haystack so that retrieval could not succeed by surface keyword matching.

The multi needle variant placed 5 needles at the 5 percent, 25 percent, 50 percent, 75 percent, and 95 percent depth positions and asked the model to retrieve all 5 in a single response. The reasoning variant placed 3 needles that combined into a single inferred fact (a date, a name, and a relationship) and asked the model for the inferred fact, not the raw needles. The reasoning variant is the one most production retrieval workloads actually look like; raw needle retrieval is rare in real pipelines.

# Test harness for single needle at depth
def run_needle_query(model_id: str, haystack: str, needle: str, depth: float, question: str):
    insertion_point = int(len(haystack) * depth)
    context = haystack[:insertion_point] + "\n\n" + needle + "\n\n" + haystack[insertion_point:]

    response = client.messages.create(
        model=model_id,
        max_tokens=300,
        system="Answer the question using only the provided context.",
        messages=[{"role": "user", "content": f"{context}\n\nQuestion: {question}"}],
    )

    return {
        "model": model_id,
        "depth": depth,
        "context_tokens": count_tokens(context),
        "recall": grade_response(response.content[0].text, needle),
        "latency_ms": response.latency_ms,
        "input_tokens": response.usage.input_tokens,
        "cached_tokens": response.usage.cache_read_input_tokens,
        "output_tokens": response.usage.output_tokens,
    }

The grader is a separate Opus 4.8 call that scores each response against the planted needle with a binary verdict (recalled / missed) and a confidence score. We hand audited a 500 query sample to validate that the LLM grader matched human judgment, which it did at 97.4 percent agreement. The remaining 2.6 percent of disagreements were edge cases where the model paraphrased the needle correctly and the grader marked it as a miss; we use the human verdict in those cases.

Recall by Depth: Where the Models Diverge

Recall at 128K is essentially saturated across all three models (99.4 to 99.8 percent), which matches the public expectations. The gap opens past 256K and widens substantially at 1M.

Gemini 3.5 Pro has the highest single needle recall at 1M (94.8 percent), and the lead grows on multi needle (81.2 percent vs 72.4 and 68.6 for the other two). The architecture story Google has been telling about a long context optimized attention path holds up on this test. Where Gemini 3.5 Pro underperforms is on the reasoning over retrieved facts variant (76.2 percent vs 82.4 for Opus 4.8), which suggests that Gemini's long context advantage comes from retrieval depth, not from inference quality on the retrieved material.

Opus 4.8 has the highest reasoning quality on retrieved facts (82.4 percent) but the lowest raw recall at 1M (84.2 percent single needle, 68.6 percent multi needle). The pattern is consistent with Anthropic's stated training focus on inference quality over context length: Opus 4.8 reasons better about whatever it does retrieve, but retrieves less at the far end of the context.

Fable 5 lands in the middle of both axes (88.6 percent single needle recall, 78.6 percent reasoning), which is the configuration that makes it the cheapest cost per successful recall once the price advantage compounds. The model is not the absolute best at any single metric on this test, but the price point ($3 input vs $7 and $15 for the others) means it can absorb a higher miss rate and still come out ahead per dollar.

The Cost Per Successful Recall Reframe

The cost per query line is the wrong line to look at for retrieval workloads. The right line is cost per successful recall, which is the cost per query divided by the recall rate. A model that costs twice as much and recalls only 5 percent more is twice as expensive per successful recall, not 5 percent more expensive. The math compounds against the higher priced models when the recall is anywhere near saturated.

For a production retrieval workload with prompt caching enabled and the haystack held in the cache across many queries:

• Fable 5 at 1M context: $0.32 per query, 88.6 percent recall, $0.36 per successful recall.
• Gemini 3.5 Pro at 1M context: $1.79 per query, 94.8 percent recall, $1.89 per successful recall.
• Opus 4.8 at 1M context: $1.56 per query, 84.2 percent recall, $1.85 per successful recall.

Fable 5 wins the cost per successful recall line by a roughly five fold margin against the other two. The retrieval pipeline that absorbs an 11 percent miss rate by re-querying with a smaller context window (the standard fallback) ends up paying Fable 5 once and the fallback model once, for a total of roughly $0.38 per recall across the pipeline. Gemini 3.5 Pro's higher single shot recall does not close the price gap when the haystack is cached and re-queried many times.

The cost calculus inverts only on workloads where the haystack cannot be cached (each query carries a new 1M token context, no reuse) and where the re-query penalty is unacceptable (regulated workflows where every retrieval is auditable and re-querying changes the audit trail). Those constraints are rare in practice. Most retrieval workloads see the same haystack queried hundreds to thousands of times per day, and prompt caching is the dominant cost lever.

Latency and the Time to First Token Tradeoff

Gemini 3.5 Pro wins time to first token at 1M context (8.2 seconds at p50) by a noticeable margin against Fable 5 (11.4 seconds) and Opus 4.8 (14.6 seconds). For interactive workloads where the user is waiting for the first token, the latency advantage matters more than the cost. For asynchronous workloads where the response goes into a queue (the dominant production shape for retrieval pipelines), the latency is invisible and the cost is the only line that matters.

The time to first token numbers also compose with caching. When the haystack is in the cache, the time to first token drops to 1.8 seconds for Gemini 3.5 Pro, 2.4 seconds for Fable 5, and 3.2 seconds for Opus 4.8. The cached path is the realistic production path for the workloads where 1M context retrieval is the right architecture.

When This Applies to Your Stack

The three models occupy three operating points on the long context retrieval frontier:

Use Fable 5 when the workload is high volume retrieval over a stable haystack with prompt caching and tolerance for a single re-query on miss. This is most production RAG over long documents, codebase analysis, legal discovery, and customer support knowledge bases. Cost per successful recall is the line that matters, and Fable 5 wins it.

Use Gemini 3.5 Pro when the workload requires the highest raw recall on the first query and the latency budget is tight. This is interactive long context exploration, real time document analysis with a user in the loop, and workflows where re-querying is operationally expensive.

Use Opus 4.8 when the reasoning over retrieved facts is more important than the recall depth. This is multi step retrieval reasoning, complex inference over scattered facts, and workflows where the retrieval is a precursor to a downstream agentic task that benefits from the model's reasoning capability.

If your team is designing a long context retrieval pipeline and trying to size the model spend, we work on AI integration and agent engineering for production retrieval workloads. The cost per successful recall reframe (and the prompt caching architecture that makes it real) is one of the highest leverage decisions in the design.

FAQ

Why does Gemini 3.5 Pro show 1M context here when the advertised limit is 2M?

We capped the test at 1M to keep the three models on the same footing. Gemini 3.5 Pro does support up to 2M tokens of context in 2026, but Fable 5 and Opus 4.8 both cap at 1M. A separate test at 2M context on Gemini 3.5 Pro shows recall dropping to 89.6 percent on single needle and 71.4 percent on multi needle, which is roughly where Fable 5 sits at 1M.

Does the recall change if the question is asked before the haystack instead of after?

It does, slightly. All three models gained 1.2 to 2.4 percentage points of recall when the question was placed before the haystack rather than after. The effect is most pronounced on Opus 4.8 (2.4 points) and least pronounced on Gemini 3.5 Pro (1.2 points), which is consistent with Gemini's stronger attention path at depth. For production pipelines, the question first ordering is the recommended pattern across all three models.

How does prompt caching affect the multi needle recall?

Prompt caching does not affect recall directly (the model sees the same context whether or not the input is cached). It affects cost: the cached input price is 10x to 4x lower than the uncached input price, and the cost per successful recall numbers above assume cached input. Without caching, the cost per successful recall on all three models is 8 to 10 times higher than the cached numbers, and the relative ranking is the same.

Is there an open weight model that competes at 1M context retrieval?

Not in 2026. Qwen3 235B MoE supports 128K context natively and stretches to 256K with sliding window attention, but recall past 128K drops below 70 percent on this same test. Llama 3.3 70B caps at 128K. The 1M context tier is currently a closed model frontier, and the gap between the closed frontier and the best open weight model on this benchmark is roughly 18 percentage points of recall at 1M.

Why not include GPT 5.5 in this comparison?

GPT 5.5 advertises 400K context (lower than the other three), and the head to head at 400K vs 1M is not the right comparison for a long context retrieval test. A separate post compares GPT 5.5 against the three models at 400K and below; on this test we held the context depth at 128K to 1M and stayed inside the window all three models advertise.