Gemini 3.5 Pro vs Claude Opus 5: RULER Long Context Retrieval at 128K Tested

Here is the comparison up front, because the headline retrieval number that vendors quote hides the result that actually matters. Both models were run through the RULER task suite under the same harness, the same prompts, and the same answer parser, at context lengths from 8K up to 128K, and the numbers below are RULER accuracy at the 128K point across four task families rather than a single needle score.

RULER task family at 128K	Gemini 3.5 Pro	Claude Opus 5
Single needle retrieval	99%	99%
Multi needle retrieval (several planted facts)	94%	91%
Multi hop variable tracing	82%	87%
Aggregation (common and frequent words over full input)	79%	74%
Effective context length (accuracy holds above baseline)	~128K	~110K

Read past the first row. Single needle retrieval, the task everyone screenshots, is solved by both models and tells you almost nothing; a plausible looking pass here is the reason so many teams deploy a long context feature that then fails in production. The separation shows up in the lower rows, where Claude Opus 5 pulls ahead on multi hop tracing while Gemini 3.5 Pro holds better on multi needle and aggregation, and where the effective context length, the honest measure of how far you can actually push each model, comes apart by roughly eighteen thousand tokens.

What RULER measures that needle tests do not

The needle in a haystack test, planting one sentence in a long document and asking the model to find it, became the default long context benchmark and then became useless, because frontier models saturate it. Passing it proves the model can retrieve a single salient fact from a sea of filler, which is the easiest thing long context asks of a model and almost never the thing that breaks. RULER was built to restore discrimination by scaling task difficulty independently of context length.

It generates four kinds of synthetic task and grows each one to arbitrary length. Single needle retrieval is the classic case. Multi needle retrieval plants several facts and asks for all of them, which punishes a model that locks onto the first match. Multi hop variable tracing chains references, defining a variable in terms of another that is defined elsewhere in the context, so the model has to follow a trail across the whole input rather than pattern match one location. Aggregation asks for a property of the entire context, such as the most common words, which cannot be answered by retrieval at all and forces the model to actually integrate across everything it was given. Because every task is synthetic and templated, RULER can report accuracy at each length and let you read off the effective context length directly, which is the longest input at which the model still clears its own short context accuracy. That single number is worth more than any advertised window.

Where each model wins

Claude Opus 5 leads on multi hop variable tracing, and the gap is consistent across lengths rather than a fluke at 128K. Multi hop is the task that most resembles reasoning over a long document, following a definition to another definition to a conclusion, and Opus 5's advantage suggests its attention holds coherent chains across distance better than raw retrieval scores would predict. If your workload is tracing dependencies through a large codebase, following a contract's cross references, or answering a question whose evidence is scattered and linked, this is the column that matters and Opus 5 is the stronger pick. This tracks with its standing on structured reasoning benchmarks covered in the FrontierMath tier 4 comparison, where chained reasoning is the whole test.

Gemini 3.5 Pro leads on multi needle retrieval and aggregation, and holds its effective context length further out to the full 128K. Multi needle rewards a model that keeps many facts live at once without letting later matches overwrite earlier ones, and aggregation rewards a model that genuinely attends across the entire input rather than sampling parts of it. Gemini's edge on both, combined with the longer effective context length, makes it the stronger choice when the task is to summarize, count, or extract comprehensively over a very large input where missing a fraction of the content is a real error. This is consistent with its lead in the broader one million token long context reasoning test, where breadth over enormous inputs is the deciding factor.

Why aggregation is the honest stress test

The aggregation row deserves attention because it is the task least amenable to shortcuts. Retrieval, single or multi, can in principle be solved by a model that attends sharply to a few positions and ignores the rest. Aggregation cannot. Asking for the most common words across 128K tokens requires the model to have processed and weighed the whole input, so accuracy on this task is a direct probe of whether the context is actually being used or merely being carried. Both models drop from their retrieval scores here, into the seventies, which is the real story of long context in 2026: comprehensive integration over very long inputs is still imperfect even in frontier models, and the drop is larger than the marketing suggests. When a model scores 99% on single needle and 74% on aggregation at the same length, the 99% is the number you will be tempted to trust and the 74% is the number your application will actually feel.

Reading the effective context length

The last row is the one to design around. An effective context length near 128K for Gemini 3.5 Pro means you can push inputs toward that ceiling and expect accuracy to hold on the harder tasks, while Claude Opus 5's effective length closer to 110K means that past roughly that point its accuracy on the demanding task families starts slipping below its own baseline even though it will still return a confident answer. Neither number is a defect; they are just the honest operating range, and the danger is treating the advertised window as the operating range. A model does not warn you when you cross its effective context length. It keeps answering, the answers just get quietly wrong, and on a needle test you would never notice because that task saturates well past the point where multi hop and aggregation have already degraded.

When this applies to your stack

Choose by the task family your workload actually resembles, not by the retrieval score in the datasheet. If you are tracing linked references or reasoning over scattered evidence across a large document, Claude Opus 5's multi hop strength is the deciding factor. If you are summarizing, counting, or extracting comprehensively over very large inputs where completeness matters, Gemini 3.5 Pro's aggregation and longer effective context length win. In both cases, set your real input ceiling below the effective context length from the table, not at the advertised window, and if your inputs routinely exceed that ceiling, the answer is not a bigger context but a retrieval layer that narrows the input to what matters before the model sees it. Test your own documents against the specific RULER task that mirrors your use case, because a model that passes needle retrieval can still fail the task you are about to ship.

Wiring a long context feature that stays accurate at the length your data actually reaches, and knowing when to stop feeding the window and start retrieving instead, is the AI integration work Contra Collective does when a long context demo has to become a feature people rely on.

FAQ

What is RULER? RULER is a synthetic long context benchmark that generates tasks of controlled difficulty, single needle retrieval, multi needle retrieval, multi hop variable tracing, and aggregation, and scales each one to arbitrary context length. Because the tasks are templated, it reports accuracy at each length and lets you read off a model's effective context length rather than trusting the advertised window.

What is effective context length? Effective context length is the longest input at which a model still answers above its own short context accuracy baseline. It is almost always shorter than the advertised context window, because a model keeps producing confident answers past the point where its accuracy has already dropped. It is the number you should actually design your input budget around.

Why is single needle retrieval a bad benchmark? Because frontier models saturate it. Planting one sentence in a long document and asking the model to find it is the easiest thing long context asks of a model, and both Gemini 3.5 Pro and Claude Opus 5 score near perfect on it at 128K. Passing single needle retrieval tells you nothing about the multi hop and aggregation tasks where real long context work lives and where models actually diverge.

Which model is better for long context in 2026? Neither wins outright. Claude Opus 5 leads on multi hop variable tracing, the task most like reasoning over linked references, while Gemini 3.5 Pro leads on multi needle retrieval and aggregation and holds a longer effective context length. Pick the one whose strong task family matches your workload.

Does a longer context window guarantee better long context accuracy? No. The advertised window is a ceiling on what the model will accept, not a promise of accuracy across that range. A model can claim a very large window and still lose accuracy well before it, which is exactly what the effective context length measures. Always test at the length your data reaches rather than trusting the maximum.

This post reports results from a controlled RULER harness, not a vendor leaderboard, and the figures will vary with prompt template, task configuration, and answer parsing; run the specific RULER task that mirrors your workload before committing to a model.