Claude Opus 5 vs Gemini 3.5 Pro: DS-1000 Data Science Coding Tested (August 2026)

A model that tops SWE-Bench can still write clumsy pandas. Software engineering benchmarks reward the ability to navigate a repository and satisfy a test suite; they say little about whether a model can reshape a dataframe, fit a scikit learn pipeline, or produce a matplotlib figure that runs the first time. DS-1000 exists to test that second skill directly. It is a suite of 1000 data science problems sourced from real questions, each paired with an execution based test and rewritten enough that a model cannot recover the answer from memory. Here is the comparison first, then the category split that actually decides which model to build a data workflow on.

Headline Comparison

Dimension	Claude Opus 5	Gemini 3.5 Pro
Type	Proprietary API	Proprietary API
Context window	500K tokens	1M tokens
Input price, per 1M tokens (list)	$6.00	$2.50
Output price, per 1M tokens (list)	$30.00	$15.00
DS-1000 overall pass@1, our run	71.4	70.2
pandas	68	65
numpy	74	72
scipy	70	69
scikit learn	75	71
matplotlib	66	73
tensorflow	69	71
pytorch	73	74

The overall gap is a little over one point, which is noise at this resolution. If you stop at the average you learn nothing useful. The categories are where the two models actually separate.

How We Ran It

Every number above is one attempt per problem, no self repair, no retries, evaluated by DS-1000's own execution tests rather than by a judge model. We held decoding settings fixed across both models and used a plain instruction prompt with the problem statement and any provided context, nothing tuned per vendor. Execution based scoring is the reason to trust DS-1000 more than a benchmark graded by string match or by another model: a solution either produces the expected result when run or it does not. Directional figures, one harness, same day, so the split between categories is comparable even where an absolute number would move a point or two on a rerun.

We report pass@1 deliberately. Data scientists rarely run a model ten times and keep the best; they run it once inside a notebook or a pipeline and either use the output or fix it by hand. A single attempt score is the honest read on that workflow.

Where Opus 5 Pulls Ahead

Opus 5 led on the categories that reward careful reading of an underspecified prompt. Its scikit learn and numpy scores were the widest gaps in its favor, and the qualitative pattern behind them was consistent: it handled multi step transformations where the correct answer depended on a detail stated once in the problem, and it was less likely to hallucinate an API that does not exist. On pandas, where problems often hinge on the difference between operations that look interchangeable but are not, it more reliably picked the one the test actually wanted.

That reliability has a cost, literally. At $6 input and $30 output per million tokens, Opus 5 is the more expensive model by a wide margin, and it tends to produce longer, more explained solutions, which pushes output token counts up on top of the higher rate. For interactive work where a human reviews each answer that verbosity is often welcome. For a high volume batch job the combined effect on the bill is real and worth modeling before you commit.

Where Gemini 3.5 Pro Pulls Ahead

Gemini 3.5 Pro won matplotlib clearly and edged ahead on the deep learning categories, tensorflow and pytorch. The matplotlib result tracks something practical: plotting problems reward getting a fussy, layered API correct in one pass, and Gemini produced figure code that ran without the small errors that force a round trip. On tensorflow and pytorch its answers matched idiomatic current usage more often, which matters because those libraries deprecate patterns fast and a model anchored on older idioms writes code that technically runs but reads as stale.

The economics run the other way from Opus. A 1M token context window, a $2.50 input rate, and a $15 output rate make Gemini the natural default for volume: batch feature engineering, large scale notebook generation, or any pipeline where you are paying per call at scale. When the average quality is within a point, price and context are not tiebreakers, they are the decision.

Which One To Build On

The honest recommendation is workload shaped, not a single winner.

If your data work lives in pandas, numpy, and scikit learn, and a human reviews outputs before they run, Opus 5's edge on careful multi step transforms is worth the premium. The failure it avoids most, quietly picking the wrong one of two similar operations, is exactly the failure that costs a data scientist an afternoon of debugging.

If your work is plotting heavy, deep learning heavy, or high volume, Gemini 3.5 Pro is the better default. It matched or beat Opus on matplotlib, tensorflow, and pytorch, and it does so at a fraction of the token cost with double the context. For a pipeline that generates thousands of cells a day, that combination compounds.

For a mixed team the pragmatic pattern is to route by category: send the deep learning and visualization work to Gemini and the tabular reasoning to Opus. We covered how to build that kind of tool aware routing in the function calling comparison, and it applies cleanly here. Neither model is strong enough everywhere to make routing pointless, and the split is stable enough to route on.

When This Applies to Your Stack

DS-1000 is a proxy, not a contract. It tells you which model writes better single shot library code in a controlled setting, which is a real and useful signal, but your actual data workflows carry context these problems strip away: your schemas, your naming, your internal helpers, your review process. Treat the category split as a starting prior and validate on a sample of your own tasks before you standardize. A model that wins DS-1000 pandas by three points can still lose on your codebase if your pandas usage is unusual.

If you are wiring a frontier model into a data pipeline or an analytics product and want the routing, evaluation, and cost model built on your workload rather than a public leaderboard, that integration work is what our AI integration and agent engineering practice does. The benchmark narrows the field. Your data decides the winner.

FAQ

What does DS-1000 actually test? One thousand data science coding problems across pandas, numpy, scipy, scikit learn, matplotlib, tensorflow, and pytorch, each graded by running the generated code against an execution test rather than by string or judge comparison.

Why is execution based scoring better here? Because a data science answer is right only if it runs and produces the expected result. Execution tests catch code that looks plausible but fails, which string match and judge scoring both miss.

Which model is better overall? By our run they are within about one point on the overall average, which is a wash. The decision is per category: Opus 5 leads scikit learn and numpy, Gemini 3.5 Pro leads matplotlib, tensorflow, and pytorch.

Which is cheaper to run at volume? Gemini 3.5 Pro, by a wide margin, at $2.50 input and $15 output per million tokens versus $6.00 and $30.00 for Opus 5, with a larger context window on top.

Should I use one model or route between them? For a mixed workload, route: visualization and deep learning to Gemini, tabular reasoning to Opus. The category split is stable enough that routing beats standardizing on either alone.