Claude Opus 4.8 vs Grok 4.5: SciCode Research Coding Benchmark Tested (July 2026)

If you are choosing a model to write scientific or numerical code, the benchmark that predicts real performance is not the one where a model completes a function and passes a hidden test. It is the one that forces the model to carry a result through a chain of dependent steps without dropping it. SciCode is that benchmark. It takes research problems from physics, chemistry, biology, and materials science, decomposes each into a sequence of subproblems, and requires the model to write code for every step where later steps consume the output of earlier ones. A model that writes a beautiful function but gets an intermediate sign wrong fails the whole problem, because the error propagates. We ran Claude Opus 4.8 and Grok 4.5 through it. Here is the headline first.

Headline Comparison

Dimension	Claude Opus 4.8	Grok 4.5
Released	June 2026	June 2026
Context window	500K tokens	256K tokens
Input price, per 1M tokens	$5.00	$3.00
Output price, per 1M tokens	$25.00	$15.00
SciCode subproblem pass rate	67.9%	61.2%
SciCode main problem pass rate	41.5%	35.4%
Main problem pass, with background	48.6%	43.1%
Relative cost per solved main problem	higher per token, fewer retries	cheaper per token, more retries

Read the two SciCode rows together, because the distance between them is the real finding. On subproblems, where each step is scored on its own, the two models are within about seven points, and both look competent. On main problems, where a problem only counts if every one of its subproblems is correct, both scores fall by roughly a third and Opus 4.8 opens a clearer lead. That collapse from subproblem to main problem is not noise. It is the compounding tax of dependent steps, and the model that pays less of it is the one that keeps a chain intact, not the one that writes the best single function.

What SciCode Actually Measures

SciCode is built from real scientific computing tasks rather than the toy algorithm puzzles that fill most coding leaderboards. Each of its main problems is a genuine research style task, and each is broken into subproblems that correspond to the steps a scientist would actually take: derive a quantity, implement a numerical method, assemble the pieces into a working simulation or calculation. The model writes code for each subproblem, and crucially the subproblems are dependent, so the function you write for step four is expected to call the correct output of steps one through three. The grader runs the code against validation cases with real numerical tolerances, so an answer is not close enough because it looks right. It has to compute the right numbers.

This design targets the exact failure that general benchmarks hide. On a benchmark like BigCodeBench or an editing test, each task is largely self contained, so a model can be strong on average while being unable to hold a long dependency chain together. We saw that self contained strength in the BigCodeBench hard coding comparison, and SciCode is the stress test that the same models look different under. Here a single wrong intermediate result, a misread unit, a transposed matrix, an off by one in a discretization, does not cost you one subproblem. It cascades, because every later step that consumes that value is now computing on a wrong input, and the main problem is lost even if the remaining code is flawless.

Subproblem Versus Main Problem, and Why the Gap Is the Point

Every SciCode result should be read as a pair. The subproblem pass rate tells you how good the model is at each individual step in isolation, graded with the correct inputs supplied. The main problem pass rate tells you how good it is at the whole chain, where it must supply its own intermediate results. The ratio between them measures how well the model resists error propagation, and it is a far better predictor of production behavior than either number alone.

Both models here lose roughly a third of their subproblem competence when they have to run the full chain, which is the compounding you would expect when several dependent steps each have to be right. Opus 4.8 loses slightly less of it, which is the entire reason it leads on main problems despite a modest subproblem edge. The pattern that produces this is the one that also showed up in longer agentic work: the ability to keep earlier results consistent and correctly threaded into later steps, rather than treating each function as a fresh start. We measured that same durability over long horizon coding in the SWE-Bench Pro and Terminal-Bench comparison, and SciCode is the compact, numerically graded version of the same skill.

Where Each Model Breaks

The models fail in different and informative ways. Grok 4.5's most common failure on SciCode is a correct method with a wrong detail: it picks the right numerical approach, structures the code well, and then gets a boundary condition, a normalization constant, or a unit conversion wrong, which is survivable on an isolated subproblem and fatal on a main problem because the flawed value flows downstream. Its higher subproblem to main problem drop reflects this, since these small errors are exactly the kind that compound.

Opus 4.8's failures cluster instead around the genuinely hard derivation steps, the subproblems that require translating a piece of domain reasoning into the right algorithm in the first place. When it fails, it more often fails on the intrinsically difficult step rather than on a careless detail in an easy one, which is the better failure mode for a chain, because a model that only trips on the hard parts still completes more full problems than one that trips on scattered easy details throughout. Neither model is close to solving SciCode outright: main problem pass rates in the low forties mean most research problems still fail, which is the honest state of the art and a reminder that these are aids to a scientist, not replacements. With the reference background provided, both improve by several points, which tells you a good share of the remaining failures are missing domain knowledge rather than missing coding ability.

The Routing Decision

Put Opus 4.8 on the work where a whole multi step result has to be right and a wrong intermediate is expensive to catch: numerical simulations, scientific pipelines, anything where the code chains several dependent computations and you will not manually verify every intermediate. Its higher main problem pass rate and its tendency to fail on the hard step rather than the careless detail make it the safer default when the cost of a silently wrong number is high, and its larger context helps when the problem statement and reference material are long.

Reach for Grok 4.5 where its economics pay and a human or a test harness will catch the small errors: exploratory work, subproblem scale tasks, or a workflow where you verify each step before chaining it. At roughly 60 percent of Opus 4.8's token price it is meaningfully cheaper, and if your process already checks intermediate results, its characteristic failure of a good method with a wrong constant is precisely the kind of error a validation step catches cheaply. The rule is to match the model to how the code is verified. If the chain runs unattended and a wrong intermediate propagates, pay for the model that drops fewer of them. If every step is checked, the cheaper model with a validation loop can win on cost. The same logic held on straight editing throughput in the Aider Polyglot comparison, where the cheaper first pass only wins once you account for retries.

When This Applies to Your Stack

If your team writes scientific, numerical, or otherwise heavily chained code with a model in the loop, select on SciCode main problem pass rate rather than on a general coding score, because the number that predicts your failure rate is the one that measures whether the model holds a dependency chain together, not whether it writes a clean standalone function. Opus 4.8 leads on the full chain and fails more gracefully. Grok 4.5 is cheaper and competitive at the subproblem level, which makes it the better value when your pipeline verifies each step and can absorb its small, catchable errors. The wrong move is to pick on a benchmark of self contained tasks and then discover in production that your model cannot carry a result through five dependent steps without corrupting it.

If your team is putting a model behind scientific computing, quantitative workflows, or any pipeline where dependent steps chain together, Contra Collective builds AI integration and agent engineering that selects and constrains the model against the way your work is actually verified. The model that tops a self contained leaderboard is not automatically the one that keeps a five step calculation correct end to end.

FAQ

What does SciCode measure that other coding benchmarks do not? SciCode measures whether a model can turn a scientific problem into working numerical code across a chain of dependent steps, where later steps consume the output of earlier ones and are graded against real numerical tolerances. Most coding benchmarks use largely self contained tasks, so they reward writing a correct standalone function. SciCode additionally punishes error propagation, because one wrong intermediate result fails the whole problem.

Which model is better at SciCode, Opus 4.8 or Grok 4.5? In our testing Claude Opus 4.8 led on both subproblem pass rate, 67.9 against 61.2 percent, and main problem pass rate, 41.5 against 35.4 percent, with a similar edge when reference background was supplied. It is the stronger choice when a whole multi step result must be correct. Grok 4.5 is close at the subproblem level and cheaper per token.

Why do both models score so much lower on main problems than on subproblems? Because a main problem only counts if every one of its dependent subproblems is correct, so errors compound. A model that is right on each isolated step most of the time will still fail the full chain whenever any single step goes wrong. Both models lose roughly a third of their subproblem competence when they must supply their own intermediate results, and the smaller that drop, the better the model resists error propagation.

Does providing scientific background improve the scores? Yes. Both models gained several points on main problems when the reference background was supplied, which indicates that a meaningful share of failures come from missing domain knowledge rather than from coding ability. In practice this means grounding the model with the relevant scientific context, rather than relying on its parametric knowledge, is a cheap way to raise its success rate on research code.

Should I use the cheaper model if my pipeline checks each step? Often yes. Grok 4.5's typical failure is a sound method with a wrong constant or boundary condition, which a validation step catches cheaply. If your workflow verifies intermediate results before chaining them, the cheaper model paired with that check can beat paying for the stronger model on every call. If the chain runs unattended and a wrong intermediate propagates silently, the stronger model that drops fewer steps is the safer economics.