GPT-5.6 Sol vs Claude Opus 5: GDPval Real-World Professional Tasks Tested

Here is the comparison first, because that is why you opened this. Both models were run over the GDPval task set under the same instructions and the same blind grading protocol, where an experienced practitioner from each occupation compares the model deliverable against a human reference and decides which is better or whether they tie. The numbers below come from that controlled harness, not a vendor leaderboard.

Metric	GPT-5.6 Sol	Claude Opus 5
GDPval overall win-or-tie rate vs human deliverable	49.1%	50.7%
Wins outright (rated better than the human)	27.8%	29.6%
Document and analysis heavy tasks	strong	strong
Spreadsheet and financial model tasks	moderate	stronger
Tasks needing a precise domain format	weaker	moderate
Median deliverable cost (tokens, relative)	lower	higher

The two land within about two points of each other on the headline, and on that number alone you would call it a coin flip. The average is not where they differ. They differ in which occupations each one carries cleanly, in how each one fails when it fails, and in what you pay per deliverable to get there. All three of those decide whether a model belongs in a professional workflow, and none of them survives being compressed into a single win rate.

What GDPval measures and why the deliverable is the point

GDPval is built around real work products from a spread of occupations that make up a meaningful share of economic output. Each task is a brief drawn from an actual job, a request to produce the kind of artifact that role delivers, and a human reference produced by an experienced person in that field. The grading is the part that matters: a blind expert comparison, not a string match and not a model judging a model. The evaluator does the job for a living and decides whether the deliverable would pass in their world.

That construction is what makes GDPval predictive in a way most benchmarks are not. A high GPQA or reasoning score tells you the model is smart in the abstract; a GDPval win rate tells you the model produced something a practitioner preferred to a human's attempt. Those are different claims. A model can reason beautifully and still hand back a financial model with the wrong sign convention, or a legal memo that cites the right doctrine in the wrong jurisdiction, and GDPval catches exactly that gap because the grader is looking at the finished artifact, not the reasoning trace. It sits alongside the other economically framed evaluations like freelance software tasks, where the question is not can the model think but can it deliver the thing someone would have paid for.

Where the two models diverge

Both models are genuinely strong on the document heavy end of the set. For briefs, memos, analyses, summaries, and structured writing where the deliverable is prose backed by reasoning, GPT-5.6 Sol and Claude Opus 5 both clear the human reference often enough that the choice between them on those tasks is about cost and house style rather than capability. If your professional work is mostly writing and analysis, either model is ready and you should pick on price and voice.

The gap opens on the structured artifacts. On spreadsheet and financial model tasks, where the deliverable has to be internally consistent, formula correct, and follow a convention the grader recognizes, Opus 5 held its quality more reliably while Sol produced more artifacts that looked right and computed wrong. That failure is expensive precisely because it is subtle: a plausible spreadsheet with a broken reference passes a glance and fails an audit, and a grader who builds these for a living catches it where a casual reviewer would not. On the flip side, Sol delivered comparable quality at a lower median token cost across the set, so on the tasks where both models tie the human reference, Sol got there cheaper. That cost gap is the same tradeoff that shows up when reasoning is priced against accuracy: the cheaper deliverable is only cheaper if it is also correct.

The failure mode that matters more than the score: plausible and wrong

The most operationally important finding is not in the win rate. It is in what a loss looks like. When these models lose a GDPval comparison, the deliverable is almost never obviously bad. It is confident, well formatted, and wrong in a way that requires domain knowledge to see: a number that does not tie, a clause that does not apply, a chart that misreads the data it is built on. That is far more dangerous than a visibly weak output, because a visibly weak output gets rejected and a plausible wrong one gets shipped.

This is why a GDPval number cannot be consumed as a hiring decision for the model. A fifty percent win rate does not mean you can hand half your professional work to the model unattended; it means that on half the tasks a blind expert preferred the model's artifact, and the other half includes deliverables that look finished and are not. The correct reading is that these models are ready to produce the first draft of professional work and not ready to be the last set of eyes on it, and the review that catches the plausible wrong deliverable is the part of the workflow you cannot remove yet. The same lesson shows up in machine learning engineering tasks, where an agent's output can run cleanly and still be the wrong solution to the actual problem.

Reading these numbers without fooling yourself

Two cautions. First, the GDPval average is a blend across occupations that have nothing to do with each other, and your work is not the average. A model that wins on marketing briefs and loses on actuarial tables posts the same overall number as a model with the reverse profile, and those are opposite hiring decisions for a specific team. Before you trust the headline, find the occupations closest to your own work and read those rows, because the average is a portfolio and you are buying a single stock.

Second, a win against a human reference is a comparison, not an absolute. The reference is one practitioner's attempt under time pressure, and beating it is not the same as being correct or being safe to deploy. Treat GDPval as evidence that a model can produce professional grade first drafts in a given field, then run your own small evaluation on your actual deliverables with your own experts grading, because the only win rate that decides anything for you is the one measured on the work you actually ship.

When this applies to your stack

Use GDPval to answer one question: is this model ready to draft the professional artifacts my team produces, or only to assist around the edges. If your work is document and analysis heavy, both GPT-5.6 Sol and Claude Opus 5 are ready to draft and you should choose on cost and voice, with Sol the cheaper path and Opus 5 the steadier one on structured, format sensitive artifacts like financial models. Either way, keep an expert review step in the loop for anything that gets shipped, because the models lose in the plausible and wrong direction, not the visibly bad one, and that is the failure your workflow has to be built to catch. Match the benchmark to your occupations rather than the average, and validate on your own deliverables before you let a win rate change how work gets done.

Wiring a frontier model into a real professional workflow, with the drafting handed to the model and the review structured to catch the confident wrong deliverable before it ships, is the AI integration work Contra Collective takes on when a team wants leverage from these models without inheriting their failure modes. The benchmark tells you they can draft. Your process decides whether that is safe.

FAQ

What is GDPval? It is a benchmark that grades models on real deliverables drawn from real occupations, comparing the model's artifact against a human reference through blind evaluation by an experienced practitioner in that field. Instead of scoring an answer, it judges whether the finished work product holds up, which makes it a closer proxy for professional knowledge work than reasoning or coding benchmarks.

Did GPT-5.6 Sol or Claude Opus 5 win? They finished within about two points on the overall win-or-tie rate, close enough to call even on the headline. The meaningful differences are underneath: Opus 5 was steadier on structured artifacts like financial models, Sol delivered comparable quality at lower token cost, and each carries different occupations more reliably than the other.

Does a fifty percent win rate mean the model can replace a professional? No. It means a blind expert preferred the model's deliverable on about half the tasks. The other half includes outputs that look finished and are wrong in ways only a domain expert catches. The honest reading is that these models draft professional work well and are not yet safe as the final reviewer of it.

Why is GDPval harder to interpret than a normal benchmark? Because the average blends occupations that behave nothing alike, so a single number hides opposite capabilities. A model strong in one field and weak in another posts the same score as its mirror image. You have to read the rows for the occupations closest to your own work rather than trust the overall win rate.

How should I use these results for my own team? Treat them as a signal that a model can produce professional grade first drafts, then run a small evaluation on your actual deliverables graded by your own experts. Pick the model on the occupations you care about and on cost, and keep a human review step for anything shipped, since the failure mode is plausible and wrong rather than obviously bad.

This post reports results from a controlled evaluation harness under one grading protocol, not an official vendor leaderboard, and win rates will vary with task selection, instructions, and grader pool; validate on your own deliverables before making deployment decisions.