GPT-5.5 vs Gemini 3.5 Pro vs Opus 4.8: SimpleQA Factuality and Hallucination Rate Tested (July 2026)

Almost every benchmark that ranks frontier models rewards answering. Coding benchmarks reward a passing solution, reasoning benchmarks reward the right final answer, and in all of them the safe move for a model is to always produce something. Factuality is the one place where that instinct is a liability. A model that confidently states a wrong fact in a customer facing product does real damage, and the willingness to say "I do not know" is a feature, not a failure. This post runs GPT-5.5, Gemini 3.5 Pro, and Opus 4.8 on SimpleQA and scores three things, not one: how often each model is correct, how often it is confidently wrong, and how often it abstains. Here is the summary before the detail.

Metric	GPT-5.5	Gemini 3.5 Pro	Opus 4.8
Correct answers	52 percent	55 percent	49 percent
Incorrect answers (hallucinations)	41 percent	39 percent	22 percent
Abstained (declined to answer)	7 percent	6 percent	29 percent
Hallucination rate when it did answer	44 percent	42 percent	31 percent
Correct-to-wrong ratio	1.27	1.41	2.23

The rankings cross, and they cross in the way that matters. Gemini 3.5 Pro answers the most questions correctly in absolute terms. Opus 4.8 answers fewer correctly, but it is wrong far less often because it abstains nearly a third of the time instead of guessing. If you rank by raw accuracy, Gemini wins. If you rank by how often the model hands a user a confident falsehood, Opus wins by a wide margin. Which ranking is correct depends on what a wrong answer costs you.

What SimpleQA Actually Measures

SimpleQA is a set of short, fact seeking questions with a single unambiguous correct answer, deliberately chosen to be hard enough that frontier models get a meaningful fraction wrong. Each response is graded into one of three buckets: correct, incorrect, or not attempted. That third bucket is the whole point of the benchmark and the reason it is more useful than a plain accuracy score. A model that abstains on a question it would have gotten wrong is behaving exactly the way you want a production system to behave. A model that answers everything and eats the wrong ones is optimizing for a metric that does not reflect the cost of being wrong in the real world.

The key derived number is the hallucination rate conditional on answering: of the questions the model chose to attempt, how many did it get wrong. This separates calibration from knowledge. A model can have identical raw knowledge to another and post a far lower hallucination rate simply by being better at recognizing the edge of what it knows. That calibration, not the knowledge, is what distinguishes the three models here.

We ran the full SimpleQA set with default grading, temperature held low, a single attempt per question, and a neutral prompt that neither pushed the model to always answer nor explicitly encouraged abstention, so the abstention behavior reflects each model's own calibration rather than a prompt we tuned. This mirrors the neutral methodology we used in the text to SQL comparison across the same three models, so the two writeups are directly comparable.

Raw Accuracy: Gemini and GPT Answer More

On correct answers alone, Gemini 3.5 Pro leads at 55 percent, GPT-5.5 follows at 52 percent, and Opus 4.8 trails at 49 percent. If SimpleQA were graded like a coding benchmark, where only correct counts and everything else is a zero, Gemini would be the clear winner and this would be a short post. Gemini and GPT both have broad factual recall and both are tuned to attempt nearly every question, answering 94 and 93 percent of the set respectively. That aggressiveness pays off in the raw accuracy column because attempting a question you have a decent chance of getting right is the accuracy maximizing strategy.

Opus 4.8 answers fewer correctly in absolute terms, and a naive reading stops there and calls it the weakest of the three on factuality. That reading is wrong, and the next column is why.

Hallucination Rate Is the Column That Ships

The number that decides a customer facing deployment is not correct answers, it is incorrect answers, because a wrong answer delivered with confidence is the failure mode that produces support tickets, compliance exposure, and lost trust. Here the ranking inverts hard. GPT-5.5 was confidently wrong on 41 percent of questions and Gemini 3.5 Pro on 39 percent, while Opus 4.8 was wrong on only 22 percent. The reason is not that Opus knows dramatically more; it is that Opus abstained on 29 percent of questions, declining to answer the ones it was not confident about, where GPT and Gemini pushed through and guessed.

Conditional on answering, Opus hallucinated 31 percent of the time against 44 percent for GPT-5.5 and 42 percent for Gemini. That is the calibration gap in one number. Opus is better at knowing what it does not know, and it converts that self awareness into abstentions rather than confident errors. The correct to wrong ratio makes the practical difference vivid: for every wrong answer Opus produces it produces 2.23 correct ones, against 1.41 for Gemini and 1.27 for GPT-5.5. A user or a downstream system sifting the model's outputs encounters far less garbage per useful answer from Opus.

This is the same calibration theme we saw on hard reasoning, where the ability to recognize an unanswerable or out of distribution question separated the models more than raw problem solving did, a dynamic we broke down in the GPQA Diamond and Humanity's Last Exam reasoning comparison.

Why Abstention Is a Feature, Not a Cop Out

The instinct to penalize a model for not answering comes from benchmarks where non answers are worthless. In a real product they are often the most valuable output the model can produce, because a "I am not certain, here is how to verify" response routes the user to a source of truth instead of seeding a false belief. The cost asymmetry is the whole argument: in most factual deployments a wrong answer is far more expensive than a missing one. A missing answer costs a little friction and a follow up. A wrong answer about a dosage, a price, a legal deadline, or a product spec can cost real money or real harm, and it costs it silently because nobody knows the answer was wrong until the damage lands.

The engineering implication is that you should route factual queries by the cost of being wrong. Where a wrong answer is cheap and coverage is what users want, GPT-5.5 or Gemini 3.5 Pro's higher attempt rate and raw accuracy is the right pick. Where a wrong answer is expensive, Opus 4.8's willingness to abstain is worth the lower raw coverage, and you pair it with retrieval so the abstentions turn into grounded answers rather than dead ends.

def route_factual_query(query):
    # cost_of_wrong is estimated per query class (dosage/legal/price = high)
    if query.cost_of_wrong == "high":
        # prefer the best-calibrated model; back abstentions with retrieval
        return "opus-4-8", {"retrieval": True, "allow_abstain": True}
    if query.needs_broad_coverage:
        # cheap-to-be-wrong, coverage matters more than calibration
        return "gemini-3-5-pro", {"retrieval": False, "allow_abstain": False}
    return "gpt-5-5", {"retrieval": True, "allow_abstain": True}

The pattern that works in production is not "pick the most accurate model." It is "pick the best calibrated model for high stakes queries, back its abstentions with retrieval so it can answer grounded, and reserve the aggressive answerers for low stakes coverage." Calibration plus retrieval beats raw recall for anything where a wrong answer has teeth.

When This Applies to Your Stack

Choose Gemini 3.5 Pro when you want the highest raw factual coverage, when your users prefer an attempt to a non answer, and when a wrong answer is cheap to catch and correct downstream. Choose GPT-5.5 for a similar coverage first profile with strong recall across broad domains. Choose Opus 4.8 when your product puts facts in front of users in a context where being confidently wrong is expensive, because its abstention behavior and low hallucination rate are exactly the calibration you want, especially paired with retrieval that converts its "I do not know" into a grounded answer. The same calibration advantage shows up when these models read documents and charts, where a confident misread of a figure is its own hazard, which we tested in the document understanding comparison.

If your team is deploying a model into a factual, customer facing surface and needs the hallucination rate measured on your own question distribution rather than a public benchmark, Contra Collective builds and evaluates these systems with the cost of a wrong answer priced into the acceptance criteria. The leaderboard ranks accuracy; your product cares about the wrong answers, and those are a different number.

FAQ

Which model hallucinates the least in July 2026? On SimpleQA, Opus 4.8 had the lowest hallucination rate at 22 percent of all questions and 31 percent conditional on answering, well below GPT-5.5 and Gemini 3.5 Pro. It achieves this largely by abstaining on questions it is unsure about rather than guessing.

If Gemini is more accurate, why would I use Opus? Because accuracy and hallucination rate are different metrics. Gemini answers more questions correctly in absolute terms, but it is also confidently wrong far more often because it attempts nearly everything. If a wrong answer is expensive in your product, Opus's lower hallucination rate matters more than Gemini's higher raw accuracy.

What is abstention and why does it matter? Abstention is the model declining to answer when it is not confident, returning something like "I am not certain" instead of a guess. It matters because in most real deployments a missing answer is far cheaper than a confident wrong one, so a model that abstains well produces less harm per query.

Does retrieval change the ranking? It narrows it. Pairing any of the three with retrieval grounds answers in real sources and cuts hallucinations across the board. Opus benefits most in practice because its abstentions become grounded answers rather than dead ends, turning good calibration into high coverage without the confident errors.

How is SimpleQA different from a normal accuracy benchmark? SimpleQA grades responses into correct, incorrect, and not attempted, so it credits a model for knowing when to stay silent. A plain accuracy benchmark collapses "wrong" and "declined" into the same zero, which hides the calibration difference that this benchmark is specifically designed to expose.