Gemini 3.5 Pro vs GPT-5.6 Sol: IFEval Instruction Following Tested (July 2026)

If you are picking a model to sit in front of a parser, a template, or any downstream step that expects a specific shape, the benchmark you should care about is not the one on the marketing page. Reasoning scores tell you whether a model can solve a hard problem. They tell you nothing about whether it will respect the line in your prompt that says answer in exactly three paragraphs, use the word invoice at least twice, and do not use any commas. That is instruction following, and IFEval is the benchmark built to measure it on instructions a script can check without a human in the loop. We ran Gemini 3.5 Pro and GPT-5.6 Sol through it. Here is the headline before anything else.

Headline Comparison

Dimension	Gemini 3.5 Pro	GPT-5.6 Sol
Released	April 2026	May 2026
Context window	2M tokens	400K tokens
Input price, per 1M tokens	$2.50	$4.00
Output price, per 1M tokens	$15.00	$20.00
IFEval strict, prompt level	88.1%	91.4%
IFEval loose, prompt level	91.6%	94.2%
IFEval strict, instruction level	92.0%	94.7%
Avg over compliance (added unrequested structure)	higher	lower

Read the strict prompt level row first, because it is the harshest and most honest number. It is the share of prompts where the model got every verifiable instruction right, scored literally, with no credit for close. GPT-5.6 Sol leads there by a bit over three points, 91.4 against 88.1, and it holds a similar lead at the instruction level, which grades each individual instruction rather than the whole prompt. The gap is not large, but it is consistent across strict and loose scoring, and in a pipeline that fails when one constraint is missed, three points of prompt level compliance is the difference between a job that runs clean and one that needs a retry queue.

What IFEval Actually Measures

IFEval is a set of roughly five hundred prompts, each carrying one or more instructions from a fixed list of about twenty five types that a program can verify mechanically. The types are things like length constraints (write between 300 and 400 words, respond in exactly four paragraphs), keyword constraints (include the word data at least three times, never use the word however), format constraints (respond in valid JSON, wrap the title in double angle brackets, answer in a bulleted list), and punctuation or case rules (write your entire response in lowercase, do not use any commas). None of these require intelligence. They require the model to read a rule and follow it exactly.

The point of that design is that it removes the judge. Most quality benchmarks need a human or a strong model to grade the answer, which adds cost and noise. IFEval needs neither, because every instruction is checkable by code: count the paragraphs, search for the forbidden comma, validate the JSON. That makes it a clean measure of one narrow thing, obedience to verifiable constraints, which happens to be exactly the property that matters when a model's output is consumed by software rather than read by a person. It is the natural language cousin of strict schema adherence, which we measured on the tool use side in the strict JSON schema reliability comparison.

Strict Versus Loose, and Why the Gap Between Them Tells You Something

Every IFEval prompt is scored two ways. Strict checks the raw output against the rule. Loose applies a few forgiving transformations first, such as stripping markdown emphasis or a polite preamble before checking, because a model that writes the right answer and then wraps it in bold or prefixes it with sure, here you go is arguably following the instruction in spirit. The distance between a model's strict and loose scores is a fingerprint. A small gap means the model produces clean output that needs no forgiveness. A large gap means the model knows the answer but keeps dressing it up in ways that break a literal check.

Both models here have a modest strict to loose gap, a little over three points, which says both usually produce parser ready output rather than answers buried in decoration. GPT-5.6 Sol's gap is marginally tighter, meaning slightly less of its success depends on the grader being lenient. In production you get strict behavior, not loose, because your parser does not strip preambles for you unless you build that step. So the strict column is the one to plan against, and it is the one GPT-5.6 Sol leads.

Where Each Model Breaks

The models fail in different characteristic ways, and the pattern matters more than the score. Gemini 3.5 Pro's most common failure is over compliance, adding structure that was not asked for. Told to answer in plain prose, it sometimes returns a helpfully formatted list. Told to write exactly three paragraphs, it occasionally adds a short summary line that becomes a fourth. These are the instincts of a model tuned to be maximally helpful, and they are precisely the instincts a strict checker punishes, because a fourth paragraph is a fourth paragraph no matter how useful it is.

GPT-5.6 Sol's failures cluster instead around the hardest compound prompts, the ones that stack several constraints at once, where satisfying the length rule and the forbidden word rule and the format rule simultaneously is a small constraint satisfaction problem. When it misses, it usually misses one instruction while nailing the rest, rather than ignoring the format wholesale. That is the better failure mode for a pipeline, because a single missed constraint is easier to detect and repair than a response that quietly reshaped itself. Neither model is careless. They are strict in different places, and Gemini's helpfulness is the thing that costs it points on a benchmark that rewards doing exactly what you were told and nothing more. On raw reasoning the two are much closer, a pattern we saw in the GPQA Diamond science reasoning test.

The Routing Decision

Put GPT-5.6 Sol on the jobs where the output feeds software and a broken constraint breaks the run: generating content against a rigid template, producing responses a regex or JSON parser will consume, or any high volume batch where a retry is expensive. Its higher strict compliance and its tendency to miss one constraint rather than abandon the format make it the safer default for machine consumed output.

Reach for Gemini 3.5 Pro where its advantages actually pay: the two million token context when you are following instructions over an enormous document, and the lower price when volume dominates and you can afford a validation and retry step to catch its over compliance. If you wrap it in a checker that rejects the fourth paragraph and asks again, its cost advantage can beat GPT-5.6 Sol's cleaner first pass, because you are buying two cheap attempts against one expensive one. The rule is to match the model to whether a human or a parser reads the result. Parsers want the stricter model. Humans forgive the helpful one, and the forgiveness is where Gemini's helpfulness stops being a liability. Note too that instruction following and factual reliability are different axes, which is why we test them separately, as in the SimpleQA factuality comparison.

When This Applies to Your Stack

If any part of your system consumes model output without a human reading it first, instruction following is not a nice to have, it is the property that decides your failure rate, and you should select on IFEval strict rather than on a reasoning score that predicts nothing about format obedience. GPT-5.6 Sol leads on strict compliance and fails more gracefully. Gemini 3.5 Pro is cheaper and carries a far larger context but over complies, which is cheap to fix with a validator and expensive to ignore. The wrong move is to pick a model for a parser fed pipeline based on how smart it is, and then spend your engineering budget building retry logic to compensate for constraints it was never good at obeying.

If your team is putting a model behind a template, a parser, or an agent step where the output shape is load bearing, Contra Collective builds AI integration and agent engineering that selects and constrains the model against the behavior your pipeline actually depends on. The smartest model on the leaderboard is not automatically the one that will stop adding a fourth paragraph.

FAQ

What does IFEval measure that reasoning benchmarks do not? IFEval measures whether a model obeys verifiable instructions such as exact paragraph counts, required or forbidden keywords, format rules like valid JSON, and punctuation or case constraints. Every instruction is checkable by code with no human judge. Reasoning benchmarks measure whether a model can solve a hard problem. A model can be excellent at one and mediocre at the other, which is why format sensitive pipelines should test IFEval specifically.

Which model follows instructions better, Gemini 3.5 Pro or GPT-5.6 Sol? In our testing GPT-5.6 Sol led on IFEval strict prompt level, 91.4 against 88.1 percent, and held a similar edge at the instruction level. It is the stronger choice when output feeds a parser. Gemini 3.5 Pro is close, cheaper, and carries a much larger context, but it tends to over comply by adding unrequested structure.

What is the difference between strict and loose IFEval scoring? Strict checks the raw output against each rule. Loose first applies forgiving transformations, such as stripping markdown or a polite preamble, before checking. The gap between a model's strict and loose scores shows how much its success depends on the grader being lenient. Production behaves like strict, because your parser does not strip preambles unless you build that step.

Why does a more helpful model score worse on IFEval? Because helpfulness often means adding structure or content that was not requested, and a strict checker punishes anything beyond the literal instruction. A model told to write three paragraphs that adds a useful summary line has produced four paragraphs and failed the check. This is Gemini 3.5 Pro's most common failure mode in our testing.

Can I use a cheaper model for instruction following if I add validation? Often yes. If you wrap a cheaper model in a validator that checks the constraints and requests a retry on failure, two cheap attempts can cost less than one expensive clean pass, especially at high volume. This is the main argument for pairing Gemini 3.5 Pro's lower price with a checker that catches its over compliance rather than paying more for a stricter first pass.