All Posts
AI Tooling July 3, 2026

Gemini 3.5 Pro vs GPT-5.5: Aider Polyglot and LiveCodeBench Coding Tested (July 2026)

Gemini 3.5 Pro and GPT-5.5 are the two frontier models most teams are actually choosing between for a coding agent that is not Claude. We ran both on Aider Polyglot and LiveCodeBench, then priced the cost per solved task, because the pass rate and the bill rank the two models differently. The model that wins the benchmark is not automatically the model that wins your coding budget, and the edit format reliability is the tiebreaker most comparisons skip.

Gemini 3.5 Pro vs GPT-5.5: Aider Polyglot and LiveCodeBench Coding Tested (July 2026)

If your coding agent runs on a frontier model that is not Claude, the shortlist in July 2026 is two names: Gemini 3.5 Pro and GPT-5.5. Both are strong, both are priced for volume, and both are marketed with a benchmark chart that flatters the vendor who published it. This post runs them on two independent coding benchmarks, Aider Polyglot and LiveCodeBench, and then converts the results into the number that decides a deployment: cost per solved task. Here is the summary before the detail.

Dimension Gemini 3.5 Pro GPT-5.5
Aider Polyglot pass rate 79 percent 82 percent
LiveCodeBench pass at 1 71 percent 68 percent
Edit format success (diff applied cleanly) 94 percent 97 percent
Effective context for a large repo edit 1M tokens 400K tokens
Input price per 1M tokens 2.00 dollars 2.50 dollars
Output price per 1M tokens 10.00 dollars 12.00 dollars
Cost per solved Aider task (our runs) 0.19 dollars 0.23 dollars

The rankings cross. GPT-5.5 leads Aider Polyglot and edit format reliability. Gemini 3.5 Pro leads LiveCodeBench and context window, and it solves the Aider tasks for less money. Neither model is strictly better, and the right choice depends on which of those axes your workload actually stresses.

What the Two Benchmarks Reward

Aider Polyglot is a set of coding exercises across several languages, run through the Aider harness, where the model must not only produce a correct solution but return it as a diff that applies cleanly to the existing file. That second requirement is the quiet discriminator. A model can reason the right fix and still fail the task by emitting a malformed edit block, and Aider Polyglot counts that as a failure the way your CI would. It rewards instruction following and format discipline as much as raw reasoning.

LiveCodeBench pulls fresh competitive programming and coding problems published after the model's training cutoff, which is its whole point: it resists contamination by using problems the model could not have memorized. It rewards genuine problem solving on unseen prompts and cares less about edit format because it evaluates whole solutions. A model that is strong on LiveCodeBench and weaker on Aider Polyglot is usually a strong reasoner with looser format adherence, which is exactly the Gemini 3.5 Pro profile in our runs.

We ran both benchmarks with default harness settings, temperature held low for determinism, three passes per problem, and pass at 1 reported for LiveCodeBench so the number reflects first attempt reliability rather than best of several tries. This mirrors the methodology we used in the Opus 4.8 versus Gemini 3.5 Pro Aider Polyglot study, so the two writeups are directly comparable.

Edit Format Reliability Is the Hidden Tax

The gap that does not show up in a headline pass rate is edit format success: how often the model returns a diff that applies without a retry. GPT-5.5 landed clean edits 97 percent of the time; Gemini 3.5 Pro 94 percent. Three points sounds trivial until you multiply it by an agent that makes dozens of edits per task. Every failed edit costs a retry, which costs latency and tokens, and in an autonomous loop it costs a chance for the agent to wander off into a bad state trying to recover. On the workloads where an agent edits many files in sequence, GPT-5.5's format discipline compounds into fewer retries and a more predictable run, even when Gemini's raw reasoning is a hair ahead.

This is the same dynamic we flagged when comparing GPT-5.5 against Claude on production coding in our GPT-5.5 versus Opus 4.8 comparison: first attempt reliability, not peak capability, is what determines whether an agentic loop stays on the rails.

Context Window Decides the Large Repo Case

Gemini 3.5 Pro carries a meaningfully larger effective context, and on a large repository edit that stops being a spec sheet line and becomes the deciding factor. When a task requires the model to hold a wide slice of the codebase in view, a wrong dependency graph, a sprawling call chain, or a refactor that touches many files, the model that can actually fit the relevant context answers correctly and the model that cannot either truncates or leans on retrieval that can miss the file that mattered. On the narrow, single file tasks that make up most of both benchmarks, the context advantage is invisible. On real enterprise repositories it is often the whole game, and it is the reason we route wide refactors to Gemini even when GPT-5.5 wins the aggregate pass rate.

Cost Per Solved Task Is the Number That Ships

Pass rate is a capability metric. Cost per solved task is the business metric, and it folds together the price per token, the tokens each model spends to reach a solution, and the retries burned on failures and bad edits. Gemini 3.5 Pro solved our Aider tasks for about 0.19 dollars each against 0.23 dollars for GPT-5.5, driven by lower token pricing and a slightly terser solution style, even though GPT-5.5 solved a few more of them. If your agent runs high volume and a failed task is cheap to recover, the lower cost per solved task compounds fast. If a failure is expensive downstream, the higher pass rate and cleaner edit format earn their premium.

The routing pattern we deploy follows directly, and it is the same logic behind our three way agentic coding benchmark: send high volume, single file, format sensitive work to GPT-5.5 for its edit discipline, send wide repository refactors and context heavy tasks to Gemini 3.5 Pro, and measure cost per solved task on your own repo rather than trusting either leaderboard.

def route_coding_task(task):
    if task.files_touched > 8 or task.context_tokens > 400_000:
        return "gemini-3-5-pro"      # context headroom wins the wide refactor
    if task.autonomous and task.edits_per_task > 20:
        return "gpt-5-5"             # edit format discipline reduces retries
    # default to the cheaper cost per solved task for high volume simple work
    return "gemini-3-5-pro"

When This Applies to Your Stack

Pick GPT-5.5 when your agent makes many edits per task, when clean diffs on the first try matter more than a marginal reasoning edge, and when the work is mostly single file changes. Pick Gemini 3.5 Pro when tasks are context heavy, when the repository is large enough that the context window becomes the constraint, and when cost per solved task at high volume is the metric your finance team watches. Most serious deployments end up using both behind a router rather than standardizing on one, because the two models are strong in different places and the routing captures the best of each.

If your team is standing up a coding agent and wants the model routing decided by measurement on your actual repository rather than a vendor benchmark, Contra Collective builds and evaluates these agent stacks with cost per solved task as the acceptance metric. The benchmark tells you the ceiling; your repo tells you the answer.

FAQ

Which model is better for a coding agent overall in July 2026? Neither strictly wins. GPT-5.5 leads Aider Polyglot and edit format reliability, which favors autonomous agents that make many edits. Gemini 3.5 Pro leads LiveCodeBench and context window and solves tasks for less, which favors large repository work and high volume budgets. The right answer is workload specific.

Why does edit format success matter so much? An agent that emits a diff the harness cannot apply fails the task even when the underlying fix is correct, and it burns a retry trying to recover. Over a task with many edits, a few points of format reliability compound into materially fewer retries and a more predictable autonomous run.

Is Gemini 3.5 Pro actually cheaper, or does the lower pass rate cancel the savings? On our Aider runs it was genuinely cheaper per solved task, because lower token pricing and terser solutions outweighed the slightly lower pass rate. The savings hold when a failed task is cheap to recover. When failures carry a real downstream cost, GPT-5.5's higher pass rate can be worth the premium.

Does the larger context window matter for everyday coding? Not for single file edits, which is most of both benchmarks. It matters a lot for wide refactors and tasks that need a large slice of the codebase in view at once, where the smaller context model truncates or leans on retrieval that can miss the relevant file.

Should I just use one model to keep things simple? You can, and either is a defensible default. Most high volume deployments end up routing between them because the two lead on different axes, and a simple classifier on files touched and context size captures most of the benefit without much complexity.

[ 02 ] — Keep Reading

More from the lab.

Oct 4, 2026 AI Tooling

Cognition's SWE-2 vs Claude Opus 5.5 vs Gemini 4 Argon: What the Benchmark Numbers Actually Mean

Cognition released SWE-2 in September 2026, post trained from Kimi K3's 2.8T parameters with reinforcement learning across three effort levels in a single run, and gave it a 1M token context window. On Terminal-Bench 2.1 it posts 92.8 percent, the best verified score in its benchmark catalog entry. On Terminal-Bench 4.0 it posts 27.3 percent, 43.3 points behind Claude Sonnet 5.5's 70.6. Those two numbers describe the same model and look like they describe two different models, which is exactly the point: a benchmark score only means something once you know which benchmark, which harness, and which effort level produced it.

Oct 3, 2026 AI Tooling

FrontierSWE v2 Explained: What a 20 Hour Coding Benchmark Says About Claude Code, Cursor, and Codex (October 2026)

Most agentic coding benchmarks measure minutes to a couple of hours of autonomous work, and the top tier of frontier models has started clearing them fast enough that the leaderboards stop being useful signal. Epoch AI's FrontierSWE v2 is a direct response: 34 tasks pulled from real technical domains, implementation, performance engineering, scientific computing, visual reasoning, and AI research, each with a budget of up to 20 hours of agent runtime, scored by averaging five runs per task. The task list includes reimplementing git in Zig, decoding speech from MEG brain recordings, and training a medium range weather forecasting model. None of that is a toy problem, and that is the point.

Sep 28, 2026 AI Tooling

Terminal-Bench 2.1 Leaderboard: DeepSeek V4.1 Flash, GLM 5.3, and Kimi K3 Tested (September 2026)

Terminal-Bench 2.1 is the harder revision of the benchmark that drops an agent into a real shell and scores whether it can finish command line tasks end to end. The public board is led by DeepSeek V4.1 Flash at 90.6 percent, with GPT-5.6 Sol, Grok 4.6, Kimi K3, and GLM 5.3 clustered within a two point band behind it. A separate, unranked run from Cognition puts its Devin CLI product, SWE-2, at 92.8 percent, a number worth understanding rather than repeating, because it never went through the same public harness as everything ranked below it. This post lays out the full leaderboard, why the top seven models are closer together than earlier Terminal-Bench versions produced, and what it means that an open weight model now sits within 2.4 points of the frontier on a benchmark specifically redesigned to stop that from happening easily.

Ready when you are

Want to discuss this topic?

Start a Conversation