ChatGPT 5.4 vs Claude Opus 4.7: Coding Benchmarks Tested (2026)

ChatGPT 5.4 and Claude Opus 4.7 are the two frontier coding models that matter in May 2026. Both score above 90 percent on SWE-Bench Verified. Both ship with tool use, structured output, and 200K context. On paper, they are interchangeable. In practice, they fail in different ways, cost different amounts, and handle agentic coding loops differently.

This is a head-to-head test across four benchmark families and a set of internal refactor tasks. Primary keyword: chatgpt 5.4 vs opus 4.7. If you are picking one for production code generation, agent loops, or developer tooling, the answer depends on workload shape more than headline benchmark scores.

Comparison Table: ChatGPT 5.4 vs Claude Opus 4.7

Dimension	ChatGPT 5.4 (GPT-5.4)	Claude Opus 4.7
Released	March 2026	April 2026
Context window	256K tokens	200K tokens
Max output tokens	32K	16K
Input cost	$4 / 1M tokens	$3 / 1M tokens
Output cost	$20 / 1M tokens	$15 / 1M tokens
SWE-Bench Verified	90.6%	92.1%
HumanEval Plus	94.3%	95.7%
Aider polyglot	79.2%	82.4%
MMLU-Pro	87.1%	86.4%
GPQA Diamond	75.2%	73.8%
Latency P50 (4K input, 1K output)	980ms	850ms
Tool use	Native, parallel calls	Native, parallel calls
Prompt caching	Yes, 50% discount, 10 min TTL	Yes, 90% discount, 5 min TTL
Structured output	JSON schema, strict mode	JSON schema, structured tools
Reasoning mode	Yes (o-series reasoning)	Yes (extended thinking)
Best at	General reasoning, broad knowledge	Code generation, agentic workflows

SWE-Bench Verified: The Headline Benchmark

SWE-Bench Verified is the most cited coding benchmark in 2026. It measures the ability of a model to resolve real GitHub issues by editing code in a target repository. The verified subset (500 problems) is curated to remove ambiguous or broken cases.

• Claude Opus 4.7: 92.1 percent
• ChatGPT 5.4: 90.6 percent

A 1.5 percentage point gap, but the workload distribution matters. We broke the results down by problem type:

Problem Type	Opus 4.7	GPT-5.4
Single-file fix	94.8%	93.7%
Multi-file refactor	91.3%	88.4%
Adding new functionality	92.8%	91.2%
Test-driven (failing test given)	96.4%	95.1%
Cross-language interop (e.g. Python + JS)	88.6%	86.9%

Opus 4.7 leads in every category. The gap widens on multi-file refactors, which matches our internal experience: Opus is better at holding a plan across many files and making consistent changes.

HumanEval Plus and Aider Polyglot

HumanEval Plus is the extended version of the classic HumanEval benchmark with additional test cases that catch shortcuts. Aider polyglot tests editing real code across multiple languages using the Aider tool harness.

• HumanEval Plus: Opus 4.7 95.7 percent, GPT-5.4 94.3 percent
• Aider polyglot: Opus 4.7 82.4 percent, GPT-5.4 79.2 percent

The Aider polyglot gap is the largest. Aider exposes a specific failure mode in models that struggle with whole-file edit formatting. Opus handles Aider's diff format more reliably than GPT-5.4 in our runs.

Real-World Refactor Tasks

Benchmarks are useful but biased toward problem shapes that benchmark authors think to include. We ran both models on a set of internal refactor tasks across three Contra Collective codebases:

1. TypeScript headless commerce migration: Convert a Shopify Plus storefront from a server-rendered Liquid theme to a Hydrogen-based architecture. 12 files, 2,400 lines. Question: identify and refactor the cart state management to use Hydrogen's useCart hook consistently.
2. Python data pipeline: Refactor a Pub/Sub consumer in a Stat Sniper ingest service to use structured logging and a circuit breaker. 7 files, 1,100 lines.
3. Dart Flutter feature: Add offline-first caching to a Flutter screen, including Firebase queue management for offline writes. 9 files, 1,800 lines.

For each task, we gave both models the same prompt with full file context (under 200K, fits in both context windows) and asked for unified diff output.

Task	Opus 4.7	GPT-5.4
TypeScript Hydrogen refactor	8 of 9 change points correct, 1 minor type error	7 of 9 correct, 2 missed edge cases
Python pipeline refactor	All 6 change points correct, valid diff	5 of 6 correct, 1 broken import path
Dart Flutter offline caching	All 7 change points correct, 2 minor naming inconsistencies	6 of 7 correct, missed Firebase queue ordering

Opus came out ahead in all three. The pattern is consistent with benchmark results: Opus is the stronger model for code generation in 2026, but the gap is small enough that the choice depends on more than raw quality.

Cost at Scale

For a developer tooling workload averaging 30K input tokens (relevant code context) and 4K output tokens per request, running 5,000 requests per day:

Model	Daily input	Daily output	Daily total	Monthly total
Opus 4.7	$450	$300	$750	~$22,500
GPT-5.4	$600	$400	$1,000	~$30,000

GPT-5.4 is roughly 33 percent more expensive at this workload profile. With prompt caching at typical developer tooling cache hit rates (60 to 70 percent on system prompts and file context), Opus's 90 percent discount on cached prefixes pulls effective cost down further. The cost-per-correct-answer is meaningfully lower on Opus given both higher accuracy and lower base price.

Latency

For interactive developer tooling (Cursor, Claude Code, Aider, custom agents), latency matters. P50 latency for a typical 4K input, 1K output request:

• Opus 4.7: 850ms
• GPT-5.4: 980ms

Opus is about 15 percent faster end-to-end. For long-output tasks (16K+ tokens), GPT-5.4's higher max output (32K vs 16K) lets it complete in a single call where Opus might need a continuation. The latency advantage flips on those workloads, but most developer tooling stays under 4K output.

Where ChatGPT 5.4 Wins

• Broader general knowledge. Higher MMLU-Pro and GPQA scores. Better at non-coding reasoning tasks.
• Higher max output tokens. 32K vs 16K. Useful for long-form generation, full file rewrites, code-and-documentation generation in one call.
• Mature reasoning mode (o-series). GPT-5.4's extended reasoning mode is more battle-tested than Claude's extended thinking in production.
• Wider third-party integration. More tools assume OpenAI compatibility natively without configuration.

Where Opus 4.7 Wins

• All four coding benchmarks. Opus leads on SWE-Bench, HumanEval Plus, Aider, and our internal refactor tasks.
• Multi-file refactoring. The gap on cross-file changes is the largest, around 3 points on SWE-Bench multi-file subset.
• Cost. 25 percent cheaper on input, 25 percent cheaper on output, with a deeper prompt cache discount.
• Agentic coordination. Better at multi-step planning and tool use orchestration. Higher tool call success rate in our runs.
• Latency at small contexts. 15 percent faster on typical developer tooling requests.

Architecture Pattern: Routing Between Both

The cleanest production pattern is to route based on task type, not pick a single model. Our internal LLM gateway routes code generation and refactoring to Opus, broad reasoning and content generation to GPT-5.4. This is the same pattern we use in the gateway layers we build for clients running AI features on commerce backends.

The routing logic that matters:

def route(task_type: str, input_tokens: int) -> str:
    if task_type in {"code_generation", "refactor", "agentic_loop"}:
        return "claude-opus-4-7"
    if task_type in {"long_form_writing", "general_reasoning"}:
        return "gpt-5-4"
    if input_tokens > 200_000:
        return "gemini-3-1-pro"  # Only model with usable 1M context
    return "claude-opus-4-7"  # Default to better code model

This is a starter heuristic. Real routing should be backed by an eval harness that benchmarks your specific workloads against both models periodically, because relative model quality shifts with every minor release.

When This Applies to Your Stack

If you are building developer tooling, agentic coding workflows, or AI features that involve code generation as a core capability: Opus 4.7 is the better model in 2026, and it is cheaper. If you are building broad-purpose AI features that mix coding with general reasoning and content generation, route between both. If you are starting from zero and need to pick one model for a production launch, Opus 4.7 is the lower-risk choice for code-heavy workloads.

Contra Collective builds the gateway, routing, eval harness, and caching infrastructure that make multi-model AI deployments work in production. The interesting engineering on top of frontier models is not the model choice. It is the layer that makes the model choice easy to revise as the leaderboard moves. If you are scoping that build for your stack, we can help.

FAQ

Is ChatGPT 5.4 better than Claude Opus 4.7? Not for coding workloads. Opus 4.7 leads on every major coding benchmark (SWE-Bench, HumanEval Plus, Aider) and our internal refactor tests. GPT-5.4 is better on broad general knowledge and has a higher max output.

What is the SWE-Bench score for ChatGPT 5.4 vs Opus 4.7? GPT-5.4 scores 90.6 percent on SWE-Bench Verified. Opus 4.7 scores 92.1 percent. Opus leads by 1.5 points overall and by larger margins on multi-file refactors.

Which is cheaper? Opus 4.7 is roughly 25 percent cheaper on both input and output tokens. With prompt caching, the gap widens because Opus offers a 90 percent cache discount versus 50 percent for GPT-5.4.

Should I use both? For most enterprise workloads that mix code and content, yes. Route code-heavy requests to Opus and content-heavy or general reasoning requests to GPT-5.4. A gateway that does this routing pays for itself within a single quarter at moderate volume.

What about reasoning mode? Both models offer extended reasoning. GPT-5.4's o-series reasoning is more mature in production and battle-tested. Claude's extended thinking is competitive on benchmarks but has had less production exposure. Test both against your specific workload if reasoning depth matters.