Meta Muse Spark 1.1 Tops SWE-bench Pro: What the Leaderboard Actually Means (September 2026)
As of September 14, 2026, Meta's Muse Spark 1.1 sits at the top of Scale's SWE-bench Pro standardized public set at 61.5 percent and its private commercial set at 51.5 percent, the first model to lead both boards at once. It took the top spot from GPT-5.4 (xHigh) at 59.1 percent and Claude Opus 4.6 at 47.1 percent. That sounds decisive until you notice that three separate numbers, 61.5 percent, 80.0 percent, and 51.5 percent, are all being quoted somewhere as the best SWE-bench Pro score right now, and most pages that cite one never say which board they mean. This post pulls those numbers apart, explains what SWE-bench Pro actually rewards, and works out whether a leaderboard swap should change what your engineering team runs in production.
Meta Muse Spark 1.1 Tops SWE-bench Pro: What the Leaderboard Actually Means (September 2026)
As of September 14, 2026, Meta's Muse Spark 1.1 leads Scale's SWE-bench Pro standardized public set at 61.5 percent and its private commercial set at 51.5 percent. That makes it the first model to sit at the top of both boards at the same time, a distinction that had not happened before. It displaced GPT-5.4 (xHigh), which held the public board at 59.1 percent, and Claude Opus 4.6, which topped the commercial set at 47.1 percent.
| Board | Muse Spark 1.1 | Previous leader | Gap |
|---|---|---|---|
| SWE-bench Pro, standardized public set | 61.5% | GPT-5.4 (xHigh), 59.1% | +2.4 pts |
| SWE-bench Pro, private commercial set | 51.5% | Claude Opus 4.6, 47.1% | +4.4 pts |
Two things are worth doing with this result. The first is understanding what the number rewards, because SWE-bench Pro is stricter and messier than the benchmark most people still picture when they hear "SWE-bench." The second is resisting the reflex to switch production models because a board reshuffled by a couple of points.
The Model Itself
Muse Spark 1.1 is Meta's agentic multimodal model, announced on July 9, 2026 alongside the public preview of the Meta Model API for US developers. It carries a 1 million token context window and up to 131K output tokens, accepts text, images, video, audio, and PDF documents, and is built around tool use, computer use, and coding. API pricing is $1.25 per million input tokens and $4.25 per million output tokens, with cache reads at $0.15 per million. That places it well below frontier closed models on sticker price, which matters for the cost-per-issue math later.
The relevant point for a coding leaderboard is that Muse Spark 1.1 was trained for agentic loops rather than single-shot completion. SWE-bench Pro rewards exactly that behavior, so a strong showing here is consistent with the model's design rather than a surprise.
What SWE-bench Pro Actually Measures
SWE-bench Pro is the harder successor to SWE-bench Verified, and the differences are the whole story. It uses a deterministic harness, so a run is not polluted by flaky tests that pass or fail at random. It draws from a larger and more diverse issue set. And it scores strictly: a patch either makes the hidden tests pass or it does not, with no partial credit for a diff that looks plausible. We went through the mechanics of why Pro and Verified disagree in our breakdown of the two benchmarks, and the summary is that Pro produces lower absolute pass rates but discriminates between models much better.
A leader on Pro is therefore telling you something more specific than a leader on Verified. It means the model resolves real repository issues, end to end, against tests it cannot see, inside an agent loop that has to read code, plan an edit, apply it, and verify. That is close to what a coding agent does in production, which is why Pro is the board worth tracking.
The Three Numbers Problem
Here is where the leaderboard becomes a trap. Three different figures are currently circulating as the best SWE-bench Pro score, and they are not measuring the same thing.
- 61.5 percent: Muse Spark 1.1 on Scale's standardized public set. This is the board where every model runs through the same scaffolding on the same public issue split.
- 51.5 percent: Muse Spark 1.1 again, on Scale's private commercial set. The issues are held back to prevent training contamination, so scores run lower and the ranking can differ from the public board.
- 80.0 percent: Claude Fable 5 in the llm-stats vendor aggregate. This is not the Scale standardized board at all; it collects vendor-reported numbers under vendor-chosen scaffolding, so it is not comparable to the two Scale sets.
The spread between 51.5 and 80.0 is almost entirely scaffolding and data splits, not raw model capability. When a page says a model "leads SWE-bench Pro," the useful follow-up is always the same: which board, which split, and whose scaffolding. A vendor-reported aggregate is a marketing surface until an independent harness reproduces it on a fixed split. The Scale standardized public set is the closest thing to an apples-to-apples comparison, which is why the Muse Spark 1.1 result at 61.5 percent is the one worth anchoring on.
Should This Change What You Run?
A 2.4 point lead on the public board is real, and against the commercial set the 4.4 point margin is more convincing because that split is harder to game. But a leaderboard position is an average across a fixed issue set, and your codebase is not that issue set. Two questions decide whether the swap matters for you.
The first is cost per resolved issue, not pass rate. At $1.25 and $4.25 per million tokens, Muse Spark 1.1 is cheap enough that even a modest pass rate can beat a pricier model on total cost to clear a backlog. Multiply the pass rate by your average token spend per attempt, including retries, and compare the cost to actually close an issue. That number reorders leaderboards constantly, which is the same lesson we drew comparing Fable 5 and GPT-5.5 on cost per resolved issue.
The second is behavior inside your harness. A model that tops SWE-bench Pro under Scale's scaffolding may behave differently under yours, because tool call formatting, retry policy, and context construction all move the result. The Muse Spark family has generally traded on cheap tokens and long context, which we looked at with Muse Spark 1.3, and 1.1 keeps that profile. If your agent rebuilds context on every turn, you lose the cache economics that make the low price compelling.
When This Applies to Your Stack
Treat a leaderboard swap as a prompt to re-run your own evaluation, not as a reason to switch by default. The durable pattern is to keep your agent harness model-agnostic, hold a fixed set of 50 recently closed issues from your own repository as a private test set, and run a quarterly bake-off that measures pass rate, tool call reliability, and cost per resolved issue on that set. When a new leader appears, you feed it into the same harness and read the number that predicts your production behavior rather than someone else's.
Contra Collective builds and maintains the agent infrastructure that makes those bake-offs tractable, from model-agnostic harnesses to the CI integration that lets a coding agent open and validate real pull requests. If your team wants a private SWE-bench-style evaluation wired to your own codebase so a leaderboard change becomes a one-hour test instead of a debate, that is work we can help structure.
FAQ
Is Muse Spark 1.1 actually the best coding model now?
It leads both SWE-bench Pro boards run by Scale as of September 14, 2026, which is the strongest standardized coding signal available. Whether it is best for you depends on cost per resolved issue and how it behaves in your harness, not on the leaderboard position alone.
Why do people quote 80 percent for SWE-bench Pro?
That figure comes from the llm-stats vendor aggregate, which collects vendor-reported scores under vendor-chosen scaffolding. It is not the Scale standardized board and is not comparable to the 61.5 percent public set number. Always ask which board a score refers to.
What is the difference between the public and commercial SWE-bench Pro sets?
The standardized public set uses issues that are openly available, so every model runs on the same known split. The private commercial set holds issues back to reduce training contamination, which lowers scores and can change the ranking. Muse Spark 1.1 currently leads both.
How much does Muse Spark 1.1 cost to run?
API pricing is $1.25 per million input tokens and $4.25 per million output tokens, with cache reads at $0.15 per million. For agentic coding, the number that matters is cost per resolved issue, which folds in retries and cache hit rate.
Should I switch my production agent to Muse Spark 1.1 today?
Not on the leaderboard alone. Run it through your own harness against a private set of your recently closed issues, measure cost per resolved issue and tool call reliability, and switch only if those numbers justify it.
Sources
More from the lab.
Claude Opus 5.5 vs GPT 6 Astra: What the Epoch Capabilities Index Actually Shows (October 2026)
The Epoch Capabilities Index stitches more than fifty distinct benchmarks into one general capability number, and as of October 2, 2026, it has Claude Opus 5.5 at 167.35 against GPT 6 Astra at 166.51, a gap under one point on a scale where the ranking has already moved twice in three weeks. Astra launched at a General ECI of 169, dropped to 166 once more software engineering benchmark results came in, and still holds a record setting Math ECI of 170 that neither Opus 5.5 nor Claude Fable 5.1 has matched. On software engineering specifically, both Astra and Opus trail Fable 5.1, whose SWE ECI of 167 beats Astra's 164. A composite leaderboard this close is more useful for what it reveals about domain specific trade offs than for crowning a single best model.
Gemini 4 Argon: Benchmarks, Pricing, and Why Cyber Defenders Got It First
Gemini 4 Argon shipped on September 30, 2026, and the rollout order is the most telling part of the release: trusted cyber defenders through Google's Fairwind Program get access before the general public does. The model posts a number one AutomationBench rank at 51.3 percent, a 68 percent score on CWE bench v1 vulnerability remediation, and a context window that jumped from 64,000 to one million tokens. Introductory pricing lands at 2 dollars per million input tokens and 10 dollars per million output tokens. This post works through what the numbers mean and what the staged access pattern tells you about how Google is positioning the model.
OpenAI's GPT-6.1 Sol Cuts Frontier Pricing 80 Percent: What DevDay 2026 Actually Changed
OpenAI's DevDay 2026 keynote on September 29 was less about a single headline model and more about six announcements landing at once: GPT-6.1 Sol priced at a fraction of the prior Sol tier, an Ultrafast inference tier running up to eight times faster in Codex, always on Dots agents for Pro and Business Premium, a new Pro 500 subscription tier, and a Decisions API for real time agentic workflows. This post separates what actually shipped from the keynote framing, with the pricing math worked out explicitly rather than taken on faith.