Terminal-Bench 2.1 Leaderboard: DeepSeek V4.1 Flash, GLM 5.3, and Kimi K3 Tested (September 2026)

Terminal-Bench 2.1 is the harder revision of the benchmark that drops an agent into a real shell and scores whether it can finish command line tasks end to end, not describe how it would. The public board is led by DeepSeek V4.1 Flash at 90.6 percent. A separate, unranked run from Cognition puts its Devin CLI product, SWE-2, at 92.8 percent, a number worth understanding rather than just repeating, because it did not go through the same process as everything ranked below it.

The leaderboard as it actually stands

Two things jump out immediately. First, the top seven ranked entries sit inside a six point band, 84.3 to 90.6 percent, which is a much tighter spread than earlier Terminal-Bench versions produced. Second, GLM 5.3 and Kimi K3, the two open weight entries near the top, land within 0.1 points of each other and within 2.4 points of the leading closed model. Terminal-Bench's own documentation flags that 2.1 scores are not directly comparable to earlier versions, because 2.1 raised task difficulty specifically to stop the ceiling clustering that made 1.x versions less useful for separating strong agents from very strong ones.

Why the Cognition number does not belong in the same sentence as the rest

92.8 percent from Cognition's SWE-2 is the highest number in this post, and it is also the number you should trust the least, not because Cognition is dishonest, but because it was not measured the same way. The ranked board entries ran through Terminal-Bench's standard harness under conditions any lab can inspect and reproduce. Cognition's number is a provider reported result on Devin CLI, run internally and published without the same third party verification path. That does not make it fake. It makes it a different category of claim, the same category our own coverage of why SWE-bench Pro and SWE-bench Verified scores disagree points to: a benchmark score is only as comparable as the methodology behind it, and a private run reported by the vendor whose product it measures is structurally the weakest form of that evidence, regardless of the number attached to it.

What the open weight showing actually means

GLM 5.3 landing at 88.2 percent, essentially tied with Kimi K3 and within striking distance of GPT-5.6 Sol, is the more interesting story than the leaderboard order itself. Six months ago, an open weight model within 2.4 points of a frontier closed model on a genuinely hard agentic benchmark, one specifically redesigned to stop ceiling clustering, would have been the headline. Now it is one line in a leaderboard table, because it has started to be the expected outcome rather than the surprise.

For a team deciding what to run in production, that closes a gap that used to be a much easier decision. GLM 5.3 at $1.40 input and $4.40 output per million tokens is meaningfully cheaper than GPT-5.6 Sol's published rate, for a terminal agentic task score within 0.6 points. GLM 5.3 Flash, the 18B MIT licensed variant, trades another four and a half points of score for a model small enough to consider self hosting, which changes the cost conversation entirely if your volume is high enough to make GPU serving cheaper than API calls.

Picking between the top tier

When this applies to your stack

If you are choosing a model for an agent that runs real terminal commands, whether that is CI automation, infrastructure changes, or a coding agent with shell access, the tight spread at the top of this leaderboard means price and self hosting feasibility should carry more weight in your decision than the last two or three points of benchmark score. The score gap between DeepSeek V4.1 Flash at the top and GLM 5.3 in fifth place is smaller than the noise you will see from prompt and harness differences in your own evaluation.

Contra Collective evaluates and deploys agentic coding tooling for engineering teams, including running our own harness against the same tasks a vendor benchmark claims to measure, before a model goes anywhere near production infrastructure. If a vendor's number is the only evidence you have for a model that will run commands against your systems, that is worth closing before you ship it.

FAQ

Is Terminal-Bench 2.1 comparable to Terminal-Bench 2.0 or 4.0 scores? No. Terminal-Bench 2.1 raised task difficulty specifically because earlier versions were clustering too many strong models near the ceiling. A score from 2.1 should not be compared directly to a 2.0 or 4.0 score for the same model.

Why is Cognition's SWE-2 number not on the ranked board? It is a provider reported result rather than a run through the standard public harness. Ranked entries on Terminal-Bench's board go through a process other labs can inspect and reproduce; Cognition's number has not gone through that same path as of this writing.

Is GLM 5.3 actually production ready for agentic terminal tasks? The benchmark result suggests it is competitive with closed frontier models on this specific task type. As with any benchmark number, run it against your own workload before committing, since Terminal-Bench tasks are a proxy for your actual use case, not a guarantee of it.

Why does the score spread matter more than who is technically first? A six point spread across the top seven ranked models is well within the range that prompt design, harness configuration, and task sampling can shift in your own testing. Treating a 2.4 point gap as decisive is a mistake most teams make once and then stop making.