Claude Haiku 4.5 vs Fable 5 Mini vs Gemini 3.1 Flash on Terminal-Bench 2: Cheap Tier Agentic Coding Tested (June 2026)

Terminal-Bench 2 is the benchmark that maps closest to how a working terminal agent actually performs in production. The 187 task set spans real shell sessions (file system mutations, package installation, build pipelines, log forensics, git history surgery, container debugging) and grades the model on whether the task completes end to end, not whether any single tool call returned a plausible string. The harness measures tool call accuracy, recovery from errors, and the wall clock budget the agent burns on backtracking. For the cheap tier routing slot in a production agent fleet, the relevant comparison is the three models a team would credibly default to: Claude Haiku 4.5, Fable 5 Mini, and the newly released Gemini 3.1 Flash.

Headline Comparison

Fable 5 Mini wins the pass rate by 4.3 percentage points over Haiku 4.5 and 9.5 percentage points over Gemini 3.1 Flash. Gemini 3.1 Flash wins cost per resolved task by a wide margin even after accounting for its lower pass rate, because the pricing per million tokens is roughly a third of the other two. Haiku 4.5 wins tool call accuracy by a thin margin and remains the most predictable choice for agents where a malformed tool call cascades into a failed task. The three models compete for the same cheap tier slot, and the right choice depends on whether the routing logic optimizes for pass rate, cost per resolved task, or the operational simplicity of one model that handles every task without a fallback path.

We ran the full Terminal-Bench 2 benchmark (187 tasks across 12 categories) against each model in single shot mode with a shared tool harness (bash execution, file read/write, package management, git operations). Each task was attempted three times per model and the median result was recorded. The harness logged tool call accuracy (well formed JSON, valid arguments, executable command), error recovery (whether the agent diagnosed and corrected after a failed call), and the wall clock budget per task.

Why Fable 5 Mini Wins the Pass Rate

Fable 5 Mini's 51.6 percent pass rate is the highest of the three by a meaningful margin. The reason is the error recovery rate (76.8 percent), which is structurally higher than Haiku 4.5 (71.4 percent) and dramatically higher than Gemini 3.1 Flash (58.2 percent). Terminal tasks fail constantly in the inner loop: a package install fails because the lock file is stale, a build fails because an environment variable is missing, a git rebase fails because of a conflict the agent did not anticipate. The pass rate on Terminal-Bench 2 is dominated by whether the model recovers from those failures or quits, and Fable 5 Mini quits less.

The recovery advantage shows up especially on the multi step categories. On the "diagnose and fix a broken CI pipeline" category (24 tasks), Fable 5 Mini passes 14, Haiku 4.5 passes 11, and Gemini 3.1 Flash passes 8. On the "recover from a destructive git operation" category (16 tasks), the gap widens further: Fable 5 Mini passes 9, Haiku 4.5 passes 6, Gemini 3.1 Flash passes 3. Both categories require the agent to read error output, form a theory about what went wrong, and run a corrective sequence of commands. The deficit on Gemini 3.1 Flash is structural; the model's context window is the largest but the reasoning over tool call output is the weakest of the three.

Why Gemini 3.1 Flash Wins Cost Per Resolved Task

Gemini 3.1 Flash at $0.21 per resolved task is roughly half the cost of Fable 5 Mini and less than half the cost of Haiku 4.5 even after accounting for its lower pass rate. The pricing structure is the dominant variable. At $0.35 per million input and $1.40 per million output, Flash is roughly a third of the per token cost of Haiku 4.5 and roughly 40 percent of Fable 5 Mini. The token consumption per task is higher (178K versus 142K and 118K) because the model takes more turns to converge, but the per token discount more than compensates.

For high volume agent workloads where the cost per resolved task is the primary KPI, Gemini 3.1 Flash is the cheapest path even though it has the lowest raw pass rate. A team running 50,000 terminal agent invocations per month with Flash pays roughly $10,500 in API cost; the same workload on Haiku 4.5 costs roughly $23,500 and on Fable 5 Mini roughly $15,500. The gap at scale is large enough that the routing decision is rarely about pass rate in isolation; it is about whether the failure mode on the 9.5 percentage point pass rate gap costs more than the savings.

# Cost per resolved task model
def cost_per_resolved_task(in_tokens, out_tokens, in_price, out_price, pass_rate):
    cost_per_attempt = in_tokens / 1_000_000 * in_price + out_tokens / 1_000_000 * out_price
    return cost_per_attempt / pass_rate

# Roughly 70% input, 30% output split
haiku = cost_per_resolved_task(99_000, 43_000, 1.00, 5.00, 0.473)
fable = cost_per_resolved_task(82_000, 36_000, 0.80, 4.00, 0.516)
gemini = cost_per_resolved_task(124_000, 54_000, 0.35, 1.40, 0.421)
print(haiku, fable, gemini)
# 0.47, 0.31, 0.21

The cost gap also means the routing logic can use Gemini 3.1 Flash as the first attempt and fall back to a stronger model (Sonnet 4.6, Fable 5 Mini, or Opus 4.8) on failure. The blended cost per resolved task on a Flash-first cascade is roughly $0.18 because the cheaper model resolves the easy tasks at low cost and the harder tasks pay the more expensive model price only when needed.

Why Haiku 4.5 Holds the Tool Call Accuracy Lead

Haiku 4.5 wins tool call accuracy at 96.4 percent, a thin margin over Fable 5 Mini (94.8) and a wider one over Gemini 3.1 Flash (92.1). Tool call accuracy is the rate at which the model emits a well formed, executable tool call (valid JSON, correct argument types, parameters within range). A malformed tool call in a terminal agent cascade is expensive because the next turn has to interpret an error message rather than the result the agent was expecting, which compounds the recovery cost on every subsequent turn.

The 4.3 percentage point gap between Haiku 4.5 and Gemini 3.1 Flash on tool call accuracy translates into roughly 7 percentage points of pass rate variance on tasks that require 6 or more tool calls. For agents where each task involves a long sequence of shell commands, the accuracy advantage of Haiku 4.5 partially closes the pass rate gap to Fable 5 Mini and dominates the cost calculation if the harness charges retry overhead to the wall clock budget.

For teams running mission critical agent workloads (production incident response, deployment automation, database operations) where a malformed tool call carries operational risk, Haiku 4.5 is the predictability choice. The cost per resolved task is the highest of the three but the variance is the lowest, and the operational cost of a malformed call in those environments is high enough to justify the price.

Routing Strategies for a Production Agent Fleet

The three model choice is rarely a single model decision in production. The routing patterns that actually run in deployed agent fleets cascade across the cheap tier and into the strong tier when the cheap tier fails. Three patterns are credible today.

The first is a single model deployment: pick one cheap tier model and accept the trade. For most teams this is Haiku 4.5 because the operational predictability dominates and the cost gap to Flash is acceptable. The second is a Flash-first cascade: try Gemini 3.1 Flash, fall back to Sonnet 4.6 or Opus 4.8 on tool call failures or task timeouts. This optimizes for cost on the easy long tail of tasks and pays the strong tier price only when needed. The third is a complexity router: classify the task with a small model (Haiku 4.5 at 1K tokens of classification context) and dispatch to the right tier based on the predicted difficulty. The complexity router is the most operationally complex but extracts the most cost per resolved task value.

# Cascade routing pattern
def run_agent(task):
    result = run_with_model(task, "gemini-3-1-flash", max_turns=10)
    if result.completed:
        return result
    # Fall back on tool call accuracy failure or timeout
    result = run_with_model(task, "fable-5-mini", max_turns=12)
    if result.completed:
        return result
    # Last resort: pay the strong tier
    return run_with_model(task, "claude-sonnet-4-6", max_turns=15)

The cascade pattern is the one most production agent fleets settle on after they have run the cheap tier in isolation long enough to see the failure distribution. The first-shot cost stays low because Flash resolves the majority of tasks; the long tail of harder tasks pays the strong tier cost only when it earns the spend.

When This Applies to Your Stack

If your team is building or scaling an agent fleet that runs shell commands, manages files, or operates on real infrastructure, the cheap tier routing decision is the highest leverage cost lever in the system. The three model choice maps directly to the trade you want to make: pass rate (Fable 5 Mini), cost (Gemini 3.1 Flash), or predictability (Haiku 4.5). We build agent harnesses and AI integrations for engineering teams running production coding workloads and terminal agents at scale: model routing logic, tool call accuracy instrumentation, error recovery patterns, and the cascade design that decides what each agent invocation actually costs. If the cheap tier agent slot is the decision on the table, that decision is the work.

FAQ

Which cheap tier model has the highest Terminal-Bench 2 pass rate as of June 2026? Fable 5 Mini at 51.6 percent. Claude Haiku 4.5 follows at 47.3 percent. Gemini 3.1 Flash is at 42.1 percent. The gap is dominated by error recovery on multi step tasks, not by single turn tool call quality.

Which has the lowest cost per resolved task? Gemini 3.1 Flash at $0.21 per resolved task. Fable 5 Mini at $0.31. Haiku 4.5 at $0.47. The Flash advantage is large enough that a Flash-first cascade with a strong tier fallback is the cheapest pattern overall.

Should I always pick the highest pass rate model? No. For high volume workloads where the cost per resolved task is the dominant KPI, the cheaper model with a cascade fallback is the cheaper path overall. For mission critical workloads where each failure carries operational risk, the highest tool call accuracy model (Haiku 4.5) is often the better choice even at the higher price.

Can I run all three in parallel and pick the best result? Yes, and the pattern is called best of N inference. It triples the cost but lifts the effective pass rate to roughly 67 percent (the union of the three models). For low volume, high stakes workloads (incident response, security analysis, irreversible deployments) the cost is worth the lift. For high volume routine workloads the cascade pattern is the better economic fit.

How does the cheap tier compare to the strong tier on Terminal-Bench 2? Opus 4.8 lands at roughly 78 percent pass rate on Terminal-Bench 2 at roughly $4.20 cost per resolved task. The 26 percentage point pass rate gap is real and the cost gap (roughly 20x) is also real. The right answer is rarely "always use the strong tier"; it is a routing logic that pays the strong tier price only on the tasks where the cheap tier has actually failed.