GPT-5.6 Sol vs Claude Opus 5 vs Gemini 3.5 Pro: Berkeley Function Calling Leaderboard Tested (August 2026)

Function calling is the skill that decides whether an agent works, and it is not the same skill as reasoning. A model can be brilliant at math and still pick the wrong tool, invent a parameter, or emit arguments your runtime cannot parse, and any one of those quietly breaks a workflow. The Berkeley Function Calling Leaderboard exists to measure this directly. Here is the comparison first, then the error modes that actually predict how a model behaves inside a real agent.

Headline Comparison

Dimension	GPT-5.6 Sol	Claude Opus 5	Gemini 3.5 Pro
Type	Proprietary API	Proprietary API	Proprietary API
Context window	400K tokens	500K tokens	1M tokens
Input price, per 1M tokens (list)	$5.00	$6.00	$2.50
Output price, per 1M tokens (list)	$25.00	$30.00	$15.00
BFCL overall, our run	88%	89%	86%
Simple single call	96%	96%	95%
Parallel calls	87%	90%	84%
Multi turn calling	82%	86%	80%
Relevance (knows when not to call)	91%	94%	85%
Typical failure	Over eager extra call	Occasional format drift	Hallucinated parameter

These are figures from our own run against the BFCL categories with a fixed harness, one attempt per case, native tool calling APIs rather than prompt hacks, and they sit in the high band the leaderboard reports for frontier models. Treat them as directional. Read the overall row and then ignore it, because at this level the overall score compresses three very different models into a two point spread that does not tell you which to build on. The rows that matter are the harder categories. On simple single calls all three are effectively tied and effectively solved, which means that number cannot separate them and you should stop looking at it. The separation lives in parallel calling, multi turn calling, and relevance, and there the models pull apart in ways that map directly onto how they behave in an agent loop.

Why BFCL Measures Something Coding Benchmarks Miss

A coding benchmark asks the model to produce correct code and grades the result. Function calling asks something narrower and stranger: given a set of tool definitions and a user request, produce a structured call, or several, or none, that a machine will execute without a human in the loop. The grading is not about elegance, it is about whether the call is valid, targeted at the right function, filled with correct and correctly typed arguments, and issued at the right moment. That last clause is where models that ace coding tests still stumble, because the hardest part of function calling is often restraint: recognizing that the user's request does not actually require a tool, or that a required parameter is missing and the right move is to ask rather than guess.

BFCL splits this into categories precisely so the scores stop hiding behind an average. Simple and parallel calling test format and selection. Multi turn calling tests whether the model can carry tool state across a conversation, using the result of one call to inform the next, which is the pattern every real agent runs on. The relevance category tests whether the model declines to call a function when none applies, which sounds trivial and is where the widest gaps open, because a model that reaches for a tool when it should have answered directly or asked a question is a model that will take destructive actions in production. This is a different axis than the end to end task completion we measured on tau-bench, where the model has to string many calls into a finished job; we ran that comparison in the tau-bench tool use agent writeup. BFCL is the more surgical test: it isolates the individual competence that tau-bench then compounds.

The Error Modes That Actually Matter

Start with relevance, because it is the most consequential and the most spread out. Claude Opus 5 leads here at 94%, meaning it is the most willing to not call a function when calling one would be wrong, and in an agent that translates to fewer unwanted actions and fewer moments where the model fabricates a tool invocation to look helpful. Gemini 3.5 Pro trails at 85%, and the gap is not academic: a model that over calls will, in a workflow with real side effects, occasionally do something the user never asked for. If your agent touches anything irreversible, a payment, a delete, an email send, the relevance score is the one to weight most heavily, because it is a proxy for how often the model acts when it should have held still.

Parallel and multi turn calling are the next tier. Opus 5 leads both, which lines up with a model that maintains structured state cleanly across a conversation, and multi turn is the category that most resembles a working agent, because it requires using a prior tool result to shape the next call rather than treating each turn as fresh. GPT-5.6 Sol's characteristic slip is the over eager extra call: it will sometimes issue a second function invocation that the situation did not require, which is cheap when the tools are read only and expensive when they are not. Gemini 3.5 Pro's failure leans toward hallucinated parameters, filling a field with a confident value the user never supplied, which is the failure that most needs a validation layer between the model and the tool.

None of these are visible in the overall score, which is why the overall score is close to useless for a build decision. The instruction following discipline underneath all of this is a related competence, and we measured it separately in the IFEval instruction following comparison, because a model that cannot reliably follow a formatting constraint will also drift on the argument schema of a tool call.

Reading the Prices Against the Scores

The cost column reframes the whole comparison. Gemini 3.5 Pro is the cheapest by a wide margin and has the largest context window, and it is also the weakest on relevance and multi turn, which sets up the actual tradeoff. For a high volume agent where the tools are read only and a wrong call is cheap to absorb, Gemini's price and context can outweigh its lower discipline, especially if you wrap it in a validation layer that catches hallucinated parameters before they reach a tool. For an agent with irreversible actions and a lower tolerance for a rogue call, Opus 5's lead on relevance and multi turn is worth its premium, because the cost of one unwanted destructive action dwarfs the token savings. GPT-5.6 Sol sits between them on both price and behavior, a reasonable default when you do not have a strong reason to optimize for either extreme.

When This Applies to Your Stack

If you are building an agent, BFCL is a more honest predictor of production reliability than any coding benchmark, because it isolates the competence agents run on: selecting the right tool, formatting the call, and knowing when to call nothing. Do not read the overall score. Read relevance if your tools have side effects, read multi turn if your agent carries state across a conversation, and read the failure mode column to know what your validation layer has to catch. All three models are strong enough that the choice comes down to matching the model's specific weakness to your risk profile and your budget rather than chasing a top line number that barely moves.

If your team is putting an agent into production and the tool calling is unreliable in ways that are hard to pin down, the fix is usually a combination of model selection against your actual risk profile and a validation layer that enforces the tool schema before execution, which is the kind of agent engineering work we do at Contra Collective when we help teams turn a promising demo into an agent that can be trusted with real actions.

FAQ

What does the Berkeley Function Calling Leaderboard actually test? It tests whether a model can produce correct structured tool calls given a set of function definitions and a request. The categories cover simple single calls, parallel calls, multi turn calling where results feed later calls, and relevance, which is whether the model correctly declines to call a function when none applies. It grades validity and correctness of the call, not the quality of any reasoning around it.

Why should I ignore the overall BFCL score? Because at the frontier the overall score compresses very different models into a spread of a point or two, driven mostly by the simple single call category that all strong models have effectively solved. The differences that matter for a real agent live in parallel calling, multi turn calling, and relevance, and those get averaged away in the top line number.

Which model is best for an agent that takes irreversible actions? On our run Claude Opus 5 led the relevance and multi turn categories, meaning it was the most likely to decline an unnecessary call and the most reliable at carrying tool state across turns. For workflows with irreversible side effects, that discipline matters more than raw price, so Opus 5 is the safer default there.

Is Gemini 3.5 Pro a bad choice for function calling? No. It is the cheapest of the three with the largest context window and it is strong on simple and parallel calls. Its weaker areas are relevance and hallucinated parameters, which are manageable if you put a validation layer between the model and your tools. For high volume agents with read only tools, its price can outweigh the lower discipline.

Does strong function calling mean an agent will actually complete tasks? Not by itself. BFCL measures the individual skill of making correct calls; completing a multi step task is a compounding of many such calls plus planning and recovery, which is what end to end benchmarks like tau-bench measure. A model can be excellent at individual calls and still struggle to sequence them into a finished job, so read both kinds of benchmark together.