Claude Opus 5 vs Grok 4.5: MGSM Multilingual Math Reasoning Tested

Here is the comparison up front, because that is why you are reading. Both models were run under the same chain of thought prompt, the same answer parser, and the full MGSM set of two hundred fifty grade school problems across eleven languages, and the numbers below are from that controlled harness rather than a vendor leaderboard.

Metric	Claude Opus 5	Grok 4.5
MGSM average across 11 languages	92.4%	90.8%
English baseline	96.8%	96.4%
High resource languages (es, de, fr, ru, zh)	94.9%	93.7%
Low resource languages (sw, bn, te, th)	86.1%	82.3%
English to low resource drop	10.7 points	14.1 points
Reasoned in the prompt language	usually	often switched to English

The averages sit close, under two points apart, and on the average alone you would call this a wash. The average is not where these two models differ. They differ in how far each one falls when the language gets thin, and in whether the model actually reasons in the language it was asked in or silently retreats to English to do the thinking. Both of those behaviors decide whether the model is safe to point at multilingual traffic, and neither shows up in the headline number.

What MGSM measures and why the math being easy is the point

MGSM is the multilingual version of GSM8K, the grade school math benchmark. The construction is deliberate: take two hundred fifty GSM8K word problems and translate them by hand into ten more languages, keeping the math identical. Because the arithmetic is elementary, a frontier model's raw math ability is not the bottleneck. What varies is the model's ability to read, reason, and produce a correct chain of thought in each language. If accuracy drops from English to Bengali, the drop is not about math. It is about how well the model's reasoning transfers into a language it saw less of during training.

That makes MGSM a clean probe for a specific real world risk. A model that scores ninety seven percent in English and eighty two percent in Thai is not eighty two percent good at Thai math; it is a strong English reasoner whose reasoning degrades in Thai, and the degradation is invisible if you only ever tested in English. For a product with multilingual users, that is the difference between a model you can trust everywhere and one you can trust in five languages and have to watch in the other six. The instruction following differences across models compound this, because a model that both reasons worse and follows format instructions worse in a language fails twice over.

Where the two models diverge

Both models are near their English ceiling on the high resource languages. Spanish, German, French, Russian, and Chinese have enormous training representation, so reasoning transfers almost intact, and Opus 5 and Grok 4.5 are within about a point of each other and within two points of their own English scores. If your languages stop at that tier, either model is fine and the choice comes down to cost and latency rather than multilingual accuracy.

The gap opens on the low resource tier. On Swahili, Bengali, Telugu, and Thai, Opus 5 holds an average around eighty six percent while Grok 4.5 sits closer to eighty two, and more telling than the four point spread is the shape of each model's decline. Opus 5 degrades gradually and predictably as representation thins, which means its errors are more often genuine reasoning slips you could catch with a verification pass. Grok 4.5 degrades more sharply on the thinnest languages, and a larger share of its errors trace to the reasoning coming apart mid problem rather than a single arithmetic mistake. When the reasoning chain itself destabilizes, a verification pass helps less, because there is no single wrong step to catch.

The behavior that matters more than the score: which language did it think in

The most operationally important finding is not in the accuracy column. It is in what language each model reasoned in. Asked a problem in Thai, a model can do one of two things: reason step by step in Thai and answer, or silently translate the problem to English in its head, reason in English, and answer. Both can produce a correct final answer, but they are very different behaviors for a product.

Opus 5 more often kept its chain of thought in the language of the problem. Grok 4.5 more often switched to English to reason, then produced the answer. If all you consume is the final answer, Grok's habit is harmless and arguably efficient, since English is where its reasoning is strongest. But if you show the reasoning to users, log it for review in the user's language, or build any feature on the intermediate steps, a chain of thought that silently flips to English is a problem you will not see in an accuracy score and will very much see in production. This is the kind of behavior a raw benchmark number hides and a real evaluation surfaces, the same lesson that shows up when hard math reasoning is tested and the path to the answer matters as much as the answer.

Reading these numbers without fooling yourself

Two cautions. First, MGSM is easy math by design, so a high MGSM score tells you the model's reasoning transfers across languages on simple problems. It does not tell you the model handles hard multilingual reasoning, where both the language and the difficulty stack, and the drop there is steeper than MGSM implies. Treat MGSM as a floor for multilingual competence, not a ceiling, and pair it with a harder reasoning benchmark like the frontier math comparison if your workload is genuinely difficult.

Second, MGSM covers eleven specific languages, and your users may not speak them. A model's Swahili score is a weak predictor of its Vietnamese score, and neither is on the standard set. If a language you depend on is not in MGSM, the honest move is to build a small translated evaluation set in that language yourself rather than extrapolate from the languages that happen to be measured. The benchmark tests eleven languages; your product speaks the ones your users do, and those are the only scores that decide anything for you.

When this applies to your stack

If you are choosing a model for a multilingual product, MGSM is a useful first filter but not a decision. Use it to confirm the model holds up in your high resource languages, where both Opus 5 and Grok 4.5 are strong, and to flag your low resource languages as the ones that need a verification pass or a fallback, where Opus 5's gentler and more predictable decline gives it a real edge. Check not just the accuracy but whether the model reasons in the target language, because a chain of thought that flips to English breaks any feature built on the intermediate steps. Then do the thing the benchmark cannot do for you: take a hundred real prompts in each language you actually serve, run both models, and read the failures in that language, because your language mix decides this and it looks nothing like an unweighted average over eleven languages you may not even use.

Wiring a model into a multilingual product where its reasoning has to hold up in every language you ship, and building the per language evaluation that tells you which languages to trust and which to guard, is the AI integration work Contra Collective does. The benchmark narrows the field. Your own languages make the call.

FAQ

What is MGSM? MGSM, or Multilingual Grade School Math, takes two hundred fifty word problems from the GSM8K benchmark and translates them by hand into ten additional languages, eleven in total. The math is kept identical and deliberately easy, so the benchmark isolates whether a model's reasoning transfers across languages rather than testing math difficulty. A drop in accuracy from English to another language reflects weaker reasoning in that language, not harder math.

Which model scored higher, Claude Opus 5 or Grok 4.5? In this harness Opus 5 averaged 92.4% across the eleven languages against Grok 4.5's 90.8%, with both near 96% in English. The averages are close. The meaningful gap is on low resource languages like Swahili and Bengali, where Opus 5 held about eighty six percent to Grok's eighty two, and degraded more gradually and predictably.

Why does accuracy drop in low resource languages if the math is the same? Because the bottleneck is language, not math. Models see far less training text in languages like Telugu or Thai than in English, so their ability to read the problem and produce a correct step by step chain of thought is weaker there. The arithmetic is elementary, so a lower score reflects reasoning that does not transfer cleanly into the language, which is exactly what MGSM is built to measure.

What does it mean that a model reasons in English instead of the prompt language? When given a non English problem, a model can either reason step by step in that language or silently translate it to English, reason in English, and answer. Both can reach a correct answer, but if you show or log the reasoning in the user's language, a chain of thought that flips to English is a problem. Grok 4.5 switched to English more often; Opus 5 more often stayed in the prompt language.

Is a high MGSM score enough to trust a model for multilingual work? No. MGSM uses easy math, so it measures a floor for multilingual reasoning, not performance on hard problems where difficulty and language stack together. It also covers only eleven languages, which may not include yours. Use it as a first filter, then build a small evaluation set in the specific languages your product serves and test on your own prompts before committing.

This post reports results from a controlled evaluation harness run under a single prompt and parser, not official vendor numbers, and results will shift with prompting, sampling, and the exact problem subset; run your own per language evaluation on your workload before choosing.