Ollama 0.19 MLX Backend on Apple Silicon: What the Switch Buys

For most of its life Ollama ran Apple Silicon inference through the llama.cpp Metal backend, which was portable and correct but not written for Apple's memory architecture. Ollama 0.19, released March 30, 2026, changed that: it switched the default Apple Silicon path to MLX, Apple's own array framework, and the headline number is a decode throughput jump from 58 to 112 tokens per second on the reference test. That is roughly a doubling of the speed at which the model streams tokens back to you, which is the part of a local workload you actually feel. The switch is worth understanding in detail, because the gain is real but conditional, and knowing the conditions is the difference between expecting a doubling everywhere and being disappointed on the models where it does not apply.

Why a backend switch moves the number at all

The reason MLX beats a ported Metal backend is architectural. Apple Silicon uses unified memory: the CPU and GPU share one pool, so there is no separate device memory to copy weights into. A framework built for that model, as MLX is, avoids the copies and layout assumptions that a portable backend carries from its origins on discrete GPUs. Apple has been pushing this since WWDC 2025, where it established MLX as the preferred framework for LLM inference on Apple Silicon, and it reinforced the point in January 2026 with a research paper on the dedicated Neural Accelerators built into each GPU core of the M5 chip running MLX. Ollama switching its default is the downstream consequence of that direction: the framework that matches the hardware wins on the hardware.

The gain concentrates in decode. Decode is the token by token generation phase, and it is memory bandwidth bound and latency sensitive, which is exactly where a framework tuned for unified memory pays off. That is why the reference test shows decode nearly doubling. It is also why the story is more complicated than one number, because prefill behaves differently and large models behave differently.

Where the gain shows up, and where it disappears

The decode improvement is largest on smaller models and shrinks as models grow. MLX leads by roughly 20 to 87 percent for models under 14B parameters. Above about 27B, MLX and llama.cpp converge, because at that size memory bandwidth becomes the binding constraint and both backends are limited by how fast the hardware can move weights, not by how cleverly the software is written. This crossover is the single most important thing to internalize about the switch. If you run a 7B or 8B model interactively, the backend change is a large, felt improvement. If you run a 32B or 70B model, the backend matters much less, and your bottleneck is the memory bandwidth of your specific Mac, which is a hardware fact you cannot code around. The deeper mechanics of that convergence are worked through in our comparison of the llama.cpp Metal backend and MLX on Apple Silicon, and the broader field including vLLM is covered in llama.cpp, MLX, Ollama, and vLLM on Apple Silicon.

Prefill, the phase where the model ingests your prompt, follows a different curve. Prefill is compute bound rather than bandwidth bound, so the backend advantage there depends more on how well each framework saturates the GPU on large matrix work. For long context prompts, prefill can dominate time to first token, and a decode win does not automatically buy you a prefill win. If your workload is long prompts and short answers, measure prefill separately, because the 58 to 112 headline is a decode number and does not describe your latency.

Verify the backend, then measure it

Do not assume the switch is active. Confirm the version and the backend, then measure on your own model and hardware. Start by checking the version through the local API, which returns the running server version.

curl http://localhost:11434/api/version

If you are on 0.19 or later, the MLX path is the default on Apple Silicon. Pull a small model to test the decode sensitive case and a larger one to test the convergence case.

ollama pull llama3.1:8b
ollama pull qwen2.5:32b

Then measure decode throughput from the generation endpoint, which returns timing fields you can turn into tokens per second for both prefill and decode so you can separate the two phases.

curl http://localhost:11434/api/generate -d '{
  "model": "llama3.1:8b",
  "prompt": "Write a 200 word summary of unified memory.",
  "stream": false
}'

The response returns eval_count and eval_duration for decode, and prompt_eval_count and prompt_eval_duration for prefill. Decode tokens per second is eval_count divided by eval_duration in seconds, which isolates the number the MLX switch improves. Run the same prompt against the 32B model and compare. You should see the small model show a large MLX era decode rate and the large model show a rate that is closer to what llama.cpp produced, because at 32B you are bandwidth bound. Record the numbers for your machine rather than quoting the reference test, because memory bandwidth differs across the M4 and M5 families and your ceiling is set by your chip. The interactive experience difference between runtimes at this level is also covered in LM Studio versus Ollama for local inference, which is useful if you are choosing a runtime and not only a backend.

The quantization and context caveats

Two variables move the numbers as much as the backend does. The first is quantization format. The weights you run, whether a 4 bit or 8 bit quantization, change both memory footprint and speed, and the format that is fastest under MLX is not always the one you already have cached from a llama.cpp workflow. When you benchmark, hold the quantization constant across backends or you are measuring two things at once. The second is context length. A long context inflates the KV cache, which consumes unified memory that would otherwise hold weights or serve throughput, and it shifts more of your total time into prefill. A decode benchmark run at a 200 token prompt tells you nothing about how the same model behaves at 32K context. Benchmark at the context length you actually use.

The batch size point is smaller for single user local inference but worth noting: Ollama's local path is optimized for one user at a time, so the decode gain is a single stream number. If you are serving concurrent requests, that is a different workload with a different bottleneck, and a single stream decode doubling does not translate directly into doubled aggregate throughput under concurrency.

FAQ

Do I have to configure anything to get the MLX backend? On Apple Silicon with Ollama 0.19 or later, MLX is the default path, so upgrading is usually enough. Verify the running version through the local API version endpoint, and measure decode from the generation endpoint timing fields to confirm you are seeing the improvement.

Why did my large model not get faster? Above roughly 27B parameters, MLX and llama.cpp converge because memory bandwidth becomes the binding constraint. The backend switch mostly helps models under 14B, where decode is not yet bandwidth capped.

Is the 58 to 112 number what I will see? That is the reference test on specific hardware and a specific model. Your number depends on your chip's memory bandwidth, the model size, the quantization, and the context length. Measure on your own machine.

Does this help prefill or only decode? The headline gain is decode. Prefill is compute bound and follows a different curve, and for long prompts it can dominate time to first token, so measure it separately.

When this applies to your stack

If your local workload is interactive use of models under 14B, upgrading to Ollama 0.19 is close to a free performance win, because decode is where you feel latency and that is where MLX pays off. If you run large models, temper expectations: your ceiling is the memory bandwidth of your Mac, and no backend rewrites that. Either way, verify the version, hold quantization and context constant when you benchmark, and record the numbers for your hardware rather than a headline.

If you are building local inference into a product and need to size the hardware, the model, and the runtime against a real latency budget, that is the AI infrastructure work we do at Contra Collective. The right answer usually comes from measuring your workload, not from a leaderboard.