Qwen3.8 27B on Apple Silicon: MLX Throughput, Quantization, and Where It Fits (2026)

Alibaba's Qwen team released Qwen3.8 27B in August 2026, and it is the strongest case yet for running a capable multimodal model entirely on a Mac. It is a dense 27.78 billion parameter checkpoint under Apache 2.0, with a native 262,144 token context, a vision encoder for text, image, and video input, and open weights on Hugging Face. It posts 90.3 on LiveCodeBench v6 and 61.7 on SWE-bench Pro, and it took the number one open weight spot on Arena.ai's Image-to-WebDev leaderboard within two weeks of launch. For local inference the numbers that matter are simpler than the leaderboard: does it fit in your unified memory, and how fast will it decode.

Attribute	Qwen3.8 27B
Vendor	Alibaba Qwen
Released	August 2026
Parameters	27.78 billion (dense)
License	Apache 2.0
Context window	262,144 tokens
Modality	text, image, video
LiveCodeBench v6	90.3
SWE-bench Pro	61.7
Terminal-Bench 2.1	73.0

Why a dense 27B is the sweet spot on a Mac

The reason this model is interesting on Apple Silicon is that it is dense, not a mixture of experts, and it sits near 30 billion parameters. A dense model reads all of its weights on every token, so its memory footprint is predictable and its throughput is bounded by memory bandwidth in a way you can actually calculate. Apple Silicon's unified memory means the GPU can address the whole model without a copy across a PCIe bus, which is exactly the advantage that makes a 27B model practical on a laptop or a Mac Studio when the same model on a consumer discrete GPU would spill out of VRAM.

The Apache 2.0 license is the other half of why this is worth the setup. Unlike models under bespoke community licenses that restrict commercial use above a revenue threshold, Apache 2.0 lets you build on Qwen3.8 27B without a lawyer in the loop. For an agency shipping client work or a product embedding a local model, that matters as much as the benchmark scores.

Sizing it for your unified memory

Start with the weights. At bf16 the model is roughly 55 GB, which rules out most machines. The practical formats are 4-bit and 8-bit, and the math is easy: bytes of weights is approximately parameter count times bits per weight divided by 8. At 4-bit that is about 27.78 billion times 0.5 bytes, so roughly 14 GB of weights. At 8-bit it is roughly 28 GB. Add framework overhead and you want headroom above those numbers.

The part people forget is the KV cache. A 262K context is enormous, and the KV cache grows linearly with the tokens you actually hold in context. You will rarely run the full 262K, but even a few tens of thousands of tokens adds gigabytes on top of the weights. A useful rule: budget for the weights plus your realistic working context plus a few gigabytes of overhead, then compare to your machine's memory.

Machine memory	4-bit Qwen3.8 27B	8-bit Qwen3.8 27B
16 GB	too tight for real context	no
32 GB	comfortable with moderate context	tight
48 GB	comfortable, long context	comfortable
64 GB and up	full context headroom	full context headroom

If your machine is on the edge, drop to a lower bit quantization before you give up. We compared the newer low bit formats and what they cost in quality in nvfp4 quantization on Apple Silicon and the M5 Max, and the tradeoffs there apply directly to fitting this model.

Which quantization to run

MLX supports 3-bit through 8-bit quantization plus mixed precision modes, including the mxfp8 and nvfp4 formats. For a 27B model the practical choice is between 4-bit and 8-bit. Four bit roughly halves the memory of 8-bit and roughly doubles the decode speed because there are fewer bytes to read per token, at some cost in quality that is usually small for a model this size on most tasks. Eight bit keeps more of the model's precision and is worth it if you have the memory and your work is precision sensitive, such as code generation where a subtly wrong token cascades.

The format choice is not only about memory. It is also about which kernels are fast on your chip. If you want the reasoning behind picking one quantization format over another rather than defaulting to whatever ships first, our GGUF versus MLX quantization comparison lays out how the formats differ in both size and speed on Apple Silicon.

Loading it under MLX

MLX is the right runtime here because it is purpose built for Apple Silicon's unified memory and, on M5, it uses the Neural Accelerators that llama.cpp's Metal backend does not touch. On the same model and quantization, MLX is consistently faster than llama.cpp on Apple Silicon, and the gap widens on M5. Install mlx-lm and load a prequantized build through the Python API:

from mlx_lm import load, generate

model, tokenizer = load("mlx-community/Qwen3.8-27B-4bit")
text = generate(
    model,
    tokenizer,
    prompt="Refactor this function for readability",
    max_tokens=512,
)
print(text)

To serve it behind an OpenAI compatible endpoint for a local agent or app, mlx-lm ships a server module you can launch with the model repo as its argument and a port of your choosing. Point your existing OpenAI client at that local port and your app talks to the model with no network hop.

Note that mlx-lm ships a release roughly every three to four weeks, so pin the version in your environment and test upgrades rather than tracking the tip, because kernel changes can shift both speed and memory between releases.

What throughput to actually expect

Do not trust a tokens per second number you saw for a different chip. Decode on Apple Silicon is memory bandwidth bound for a dense model, so you can estimate the ceiling yourself. Divide your chip's memory bandwidth by the bytes of weights read per token. For a 4-bit build reading roughly 14 GB per token, a chip with 400 GB/s of bandwidth has a theoretical ceiling near 28 tokens per second, and real world decode lands below that because of attention overhead and the KV cache. A chip with 800 GB/s roughly doubles that ceiling. This is why memory bandwidth, not core count, is the spec that predicts local LLM speed, and why the higher tier chips pull ahead. We measured this relationship across the lineup in M5 Ultra local AI inference on MLX.

The practical takeaway is that Qwen3.8 27B at 4-bit is comfortably interactive on a well specced Mac for single user coding, chat, and agent work, and it holds a long enough context to do real repository level tasks. It is not a throughput monster for high concurrency serving; that is a job for a datacenter GPU stack. On a Mac it is a serious single user or small team model that happens to be free to run.

FAQ

Will Qwen3.8 27B run on a 16 GB Mac? The weights fit at 4-bit, near 14 GB, but that leaves almost nothing for the KV cache and overhead, so real context length will be very limited. Plan on 32 GB or more for comfortable use.

Is MLX faster than llama.cpp for this model? On the same quantization, yes on Apple Silicon, and the gap is largest on M5 because MLX uses the Neural Accelerators that llama.cpp's Metal backend does not. If you are on Apple Silicon and running this model, MLX is the default.

Can I use it commercially? Yes, Qwen3.8 27B is Apache 2.0, which permits commercial use without a revenue threshold or special license. Still read the license text yourself before shipping.

How much does the 262K context cost in memory? The KV cache grows with the tokens you hold in context, adding gigabytes on top of the weights at long context. You rarely need the full 262K, so budget for your realistic working window rather than the maximum.

When this applies to your stack

A local model earns its place when data cannot leave your environment, when per token API costs at your volume exceed the cost of running your own, or when you want a capable coding and multimodal model with no network dependency. Qwen3.8 27B on Apple Silicon hits all three for single user and small team work, and Apache 2.0 keeps it clean for commercial use. Contra Collective helps teams decide where local inference beats an API and builds the serving, quantization, and routing setup to run it in production. If you are weighing a local model against a hosted one, we can size it for your hardware and your actual workload.

External references: the Qwen3.8 27B model card and Hugging Face repo, the Qwen3.8 27B benchmark analysis, and the Ollama MLX backend announcement on why runtimes are moving to MLX on Apple Silicon.