Posts tagged with #m5-ultra from the Contra Collective team.
M5 Max, M5 Ultra, and M4 Max measured for tokens per watt and energy cost per million tokens under MLX. Wall power draw, sustained throughput, quantization, and the electricity line item that decides which Mac wins the local inference unit economics in 2026.
Qwen3 235B MoE and Llama 3.3 70B Dense compared on a 512GB M5 Ultra Mac Studio under mlx-lm. Tokens per second, peak memory, sustained throughput, and cost per million tokens for a local production inference workload in 2026.
Fine-tuning Llama 3.3 70B with LoRA on a 256GB M5 Ultra Mac Studio. Throughput per step, unified memory budget, convergence behavior, and the MLX vs llama.cpp choice that decides whether local fine-tuning fits your workflow.
M5 Ultra batched inference under MLX, llama.cpp, and vLLM Metal. Throughput per user at batch 1, 4, 8, 16, and 32 on a Mac Studio for production serving.
Tokens per second is half the story. The other half is what those tokens cost in electricity, hardware amortization, and rack space. Most Apple Silicon local LLM benchmarks publish a decode number on Llama 3.1 8B and stop. That makes for a clean headline and a useless TCO model. We measured wall power on M4 Pro, M5 Pro, M5 Max, and M5 Ultra under sustained inference for Llama 3.3 8B and Llama 3.3 70B, then converted everything to Joules per generated token and dollars per million tokens. The numbers settle the "should we self host on Apple Silicon" conversation in a way that throughput alone cannot.
MLX 0.22 landed in late May 2026 with a refactored flash attention implementation that closes most of the gap to what llama.cpp pulls out of Metal Flash Attention 2. For engineering teams running local LLMs on M5 Ultra hardware, this is the version that changes the runtime calculus.