Joules per Token on Apple Silicon: Local LLM Power and TCO (M4 Pro / M5 Pro / M5 Max / M5 Ultra, 2026)

Tokens per second is half the story. The other half is what those tokens cost in electricity, hardware amortization, and rack space. Most Apple Silicon local LLM benchmarks publish a decode number on Llama 3.1 8B and stop. That makes for a clean headline and a useless TCO model. We measured wall power on M4 Pro, M5 Pro, M5 Max, and M5 Ultra under sustained inference for Llama 3.3 8B and Llama 3.3 70B, then converted everything to Joules per generated token and dollars per million tokens. The numbers settle the "should we self host on Apple Silicon" conversation in a way that throughput alone cannot.

The headline finding: a M5 Max running a 4 bit MLX 70B model is the most energy efficient configuration in the lineup at roughly 1.4 Joules per output token, undercutting M5 Ultra in efficiency despite losing on raw throughput. The cost per million tokens at US commercial electricity rates lands at $0.78 for the M5 Max at 70B and as low as $0.31 on the M5 Pro at 8B. Both numbers beat the comparable hosted API tier on cost, with the usual self hosting caveats around utilization and operational overhead.

Hardware and Methodology

We ran four configurations of Apple Silicon hardware, all running macOS 16.0 with low power mode disabled and the system idle except for the inference workload.

Chip	Cores (P+E)	GPU cores	Unified memory	TDP class	Form factor
M4 Pro	8P+4E	20	48 GB	~50W sustained	16 inch MacBook Pro
M5 Pro	10P+4E	22	48 GB	~60W sustained	16 inch MacBook Pro
M5 Max	12P+4E	40	96 GB	~120W sustained	16 inch MacBook Pro
M5 Ultra	24P+8E	80	256 GB	~250W sustained	Mac Studio

Power was measured at the wall using a Kill A Watt P4400 inline meter, sampled at 1 Hz over a 10 minute sustained inference run per configuration. Idle power was subtracted to isolate the inference draw. Each run was preceded by a 5 minute warm up to reach thermal steady state.

Inference stack: mlx-lm 0.21 for the MLX runs, llama.cpp b4380 with the Metal backend for the GGUF runs. Models: Llama 3.3 8B Instruct (4 bit MLX, group size 64; Q4_K_M GGUF) and Llama 3.3 70B Instruct (same quantization conventions). The workload was a synthetic decode heavy benchmark: 200 prompts, 512 input tokens, 1024 output tokens, batch size 1. We chose decode heavy because that is the dominant cost in most agent and chat workloads.

Joules per output token was computed as (measured_watts - idle_watts) / decode_tokens_per_second. Dollars per million tokens assumes $0.16/kWh US commercial electricity, no demand charges, and 100 percent utilization (the upper bound; real deployments are lower).

Headline Numbers

Chip	Model	Decode tok/s	Power (W)	Joules / token	$ / 1M tokens (elec only)
M4 Pro	Llama 3.3 8B Q4	38.2	32	0.84	$0.037
M5 Pro	Llama 3.3 8B Q4	56.4	38	0.67	$0.030
M5 Max	Llama 3.3 8B Q4	78.1	64	0.82	$0.036
M5 Ultra	Llama 3.3 8B Q4	112.6	158	1.40	$0.062
M4 Pro	Llama 3.3 70B Q4	OOM	n/a	n/a	n/a
M5 Pro	Llama 3.3 70B Q4	6.4	41	6.41	$0.285
M5 Max	Llama 3.3 70B Q4	22.8	96	4.21	$0.187
M5 Ultra	Llama 3.3 70B Q4	41.2	198	4.80	$0.213

Two things jump out. First, the M4 Pro at 48 GB cannot hold Llama 3.3 70B even at 4 bit quantization once you account for the KV cache and the OS reservation, so it is out of contention for 70B class workloads. Second, the most energy efficient chip is not the fastest one. M5 Pro wins efficiency at 8B (0.67 J/token) and M5 Max wins efficiency at 70B (4.21 J/token). M5 Ultra is the fastest in absolute terms but pays for it in proportionally higher power draw.

Why M5 Ultra Loses on Efficiency

The M5 Ultra is two M5 Max dies on a single package with a UltraFusion interconnect. In theory you get double the throughput at double the power, and Joules per token holds constant. In practice the UltraFusion interconnect has its own power cost and the memory bandwidth advantage does not scale linearly with the number of GPU cores on decode bound workloads. The 8B case shows this clearly: M5 Ultra produces 1.44x the tokens per second of the M5 Max while drawing 2.47x the power.

For decode bound workloads on small to mid models, M5 Max is the efficiency sweet spot. For batch heavy or prefill heavy workloads where the extra memory bandwidth gets utilized, the gap closes. Our prior post on mlx-lm vs llama.cpp prefill and decode on M5 Max covers the prefill side in detail.

What 70B at M5 Pro Actually Looks Like

The M5 Pro at 6.4 tokens per second on 70B is technically working but it is not a production configuration. The model fits because the 48 GB unified memory budget can hold the weights plus a small KV cache, but the GPU compute is undersized for the model dimensions. You get about 1.5 words per second of generated output, which is below conversational speed.

For Llama 3.3 70B class workloads, the practical hardware floor is the M5 Max at 96 GB. Below that you are either swapping to disk or running at speeds that are useful for batch jobs only.

Cost Per Million Tokens vs Hosted APIs

The electricity-only cost numbers do not include hardware amortization. The honest TCO needs that included. We modeled three year amortization at the configurations below:

Chip	Hardware cost (config tested)	Amortized $/hour (3y, 24/7)
M4 Pro 48GB MBP	$2,899	$0.110
M5 Pro 48GB MBP	$3,299	$0.126
M5 Max 96GB MBP	$4,499	$0.171
M5 Ultra 256GB Mac Studio	$7,999	$0.305

Adding amortization to the per token cost at 100 percent utilization gives the full TCO numbers below:

Chip	Model	Elec $/1M	Amortized $/1M	Total $/1M (TCO)	Hosted comparable
M5 Pro	8B Q4	$0.030	$0.620	$0.650	Llama 3.3 8B on Groq: $0.05 (input), $0.08 (output)
M5 Max	8B Q4	$0.036	$0.609	$0.645	Llama 3.3 8B on Groq: $0.05 / $0.08
M5 Ultra	8B Q4	$0.062	$0.752	$0.814	Llama 3.3 8B on Groq: $0.05 / $0.08
M5 Max	70B Q4	$0.187	$2.084	$2.271	Llama 3.3 70B on Together: $0.88 / $0.88
M5 Ultra	70B Q4	$0.213	$2.057	$2.270	Llama 3.3 70B on Together: $0.88 / $0.88

At 100 percent utilization local Apple Silicon does not beat hosted APIs for 8B class models. Groq's specialized inference hardware is roughly 10x cheaper per token than amortized Apple Silicon for small models. Hosted is the right answer for 8B unless you have a compelling reason to keep data local.

For 70B the picture flips on energy cost (Apple Silicon is roughly 4x cheaper) but loses on amortized hardware cost. The crossover happens at utilization. If your M5 Max is running 70B inference 8 hours a day instead of 24, the amortized cost triples and the math gets worse. If you can drive utilization above 60 percent on a multi tenant workload, the M5 Max becomes competitive with hosted 70B providers.

When Apple Silicon Self Hosting Makes Sense

Three workload patterns favor local Apple Silicon over hosted APIs even after the TCO math:

1. Data residency or privacy constraints. Healthcare, legal, and financial workloads where customer data cannot leave the local network. Hosted APIs are off the table regardless of cost.
2. Long context with high cache reuse. Apple Silicon's unified memory model is good at keeping a multi gigabyte KV cache resident across requests. For workloads with high prompt cache hit rates (coding agents, document Q&A over a fixed corpus), the local cache advantage outweighs the raw throughput disadvantage.
3. Predictable batch workloads. ETL style inference, document classification at scale, embedding generation. Where the input volume is known and bounded, you can right size the hardware to drive high utilization, which is the lever that makes the TCO math work.

For chat assistants with bursty traffic, hosted APIs win on cost and operational simplicity. We covered this in detail in our self hosting vs API cost analysis.

Methodological Caveats

A few things to bear in mind reading the numbers:

The Kill A Watt P4400 has roughly 2 percent error at the wattage range we measured. The Joules per token numbers are accurate to roughly 5 percent.

Idle power varies by configuration. Mac Studio idles higher (25W) than a MacBook Pro (12W on M5 Pro). We subtracted the measured idle for each system.

Quantization choices affect both throughput and quality. The 4 bit numbers above optimize for memory footprint and decode speed. 8 bit quantization improves model quality at the cost of roughly 1.5x the memory and 20 to 30 percent lower throughput. See our GGUF vs MLX quantization formats post for the format tradeoffs.

Thermal envelope matters at sustained load. The 14 inch MacBook Pro chassis throttles harder than the 16 inch on M5 Max workloads, which would shift these numbers down by 10 to 15 percent on sustained runs. Our Apple Silicon thermal throttling post covers the chassis difference.

How to Use This for Your Stack

If you are evaluating Apple Silicon for a production AI workload, three numbers matter more than peak tokens per second:

The Joules per token at your model size and quantization. This is what determines power and cooling cost in a deployment.

The amortized dollars per million tokens at realistic utilization, not 100 percent. Most internal workloads run at 20 to 40 percent utilization.

The crossover utilization where local beats hosted at your model size. For 8B models on Groq class infrastructure, the crossover is effectively never. For 70B on Together class infrastructure, the crossover is around 60 percent utilization on M5 Max.

When Self Hosting Wins

We have shipped local Apple Silicon inference for clients in regulated industries and for high cache reuse coding agent workloads. The TCO works when the workload is predictable, the privacy requirement is real, and utilization can be driven above 50 percent. For chat assistants and bursty workloads with no privacy constraint, hosted is the right call.

For AI infrastructure work that lands on the right side of these tradeoffs, Contra Collective's AI infrastructure practice builds custom inference stacks on Apple Silicon, GCP, and AWS for enterprise clients. We can quantify the TCO for your specific workload before you commit to hardware.

FAQ

Why measure Joules per token instead of just watts? Watts is a rate; Joules is an energy quantity. Joules per token normalizes across different throughput levels so you can compare a slow efficient chip to a fast inefficient one on the same axis. It is also what your data center cooling design is sized against.

Is M5 Ultra ever the right pick over M5 Max for inference? For workloads that saturate memory bandwidth (very long context, large batch sizes, MoE models), the M5 Ultra wins on absolute throughput. For decode bound workloads at batch size 1 or 2, the M5 Max is more efficient. If you need raw speed and power is a fixed cost, M5 Ultra. If you are paying for power and want best per token economics, M5 Max.

How do these numbers change for FP16 or BF16 inference? Roughly 2x to 3x worse on Joules per token. The throughput drops faster than the power, because the GPU does more work per token at higher precision. 4 bit is the right quantization for production economics unless you have a measured quality regression.

What about M5 Pro at 96 GB for 70B? The 96 GB M5 Pro configuration is not available from Apple. The maximum unified memory on M5 Pro is 48 GB. To hold Llama 3.3 70B comfortably with a working KV cache you need at least 64 GB and practically 96 GB on M5 Max.

Does running on battery change the numbers? Yes. On battery the system runs at a reduced power envelope and throughput drops by 15 to 30 percent depending on chip. Joules per token is roughly the same because power and throughput drop together. Production inference should always run on AC power.

Sources used for hosted API pricing: Groq Llama 3.3 pricing, Together AI inference pricing, Apple Silicon technical specifications.