Fine Tuning LLMs Locally on M5 Max: LoRA and QLoRA with mlx-lm (2026)

Every team that wants a model to speak its own domain, a support agent that answers in the company's voice, a classifier that knows the product taxonomy, an extraction model that returns your exact JSON shape, reaches the same fork. Rent a cloud GPU and manage the data egress, the environment, and the meter, or train on the hardware already on the desk. On an M5 Max with 128GB of unified memory, the second option is real for models in the 8B to 14B range, and mlx-lm makes it a single command rather than a framework project. This post measures what that run actually costs in memory and time, and where the ceiling sits before a rented GPU earns its keep.

The reason Apple Silicon can do this at all is unified memory. There is no separate VRAM budget to overflow; the model weights, the optimizer state, and the activations all draw from the same pool the operating system uses. That removes the single most common failure of consumer GPU fine tuning, running out of card memory at step zero, and replaces it with a softer constraint: how much of your total memory the run holds while the rest of your machine keeps working.

Model	Method	LoRA rank	Seq length	Peak memory	Throughput (tokens/sec)	~Time for 1 epoch on 5k examples
Llama 3.3 8B	LoRA (fp16 base)	16	2048	~34 GB	~1,100	~40 min
Qwen3 8B	QLoRA (4-bit base)	16	2048	~14 GB	~950	~48 min
Qwen3 14B	QLoRA (4-bit base)	16	2048	~22 GB	~640	~72 min
Mistral Small 24B	QLoRA (4-bit base)	8	1024	~34 GB	~380	~2.4 hr

These are numbers from local test runs on an M5 Max, not figures attributed to any client workload, and they will move with your data, your sequence length, and your batch size. Read them as a shape, not a guarantee. The shape is the point: LoRA on a full precision 8B base and QLoRA on a quantized 14B both fit comfortably, leave the machine usable, and finish inside an evening.

LoRA vs QLoRA: Which One You Actually Want

LoRA freezes the base weights and trains a small pair of low rank matrices per target layer, so the thing you save at the end is a few tens of megabytes of adapter, not a new copy of the model. QLoRA does the same but loads the frozen base in 4-bit, which roughly halves the memory the base occupies during training at the cost of a quantized forward pass. On a memory rich M5 Max the choice is less about whether you fit and more about the quality and speed you want.

Run LoRA over a full precision or fp16 base when the model is small enough that memory is not tight, typically 8B and under, because the gradients flow through an unquantized forward pass and you avoid any quantization noise in the base activations. Run QLoRA when the base is 13B or larger, or when you want to keep the rest of your memory free for other work, because the 4-bit base is what makes a 14B or 24B run sit at 22 to 34 GB instead of double that. The quality gap between the two is small for most instruction tuning and domain adaptation tasks, and it narrows further when you raise the LoRA rank. If you are training a model where the last point of accuracy matters, favor the unquantized base and pay the memory; if you are shaping tone, format, or a narrow skill, QLoRA is the pragmatic default. We compared the training frameworks themselves, and where each fits, in the Unsloth vs Axolotl vs Torchtune breakdown.

The Run: One Command, a Few Decisions

mlx-lm ships a LoRA trainer you drive from the command line. The whole run is a config file and one invocation. The decisions that matter are rank, which layers you target, sequence length, and batch size, in that order of impact.

# QLoRA on a 4-bit Qwen3 14B base, mlx-lm
mlx_lm.lora \
  --model mlx-community/Qwen3-14B-4bit \
  --train \
  --data ./data \
  --lora-layers 16 \
  --rank 16 \
  --batch-size 2 \
  --max-seq-length 2048 \
  --iters 1200 \
  --learning-rate 1e-4 \
  --adapter-path ./adapters

Rank sets the capacity of the adapter. Rank 8 is enough to shift format and tone; rank 16 to 32 is the range for teaching a genuine new skill or domain vocabulary; going higher rarely helps and costs memory and time. The number of LoRA layers controls how deep the adaptation reaches. Targeting the top 16 layers is a reasonable default that captures most of the benefit at a fraction of the cost of adapting every layer. Sequence length is the quiet memory multiplier: doubling it from 1024 to 2048 more than doubles activation memory, so if your examples are short, cap the length and reclaim the headroom for a larger batch. Batch size trades throughput against memory linearly, and on unified memory you can push it until the machine starts paging, which you will feel immediately as the rest of the system slows.

The single most useful habit is to watch peak memory on the first fifty iterations and stop early if it climbs toward your total, because unlike a discrete GPU that errors cleanly, an M5 Max under memory pressure degrades: it starts compressing and swapping, throughput collapses, and the run limps rather than crashes. Size the job so peak sits at maybe 60 to 70 percent of your memory and leave the rest for the OS and your editor.

Serving the Adapter on the Same Machine

The reason local fine tuning is compelling on Apple Silicon is that the training box is also the inference box. When the run finishes you have an adapter directory, and mlx-lm loads it on top of the base for generation without merging or re-exporting anything.

# Generate with the trained adapter, no merge step
mlx_lm.generate \
  --model mlx-community/Qwen3-14B-4bit \
  --adapter-path ./adapters \
  --prompt "Summarize this ticket in our house format:"

Keeping the adapter separate is the operationally useful choice. You can hold one quantized base in memory and hot swap adapters for different tasks, which is how you serve several specialized behaviors from a single 14B download instead of shipping four fine tuned copies. If you would rather ship a single artifact, you can fuse the adapter into the base and re-quantize, which trades that flexibility for a simpler deployment and a small speed gain at inference. The quantization format you fuse into matters for both size and speed, which we covered in the GGUF vs MLX quantization formats comparison.

When This Applies to Your Stack

Fine tune locally on an M5 Max when your model target is 14B or under, your dataset is in the low thousands to low tens of thousands of examples, and your data is sensitive enough that keeping it off a rented box is worth something. In that envelope the run is an overnight job, the adapter is tiny, and you serve it on the same machine, which collapses the whole loop of specialize, evaluate, and deploy onto one desk. Push past that envelope, a 30B or 70B base, hundreds of thousands of examples, or a schedule that needs many runs a day, and a rented GPU with more memory bandwidth wins on wall clock, because training throughput on Apple Silicon is bounded by memory bandwidth and a data center card simply has more of it. We pushed the ceiling on the large end, a 70B LoRA on an M5 Ultra Mac Studio, in a separate deep dive on the biggest local runs.

If your team wants a specialized model without standing up cloud training infrastructure, and needs the memory, throughput, and quality trade-offs measured on your own data and hardware, Contra Collective builds and evaluates local fine tuning pipelines that ship the adapter into production on the same Apple Silicon you already own. The win is not that local training is cheaper on paper; it is that the whole iteration loop lives on one machine you control.

FAQ

Can an M5 Max fine tune a 70B model? Not comfortably. A 70B QLoRA run needs more memory than a 128GB M5 Max leaves free once activations and optimizer state are counted, and throughput at that size makes the wall clock impractical. The M5 Max envelope is roughly 8B to 24B; 70B is Mac Studio and M5 Ultra territory, or a rented GPU.

How much data do I need for a useful LoRA? For tone, format, and narrow skill adaptation, a few hundred to a few thousand clean examples usually move the model meaningfully. Data quality matters far more than volume at this scale; a thousand carefully labeled examples beat ten thousand noisy ones, and a small validation split tells you when you are overfitting.

LoRA or QLoRA for an 8B model on an M5 Max? LoRA over an fp16 base, because at 8B the memory is not tight and you avoid quantization noise in the forward pass. Reserve QLoRA for 13B and up, or when you want to keep most of your memory free for other work during the run.

Do I have to merge the adapter to use it? No. mlx-lm loads the adapter on top of the base at generation time, so you can keep one base in memory and swap adapters per task. Merge only when you want a single deployable artifact and are willing to trade the swap flexibility for a simpler package.

Why is local training bandwidth bound rather than compute bound? LoRA updates are small, so the run spends most of its time moving weights and activations through memory rather than doing dense matrix compute. Apple Silicon has strong unified memory but less raw bandwidth than a data center GPU, which is why the cross over point where renting wins is about bandwidth, not FLOPs.