Unsloth vs Axolotl vs torchtune: Fine-Tuning LLMs on Local Hardware in 2026

The local inference renaissance of the past two years has created a natural next question: if you can run a capable model on your own hardware, can you also train one on your own data? The answer in 2026 is yes, with meaningful caveats, and the tooling has matured enough that the caveats are mostly about hardware constraints rather than software limitations.

Fine-tuning a large language model from scratch requires compute that no local workstation has. But LoRA (Low-Rank Adaptation) and QLoRA (quantized LoRA) change the math dramatically. Instead of updating all model weights, you train a small set of adapter weights that modify the model's behavior at inference time. A 7B parameter model that requires 80GB of GPU memory for full fine-tuning can be QLoRA-tuned on a consumer GPU with 16GB of VRAM, or on an Apple Silicon Mac with 32GB of unified memory.

The three tools that define the current landscape for this kind of accessible fine-tuning are Unsloth, Axolotl, and torchtune.

INTERNAL LINK: local LLM inference options → llama.cpp vs MLX vs Ollama for Apple Silicon

Why the Tool Choice Matters

Fine-tuning, even with LoRA, is computationally expensive relative to inference. A training run that takes 8 hours on a well-optimized setup can take 20 hours on a naive one. Memory fragmentation can cause OOM crashes mid-run that waste hours of compute. The quality of the training loop implementation (gradient accumulation, learning rate scheduling, checkpointing) directly affects how well the adapter generalizes.

Picking the right tool is not about which one has the longest feature list. It is about which one runs reliably on your hardware, fails informatively when something goes wrong, and produces adapters that actually improve the model for your use case.

Unsloth: Speed as a First Principle

Unsloth was built around a single obsession: make LoRA fine-tuning as fast as possible. The library manually rewrites the GPU kernels for the most expensive operations in the transformer training loop: attention, RoPE embeddings, and the cross-entropy loss. The result is fine-tuning that runs 2x to 4x faster than a naive HuggingFace PEFT implementation on the same hardware, with lower peak memory usage.

The numbers are not marginal. On an RTX 4090 fine-tuning a Llama 3.1 8B model with QLoRA, Unsloth achieves roughly 4,200 tokens per second during training compared to approximately 1,600 with vanilla PEFT. The memory savings are also substantial: the same run that uses 12GB of VRAM under PEFT uses roughly 6GB under Unsloth, which means you can fine-tune larger models on smaller GPUs or run larger batch sizes on the same hardware.

Where Unsloth wins:

Any fine-tuning task where training time is the constraint. If you are iterating on dataset quality, adapter hyperparameters, or prompt templates, the 2x to 4x speed improvement translates directly into the number of experiments you can run per day. For teams doing serious domain adaptation work, Unsloth makes the iteration loop fast enough to run multiple experiments overnight.

Unsloth also ships an excellent Colab notebook ecosystem. If you are fine-tuning on cloud GPUs (Google Colab, Lambda Labs, RunPod), the notebooks provide a validated starting point for dozens of model architectures with the correct hyperparameter ranges for QLoRA fine-tuning.

Apple Silicon: Unsloth does not support Apple Silicon. The kernel optimizations are CUDA-specific. On an M-series Mac, you cannot use Unsloth. For Apple Silicon fine-tuning, your options are MLX's native fine-tuning support (which is solid for LoRA) or falling back to a standard HuggingFace PEFT setup via the MPS backend. Neither matches Unsloth's CUDA throughput, but MLX on Apple Silicon is competitive with HuggingFace PEFT on a mid-range NVIDIA GPU.

Unsloth's limits:

Model support is the main constraint. Unsloth maintains hand-optimized kernels for specific architectures: Llama variants, Mistral, Phi, Gemma, Qwen. New models are not supported on day one. When a new architecture drops, you may wait weeks for Unsloth support. Teams tracking the frontier closely will hit this gap.

The library also abstracts away some training loop details that experienced practitioners want to control. Advanced users may find the configuration surface less expressive than Axolotl for edge cases.

Axolotl: The Configurable Workhorse

Axolotl is a training framework built on top of HuggingFace's ecosystem: transformers, PEFT, TRL, and datasets. Where Unsloth optimizes for speed through kernel rewrites, Axolotl optimizes for configurability through a comprehensive YAML configuration system.

A single Axolotl config file can specify the base model, quantization settings, LoRA rank and alpha, dataset format and preprocessing, learning rate schedule, gradient accumulation steps, evaluation frequency, checkpointing strategy, and dozens of other parameters. Teams that need fine-grained control over the training setup, whether for research reproducibility or production-quality adapters, will find Axolotl's configuration depth valuable.

Where Axolotl wins:

Complex dataset pipelines benefit most from Axolotl. The framework supports a wide variety of dataset formats out of the box (alpaca, sharegpt, completion, instruction, chat, code) with configurable preprocessing. Teams working with proprietary data in non-standard formats can write custom dataset processors that plug into the Axolotl pipeline without modifying framework internals.

Multi-GPU training is a strength. Axolotl has solid support for DeepSpeed ZeRO stages 1, 2, and 3, which enables fine-tuning models too large to fit on a single GPU by partitioning optimizer state and gradients across multiple GPUs. For teams with access to multi-GPU cloud instances, Axolotl scales cleanly.

Axolotl also supports a wider range of model architectures than Unsloth, because it relies on HuggingFace's model support rather than hand-optimized kernels. New model architectures are typically available in Axolotl within days of their HuggingFace release.

Axolotl's limits:

Speed. Without Unsloth's kernel optimizations, Axolotl training runs are slower. For simple fine-tuning tasks on well-supported architectures, Unsloth will finish the same job faster. The YAML configuration system is expressive but also verbose: a complete Axolotl config for a production training run is often 80 to 120 lines, and debugging misconfiguration requires understanding which settings interact with which others.

INTERNAL LINK: open source model selection for fine-tuning → choosing base models for domain adaptation

torchtune: PyTorch's Official Answer

torchtune is PyTorch's first-party fine-tuning library, released by Meta in 2024 and now a core part of the PyTorch ecosystem. Where Unsloth wins on speed and Axolotl wins on configurability, torchtune wins on composability and correctness.

The library is built around "recipes": Python scripts that implement specific training workflows. A LoRA fine-tuning recipe for Llama 3.1 8B is a readable Python file you can modify directly. There are no magic YAML keys to discover and no abstracted training loops to reverse-engineer. The implementation is transparent.

Where torchtune wins:

Teams that want to understand and modify the training loop directly will find torchtune's recipe architecture more productive than Axolotl's configuration system. Each recipe is a standalone Python script with clear dependencies. Custom training behaviors (custom loss functions, custom data augmentation, custom checkpoint formats) are implemented by editing the recipe, not by navigating a plugin architecture.

torchtune is the framework most likely to support new PyTorch features quickly. Flash Attention 3, torch.compile optimizations, and new quantization formats land in torchtune close to their upstream release because the library is maintained by the core PyTorch team.

Apple Silicon: torchtune has the best Apple Silicon support of the three frameworks via the MPS (Metal Performance Shaders) backend. Training is slower than CUDA-based tools on equivalent hardware, but it works reliably for smaller models. Llama 3.2 3B and Phi-3 Mini fine-tuning on an M3 Pro with 36GB unified memory is practical with torchtune's MPS backend.

torchtune's limits:

The speed advantage of Unsloth is not available. torchtune relies on PyTorch's standard kernels without the manual optimization that Unsloth brings. Training runs are faster than HuggingFace vanilla PEFT but slower than Unsloth for equivalent setups.

The ecosystem of pre-built configurations is also smaller. Axolotl has a large library of community configs for popular model and dataset combinations. torchtune's recipe library is growing but requires more upfront work to adapt to specific model and dataset combinations.

Comparison: Unsloth vs Axolotl vs torchtune

The Hardware Reality for Local Fine-Tuning

Fine-tuning a 7B model with QLoRA at rank 16 requires roughly:

NVIDIA CUDA: 6 to 10GB VRAM with Unsloth, 10 to 14GB with vanilla PEFT. An RTX 3090 (24GB) or RTX 4090 (24GB) handles 13B models comfortably.

Apple Silicon: 32GB unified memory handles 7B QLoRA reliably. 64GB handles 13B. Fine-tuning is 2x to 3x slower than equivalent NVIDIA setups because the MPS backend lacks Unsloth's optimizations, but it works without CUDA.

Cloud options: Lambda Labs, RunPod, and Vast.ai offer A10G (24GB) and A100 (40/80GB) instances that work well with any of the three frameworks. A typical 7B model fine-tuning run on an A10G with Unsloth costs under $5 in compute.

INTERNAL LINK: self-hosting AI models vs cloud APIs → cost analysis for local inference vs API-based deployment

Which Tool to Start With

If you have a NVIDIA GPU and want to iterate fast on adapters for a well-supported model architecture (Llama, Mistral, Phi, Gemma): start with Unsloth. The speed advantage alone justifies the narrower model support.

If you have complex dataset preprocessing requirements, need multi-GPU training, or are working with a model architecture that Unsloth does not yet support: use Axolotl. The YAML configuration system pays for itself on complex setups.

If you are on Apple Silicon, value training loop transparency, or want to build custom training behaviors: use torchtune. The recipe architecture and PyTorch-native design make it the most maintainable choice for teams that expect to modify the training code.

How Contra Collective Can Help

Contra Collective helps engineering teams build and fine-tune AI models for domain-specific applications. We have run fine-tuning workflows on NVIDIA and Apple Silicon hardware across all three frameworks and know which setups produce reliable adapters versus which ones look good in tutorials and fail in practice. Talk to our team before you invest in a fine-tuning pipeline that may not fit your hardware or use case.