Fine-Tuning LLMs: LoRA & QLoRA

Fine-tuning is powerful, frequently reached for too early, and often the wrong tool. This tutorial explains the techniques an engineer should understand — LoRA, QLoRA, alignment methods — and, most importantly, the decision of when to fine-tune versus when retrieval or prompting solves the problem better and cheaper.

The decision that matters most#

Before any technique, the real question: should you fine-tune at all? The answer follows one rule.

Full fine-tuning vs parameter-efficient#

Full fine-tuning updates every weight in the model. It is expensive — you need the compute and memory to train a model with billions of parameters — and produces a full-size model per task.

Parameter-efficient fine-tuning (PEFT) updates only a small number of new parameters, leaving the base model frozen. LoRA is the dominant PEFT method. For almost everyone, PEFT is the practical choice.

LoRA#

LoRA (Low-Rank Adaptation) inserts small trainable matrices alongside the model's existing weights and trains only those. The base model is untouched. The result is a small adapter file — often a few megabytes against a multi-gigabyte model — that you load on top of the base at inference.

The benefits:

• Cheap — far less compute and memory than full fine-tuning.
• Swappable — different adapters for different tasks, on one base model.
• Portable — the adapter is small and easy to store and version.

QLoRA#

QLoRA adds quantization: the base model's weights are stored at lower precision (typically 4-bit) to slash memory use, while the LoRA adapters train at higher precision. This is what makes it possible to fine-tune a large model on a single consumer or modest cloud GPU, where full fine-tuning would need a cluster.

The trade-off is a small quality cost from running the base at reduced precision — usually acceptable, and a good deal for the memory saving. See model distillation & quantization for quantization in depth.

Instruction tuning and alignment#

The models you use as assistants have been through more than base training:

• Instruction tuning — training on examples of following instructions, turning a raw next-token predictor into something that responds to requests.
• RLHF (Reinforcement Learning from Human Feedback) — training a reward model from human preference comparisons, then optimising the model against it. This is much of what makes a model helpful and harmless.
• DPO (Direct Preference Optimization) — a simpler alternative to RLHF that optimises directly on preference pairs, achieving similar alignment with less machinery.

These are typically done by model providers, not application teams. You benefit from them; you rarely perform them.

When fine-tuning genuinely helps#

Legitimate cases, once prompting has been tried and found wanting:

• A consistent output format or style that prompting cannot reliably enforce at your volume.
• A narrow, high-volume classification where a fine-tuned small model beats prompting a large one on both cost and accuracy.
• A specialised domain language where the base model's register is persistently wrong.
• Latency or cost — a fine-tuned small model can replace a prompted large one for a fixed task.

A practical path#

1. Prompt engineering first. Most problems dissolve here. Free, instant, no maintenance.
2. RAG for anything knowledge-related. Updatable, citable, no training.
3. Few-shot examples in the prompt for format consistency.
4. Fine-tune only when the above genuinely fall short, and the case is behaviour, not facts.

Reaching step four should be the exception, backed by evidence that steps one to three were insufficient — not the default first move it is often treated as.

Next#

• Model distillation & quantization
• LLM evaluation & benchmarks
• Building a RAG pipeline with Spring Boot