LoRA adapter
A small trained add-on that changes a frozen base model's behaviour without saving or retraining a whole new copy of the model.
See it
What it is
A LoRA adapter is a small set of trainable low-rank matrices inserted alongside selected layers of a frozen base model. Training updates the adapter instead of rewriting billions of original weights, which cuts the memory and storage needed for fine-tuning. The technique comes from the 2021 LoRA paper by Edward Hu and collaborators.
Reach for one when you want a base model to learn a repeated behaviour, format, vocabulary, or style without keeping a full model copy per variant. Adapters are small enough to version and swap, and they can be loaded separately at inference time or merged into the base weights for deployment. Image generation is where the word reached consumers: the LoRA files traded on Civitai bolt one character or art style onto a shared Stable Diffusion checkpoint.
Gotcha: an adapter is tied to the exact base model and the modules it targets. Loading it onto a different checkpoint can fail or quietly produce bad output. LoRA also inherits the ordinary fine-tuning traps: dirty examples teach dirty behaviour, too little data overfits, and changing facts still belong in retrieval rather than weights.
Ask AI for it
Fine-tune this causal language model with Hugging Face PEFT instead of updating the full model. Create a `LoraConfig` with `r=16`, `lora_alpha=32`, `lora_dropout=0.05`, `bias="none"`, and `task_type=TaskType.CAUSAL_LM`; inspect the base model and set `target_modules` to its attention projection layers rather than guessing names. Wrap it with `get_peft_model()`, assert that only adapter parameters require gradients, train against a held-out eval split, save with `save_pretrained()`, reload it with `PeftModel`, and compare its task score and artifact size with the untouched base model.