Fine-tuning

Fine-tuning continues training an existing model on your own input/output pairs so the behaviour is baked into the weights. A common and cheap way to do it is a LoRA adapter: a small set of extra weights trained on top of a frozen base model, quick to train and swap, and the usual choice when you run the training yourself. Hosted fine-tuning APIs may use that or something else entirely, and often will not say which. The pay-off is behaviour you no longer have to describe: format, tone, labelling conventions, and house style come out right with a much shorter prompt.

Reach for it after prompting and few-shot examples have plateaued, when you have a narrow repeated task, thousands of daily calls where the instruction tokens actually cost money, or when you want a small fast model to imitate an expensive one (distillation). Budget a few hundred to a few thousand clean pairs plus a held-out eval set you never train on.

Gotcha, and it is the big one: fine-tuning is much better at shape than at facts. It can shift what a model asserts, so it is not literally fact-free, but it is a terrible store for knowledge that changes or that you need to cite a source for. People try it as a memory upgrade ('train it on our docs'), get a model that confidently talks like the docs while inventing their contents, and blame the model. Anything fresh, specific and checkable belongs in retrieval and context. Also budget for maintenance: your tuned model is pinned to one base snapshot, so every base upgrade means retraining and re-running evals.