A base model rarely meets all your application's needs out of the box. It might use the wrong tone, occasionally ignore your required output format, lack knowledge about your company, or be too expensive to run at scale.

There are several ways to address these problems. You can improve the instructions, provide better context, or fine-tune the model on your own data. Each approach solves different problems and comes with its own trade-offs in cost, complexity, and maintenance.

In this chapter, we'll cover how to choose the right approach, how fine-tuning works, how to prepare training data, how to serve fine-tuned models at scale, and the common mistakes to avoid.

There are several ways to adapt a model to your application's needs. These approaches form a ladder, from simple prompt changes to training your own model. As you move up, the effort, cost, and complexity generally increase.

A good rule is to start with the simplest approach that could meet your requirements and introduce more complexity only when you have evidence that it's needed. Let's look at what each approach offers and when to use it.

Prompting is the simplest starting point. You write clear instructions, provide a few examples, and specify the expected output format. Changes are quick to make, and you can measure their impact against an evaluation dataset. Before trying more complex approaches, make sure you've tested and improved your prompts.

RAG retrieves relevant information and adds it to the prompt at request time. It's useful when the model lacks the knowledge needed to answer correctly, such as your product catalog, internal policies, or recent support tickets. You can keep this information up to date by updating your document index without retraining the model.

Fine-tuning updates the model's weights using examples of the behavior you want. It's useful when prompting alone isn't reliable enough, such as when the model struggles to follow a specific format, maintain a consistent tone, or perform a specialized task. Fine-tuning can also help a smaller model perform a narrow task well, reducing inference costs at scale.

Pretraining involves training a model from scratch on a large dataset. Building a competitive large model can cost millions of dollars and requires substantial compute, data, and engineering expertise. It's rarely practical for product teams. One example is BloombergGPT, a 50-billion-parameter model trained on a mixture of financial and general-purpose text for financial language tasks.

Between fine-tuning and training from scratch, there's another option called continued pretraining. Instead of starting from scratch, you continue training an existing model on large amounts of domain-specific text, such as medical literature or legal documents. This can improve the model's understanding of specialized language and concepts before fine-tuning it for specific tasks.

These approaches aren't mutually exclusive. A production system might use a fine-tuned model to follow a consistent response format, RAG to provide current information, and prompting to define the task. The ladder is a guide to choosing the right level of investment, not a sequence you must follow.

In practice, the decision that comes up more often is whether fine-tuning is worth the additional effort. Let's look at when it makes sense.

Fine-tuning is most useful for changing how a model behaves, rather than teaching it facts. It works well when you want the model to learn consistent patterns from examples, but it's less reliable for information that needs to stay accurate and up to date.

Where Fine-Tuning Works Well

Where Fine-Tuning Falls Short

Fine-tuning is a poor fit for teaching a model factual knowledge. Suppose you fine-tune a model on your product documentation so it can answer customer questions. Several problems can arise:

Access control is another concern. RAG can restrict retrieval to documents each user is authorized to access. Fine-tuning offers no equivalent way to enforce document-level permissions on information learned in the weights. Sensitive training data may also be memorized and exposed.

Here's a quick guide to choosing the right approach:

Once you've decided that fine-tuning is worth the effort, there are two important choices to make: how much of the model to update and what training method to use.

Full Fine-Tuning vs. Parameter-Efficient Fine-Tuning

Full fine-tuning updates all the model's weights during training. This gives the model the most flexibility to learn new behavior, but requires substantial GPU memory.

During training, the GPU must store more than just the model weights. With mixed-precision training and the Adam optimizer, a common estimate is 16 bytes per parameter: 2 bytes for the weights, 2 for gradients, 4 for a full-precision copy of the weights, and 8 for optimizer states.

For an 8B model, that's:

That's already more than a single 80 GB GPU can hold. A 70B model requires roughly 1.12 TB before activations. Each fine-tuned 8B model also produces a separate checkpoint of around 16 GB in FP16, which must be stored and deployed.

Parameter-efficient fine-tuning (PEFT) reduces this cost by freezing the original weights and training a small number of additional parameters. The most widely used approach is LoRA (Low-Rank Adaptation).

LoRA is based on the idea that the changes needed to adapt a model can often be represented using much smaller matrices. Instead of updating a large weight matrix W, LoRA keeps it frozen and learns two smaller matrices, A and B, whose product represents the weight update.

The input passes through the frozen weights and the two trainable matrices in parallel. Their outputs are added together. In this example, 16 is the rank, which controls the adapter's capacity. Common ranks range from 8 to 64.

The parameter savings are significant. A 4096 × 4096 weight matrix contains about 16.8 million parameters. With rank 16, matrices A and B contain just 131,072 parameters combined, less than 1% of the original.

For a typical 8B model, applying rank-16 LoRA to all major linear layers adds roughly 40 million trainable parameters, depending on the architecture. That's around 80 MB in FP16, compared with 16 GB for the full model.

Training memory also drops because gradients and optimizer states are only needed for the adapter. An 8B model can be fine-tuned with LoRA on a single 80 GB GPU, often with room to spare.

QLoRA reduces memory usage further by storing the frozen base model in 4-bit precision while training the adapters at higher precision. An 8B model's quantized weights take roughly 4 to 5 GB, making fine-tuning possible on a single 24 GB GPU with appropriate batch sizes, sequence lengths, and memory optimizations.

Here's how the three approaches compare:

For most product use cases that need fine-tuning, LoRA is a good starting point. It's cheaper to train, easier to experiment with, and produces small adapters that can share the same base model during serving.

Full fine-tuning is worth considering when you need substantial changes to the model's behavior, have enough training data and compute, and evaluations show that LoRA isn't meeting your requirements.

Supervised Fine-Tuning vs. Preference Tuning

The second decision is how to train the model. There are two common approaches, depending on the type of training data you have.

Supervised fine-tuning (SFT) trains the model on examples of inputs and desired outputs. For instance, you provide a customer support ticket along with a response written by an experienced support agent. The model learns to produce similar responses for new tickets.

SFT works well when you can provide clear examples of what a good answer should look like.

But for some tasks, it's easier to compare two responses than to write an ideal one from scratch. You might know that one summary is clearer than another, or that one support response is more helpful.

Preference tuning learns from these comparisons. Each training example contains a prompt, a preferred response, and a rejected response. The goal is to make the model more likely to produce responses that match the preferred behavior.

Two common approaches are:

These methods are often used in sequence:

SFT teaches the model how to perform the task. Preference tuning then refines its behavior, such as favoring concise, helpful responses over unnecessarily long or vague ones.

SFT is usually the simpler place to start. Preference tuning becomes useful when SFT alone isn't producing the desired behavior and you have reliable comparison data, such as users choosing between two generated drafts.

Another common reason to fine-tune is to replace an expensive model with a smaller, faster one. This approach is called model distillation, and it has its own techniques and trade-offs.

Distillation trains a smaller student model to learn from a larger teacher model. The goal is to achieve comparable quality on a specific task while reducing inference cost and latency.

In production, the process is often straightforward. You run the teacher model on a representative set of inputs, filter its responses for quality, and use the resulting input-output pairs to fine-tune the student.

Filtering is important because even strong teacher models make mistakes. If you train the student on incorrect responses, it can learn those mistakes too. You can filter outputs using schema validation, rule-based checks, LLM judges, and human review for cases that are difficult to evaluate automatically.

The main benefit of distillation is lower inference cost at scale.

Suppose you're building a support ticket classifier that handles 5 million requests per day. Each request contains around 1,000 input tokens and produces 50 output tokens.

Here's an illustrative cost comparison:

The actual savings depend on the models, hardware, throughput, and GPU utilization. The self-hosted estimate assumes efficient serving and doesn't include training costs. Still, for narrow, high-volume tasks, distillation can reduce inference costs by 10x or more while improving latency.

There are two important considerations:

If you host the teacher yourself, you can also use logit distillation, where the student learns from the teacher's probability distribution over tokens rather than just its generated text. This provides a richer training signal, but typically requires access to model internals that most hosted APIs don't expose.

Whether you're distilling a larger model or fine-tuning on human-written examples, the quality of the training data is one of the biggest factors in the final result.

Training data is often the biggest factor in whether fine-tuning succeeds. A few hundred to a few thousand high-quality examples can be enough to teach a model a specific format, tone, or narrow task. A thousand clean, representative examples can outperform ten thousand noisy ones.

There are three main sources of training data.

Production traces. If your application already uses an LLM, its logs contain real user inputs and model responses. User feedback can help identify good responses, such as a thumbs up, an accepted draft, or a support ticket resolved without escalation. These are useful signals, though they don't always guarantee correctness.

When users edit generated content, the corrected version can become a supervised training example. The original and corrected responses may also serve as preference data if the correction represents a clear improvement.

However, production data requires careful handling. Remove or mask personal information, follow data retention policies, and verify that your agreements permit using customer data for training. Some enterprise customers explicitly prohibit it.

Synthetic data. You can use a stronger model to generate training examples, including cases that rarely appear in production logs, such as uncommon languages, unusual document formats, or difficult edge cases.

Synthetic data helps expand coverage quickly, but it can contain errors and reflect the generating model's biases. It may also lack the diversity of real user inputs. Validate generated examples and combine them with real data rather than relying entirely on synthetic outputs.

Human labeling. Domain experts can write responses, correct model outputs, or rank multiple answers. This is usually the most expensive approach, but it's particularly valuable in specialized fields such as medicine and law, where evaluating correctness requires domain expertise.

Provide clear labeling guidelines and measure agreement between reviewers. If experts frequently disagree about the correct answer, you need to resolve those inconsistencies before using the examples for training.

These sources feed into a data preparation pipeline:

The train/evaluation split is especially important. Ideally, create a representative evaluation set before training and keep it isolated from training and synthetic data generation. Otherwise, near-duplicate examples or leaked answers can make your model appear better than it actually is.

Split data carefully, grouping related conversations, users, or documents where necessary to prevent leakage. Time-based splits are also useful for measuring performance on newer inputs.

Finally, pay attention to dataset balance. If 90% of your production logs involve password resets, training directly on that distribution may leave the model poorly prepared for other support requests. Include enough examples of less common but important cases, and evaluate performance across categories rather than relying only on overall accuracy.

Once you've trained a model, the next challenge is serving it efficiently, especially when you need to support multiple fine-tuned versions.

Serving one fine-tuned model is similar to serving any other LLM. The challenge comes when you need to serve many of them.

For example, a B2B writing assistant might have a separate fine-tuned model for each enterprise customer to match their writing style. Another platform might use different fine-tuned models for summarization, classification, and data extraction.

There are two main approaches to serving these models.

One Deployment per Model

With full fine-tuning, each model has its own complete set of weights. An 8B model in 16-bit precision requires roughly 16 GB just for its weights. If you have 100 customer models, that's 1.6 TB of model weights.

Each model also needs GPU capacity when serving requests. Since most customers don't generate traffic continuously, maintaining dedicated GPU instances can leave a lot of compute capacity idle.

This approach works when you have a small number of high-traffic models, customers that require strict isolation, or models that need full fine-tuning. However, it becomes expensive when you have hundreds of models serving relatively little traffic.

Many LoRA Adapters on One Base Model

LoRA offers a more efficient alternative. Instead of maintaining a full model for every customer, you load the base model once and serve multiple small adapters alongside it.

Serving engines such as vLLM support multi-LoRA inference, allowing requests using different adapters to share the same base model. Systems like S-LoRA have also demonstrated how to serve thousands of adapters by managing them across GPU and CPU memory.

Each request includes an adapter ID, typically assigned by the gateway based on the tenant or task. The serving engine selects the corresponding adapter and can batch requests using different adapters together.

Because the expensive base model weights are shared, only the adapter-specific computation differs between requests. Frequently used adapters can remain in GPU memory, while less frequently used ones are loaded from CPU memory or storage when needed.

For 100 customer models, the difference in weight memory is substantial:

That's a reduction from 1.6 TB to roughly 24 GB of model and adapter weights. Actual GPU memory requirements will be higher because inference also needs KV cache, temporary buffers, and other runtime resources. Not all adapters need to remain in GPU memory simultaneously.

However, shared adapter serving introduces a few trade-offs:

Another option is managed fine-tuning from hosted model providers. You upload your training dataset, run a fine-tuning job, and receive a model ID to use for inference. The provider handles deployment and scaling, but you have less control over model selection, training methods, and serving costs.

For multi-tenant applications with many low-traffic adapters, shared LoRA serving is a good starting point. It reduces memory requirements and improves GPU utilization, while dedicated deployments remain useful for high-traffic customers or workloads that require stronger isolation.

Efficient serving addresses the infrastructure side of fine-tuning. The next challenge is making sure your adapted model actually performs better than the original.

Fine-tuning doesn't always improve a model. It can introduce problems that aren't obvious from training metrics alone. Watch for three failure modes.

Overfitting to the Training Set

When you train for too many epochs on a small dataset, the model may memorize examples instead of learning patterns that generalize to new inputs.

A common sign is that training loss keeps decreasing while validation loss starts increasing. The model performs well on familiar inputs but struggles with slightly different ones. It may even reproduce phrases from the training data word for word.

Monitor validation loss and task-specific metrics during training, and use early stopping or select the checkpoint that performs best on held-out data. For LoRA with small datasets, one to three epochs is often a reasonable starting point, though the right number depends on the task and dataset.

Most importantly, evaluate on representative inputs the model hasn't seen, including cases that differ from the training examples.

Losing General Ability

Fine-tuning a model on a narrow task can weaken capabilities it already had. This is known as catastrophic forgetting.

For example, a model heavily fine-tuned for JSON extraction might become worse at following general instructions or answering questions in natural language.

Safety behavior can also degrade. A 2023 study by Qi et al. found that fine-tuning an aligned model, even on seemingly harmless data, can weaken its ability to refuse harmful requests. If users interact directly with your fine-tuned model, include safety evaluations rather than assuming the base model's behavior will remain unchanged.

LoRA preserves the original weights, but the adapted model can still lose capabilities when the adapter is active. Mixing some general instruction-following examples into the training data can help preserve broader behavior.

Run general capability and safety evaluations alongside task-specific tests to catch regressions before deployment.

Evaluation Data Leaking Into Training

Evaluation leakage is hard to spot because it can make a poorly performing model look successful.

If evaluation examples, or near-duplicates of them, appear in the training data, the model may score well simply because it has already seen similar answers. The results look promising during development, but performance drops when the model encounters real production traffic.

Leakage can happen in several ways:

Preventing leakage requires a disciplined data preparation process.

Create and freeze your evaluation set before training. Check for duplicates across datasets using exact matching and semantic similarity. Where appropriate, split by time so evaluation data comes from a later period, and group related records by customer, conversation, or document so they don't appear in both sets.

Also avoid repeatedly tuning your model against the same evaluation examples. Over time, this can lead to overfitting to the evaluation set itself. Keep a separate test set for the final assessment.

Here's a summary of the three failure modes:

Suppose you're designing an email drafting feature for a B2B customer support platform with 300 enterprise customers. Each customer wants replies that match its brand voice and follow its own policies. The platform generates 2 million drafts per day.

Here's how you could apply the adaptation ladder:

The resulting architecture looks like this:

Each component has a clear responsibility. Fine-tuning helps maintain a consistent brand voice and output format. RAG supplies current policies and product information. Shared LoRA serving keeps the cost of supporting 300 customer-specific models manageable, while the fallback ensures customers can still use the feature without a fine-tuned adapter.

One important maintenance cost is upgrading the base model. LoRA adapters are tied to the model they were trained on, so moving to a new base model generally means retraining or adapting the customer adapters and evaluating them again. For 300 customers, that requires a versioned training pipeline and a gradual rollout rather than a simple model replacement.

Here are the key decisions when adapting models for a GenAI application:

Regardless of the approach, invest in training data quality and representative evaluations. Keep a separate, deduplicated evaluation set to measure whether the adapted model actually improves on the baseline, including any regressions in general capabilities or safety.

Start with prompting or RAG, measure where they fall short on your evaluation set, and move to fine-tuning, preference tuning, or distillation only when the results show a clear improvement.