RAG or fine-tuning?
Put knowledge where it belongs.

Retrieval changes the context. Fine-tuning changes the weights.

Retrieval-augmented generation (RAG) searches a collection outside the model, selects material relevant to a question, and adds it to the context used to produce an answer. The collection can be updated without retraining the model, although the ingestion and indexing pipeline still has to run. Whether the answer is traceable depends on the system: it must preserve source references and show which evidence actually supports the claim. For the full search-to-answer flow, read what RAG is and how each handoff works ↗.

Fine-tuning uses examples to update a model’s parameters. It can teach a model to follow a task pattern, use a consistent format, or respond in a particular style. It does not create a dependable, easily editable index of facts: changing a document in a database and changing what a model has learned are different operations.

Retrieval itself can run as a fixed application step or through a model-selected search tool. RAG describes how external evidence is brought into the context; tool calling describes one possible way a model can request an operation. They are separate design choices. See the tool-calling round trip ↗.

Ask whether the knowledge changes—or the behavior does.

These are starting points, not rules of nature. A knowledge task may also need a better prompt or a smaller, curated source set. See how retrieval fits into the agent’s wider context ↗. A model may already follow the desired format without tuning. Measure the simplest design that meets the need.

RAG can retrieve the wrong thing. A tuned model can learn the wrong pattern.

Retrieval quality depends on how documents are split, embedded, indexed, ranked, and filtered. An answer can sound confident when the relevant passage was never retrieved—or when the source is stale, contradictory, or outside the caller’s access. A source link is useful only when it points to evidence that supports the answer. For the search layer itself, read how embeddings and vector retrieval work ↗.

Fine-tuning depends on the quality and coverage of its examples, as well as the training process. A model can learn shortcuts that look successful on familiar examples but fail on new cases. A change to its parameters is not a simple content update, so keep evaluation data separate and test the behavior after each training change.

Neither technique guarantees truth. Retrieval creates a route to inspect external evidence; tuning can improve performance on particular tasks. The application still needs permission checks, evaluation, and a clear way to handle missing or conflicting information.

Keep changeable facts outside. Teach the model how to work with them.

A combined system can use retrieval for current, attributable material and fine-tuning for a stable response pattern. For example, a support assistant might retrieve the current policy and learn a consistent, concise response format. The retrieved source still needs to support each policy claim; tuning the response style does not make a source current.

Combining methods adds moving parts: an index and its update process, a trained model and its version history, and evaluation for the whole interaction. It earns that complexity only when the same task set shows a benefit over the simpler approach.

For a broader look at persistent context and its trade-offs, read Memory is a design choice ↗. For a practical diagnostic of retrieval and answers, see How to evaluate a RAG system ↗. For benchmark interpretation, see How to read AI benchmarks ↗.

Match the fix to the error you can reproduce.

Start with a prompt-only baseline and a small set of representative requests. For every failure, record the expected answer, the information the model received, and the point where the result stopped meeting the requirement. This makes a missing source distinguishable from a poor search or a model that ignores relevant context.

1. The answer needs a fact that was never supplied. Confirm the source is authoritative and permitted, then test whether retrieval can find the relevant passage.
2. The correct passage was retrieved but ignored or misused. Improve the context, instructions, or answer checks first; consider tuning only if the same behavior problem repeats across varied cases.
3. The task is a stable transformation. If inputs contain the required information but outputs repeatedly miss a format or label convention, evaluate prompt changes and a small, representative supervised dataset.
4. Neither method fixes the root cause. The problem may be bad source data, an ambiguous requirement, a missing permission, or a task better handled by deterministic code.

Retrieval can fail before generation begins; fine-tuning cannot supply a source the system never received. Likewise, a perfectly retrieved passage cannot guarantee that a model will apply it correctly. Evaluate each stage separately and then test the complete task.

Compare the cost of keeping each approach useful.

Prices vary by provider, model, corpus, request pattern, and deployment. Compare a realistic period of operation, including source updates and evaluation—not just the cost of one model call or one training run. A smaller prompt-only baseline may meet the same target at lower total effort.

For the wider system view, continue to the cost of intelligence ↗ and how inference behaves at runtime ↗.

Measure answers, evidence, operations, and regressions together.

Keep the test cases fixed while comparing the baseline prompt, retrieval, tuning, or a combination. Use realistic cases that were not copied into training data. Include the ordinary workload and the cases that reveal whether the approach fails safely.

• For retrieval: Was a relevant and current source found? Was it authorized for this user? Did the final answer follow it?
• For fine-tuning: Did the model perform better on held-out examples? Did it keep the expected format without inventing missing information?
• For both: Could the system abstain or ask for clarification when the source was missing or the request ambiguous?
• For operations: What changed in latency, total cost, data handling, update time, and review effort?
• For regressions: Did the change harm another task, language, user group, or safety boundary?

Keep retrieval metrics and answer metrics separate: strong search does not guarantee a supported answer, and a polished answer does not prove the right evidence was found. Use human review to calibrate automated scoring where the outcome requires judgment. See the RAG evaluation guide ↗ and how to interpret model benchmarks ↗.

A method can be sound while a particular service changes.

Fine-tuning is a general model-adaptation method, but managed platforms, access rules, and supported base models are provider-specific. Verify availability, data terms, model support, export options, and retirement dates before making the method a product dependency.

Availability changes quickly. Recheck provider documentation when evaluating an implementation, and keep the architecture portable enough to reassess if its model or training service changes.