---
title: "Fine-tuning decisions: SFT, DPO, RFT, distillation, or RAG"
chapter: "18"
---

# Fine-tuning decisions: SFT, DPO, RFT, distillation, or RAG

Fine-tuning continues training to change model behavior. It is not the first
answer to “the model does not know our latest policy.”

## Decision table

| Need | First choice |
|---|---|
| Current private knowledge | RAG or a governed tool |
| Consistent format/tone | Prompt + structured output; then SFT |
| Subjective preference | DPO if supported |
| Complex measurable reasoning | RFT if supported and experts agree |
| Lower cost/latency | Distill strong outputs into a smaller supported model |
| Local domain behavior | LoRA/QLoRA adapter plus evaluation |

## Methods

- SFT learns from prompt/ideal-response examples;
- vision SFT includes image inputs;
- DPO learns from preferred versus rejected response pairs;
- RFT samples outputs and learns from a grader/reward;
- distillation uses a stronger model to create/label data for a smaller model;
- LoRA trains low-rank adapters instead of all weights.

## Data

Split train/validation/test before iteration. Remove duplicates and leakage.
Include hard negatives, edge cases, refusals, tool decisions, and production
distribution. Every example needs provenance, rights, privacy review, and a
clear expected behavior.

## Current provider reality

Support changes rapidly. Gemini API tuning is currently unavailable. OpenAI's
current optimization docs describe SFT, DPO, vision, and RFT methods, while
current RFT documentation announces platform wind-down restrictions for new
users. Verify account/model availability before designing a program.

## Evaluation

Compare base prompt, RAG/tool baseline, and tuned model on a held-out set.
Measure regressions, safety, latency, token savings, and drift—not training
loss alone.

## Feynman check

RAG gives the student a current book. Fine-tuning changes study habits. Do not
retrain the student every time the policy manual changes.