Fine-tuning decisions: SFT, DPO, RFT, distillation, or RAG
Fine-tuning continues training to change model behavior. It is not the first answer to “the model does not know our latest policy.”
Decision table
| Need | First choice |
|---|---|
| Current private knowledge | RAG or a governed tool |
| Consistent format/tone | Prompt + structured output; then SFT |
| Subjective preference | DPO if supported |
| Complex measurable reasoning | RFT if supported and experts agree |
| Lower cost/latency | Distill strong outputs into a smaller supported model |
| Local domain behavior | LoRA/QLoRA adapter plus evaluation |
Methods
- SFT learns from prompt/ideal-response examples;
- vision SFT includes image inputs;
- DPO learns from preferred versus rejected response pairs;
- RFT samples outputs and learns from a grader/reward;
- distillation uses a stronger model to create/label data for a smaller model;
- LoRA trains low-rank adapters instead of all weights.
Data
Split train/validation/test before iteration. Remove duplicates and leakage. Include hard negatives, edge cases, refusals, tool decisions, and production distribution. Every example needs provenance, rights, privacy review, and a clear expected behavior.
Current provider reality
Support changes rapidly. Gemini API tuning is currently unavailable. OpenAI's current optimization docs describe SFT, DPO, vision, and RFT methods, while current RFT documentation announces platform wind-down restrictions for new users. Verify account/model availability before designing a program.
Evaluation
Compare base prompt, RAG/tool baseline, and tuned model on a held-out set. Measure regressions, safety, latency, token savings, and drift—not training loss alone.
Feynman check
RAG gives the student a current book. Fine-tuning changes study habits. Do not retrain the student every time the policy manual changes.