Inference Foundrymodern AI systems fieldbook
Spring AI 2 · Grounded Agentic SystemsView Markdown source

Fine-tuning decisions: SFT, DPO, RFT, distillation, or RAG

Fine-tuning continues training to change model behavior. It is not the first answer to “the model does not know our latest policy.”

Decision table

Need First choice
Current private knowledge RAG or a governed tool
Consistent format/tone Prompt + structured output; then SFT
Subjective preference DPO if supported
Complex measurable reasoning RFT if supported and experts agree
Lower cost/latency Distill strong outputs into a smaller supported model
Local domain behavior LoRA/QLoRA adapter plus evaluation

Methods

  • SFT learns from prompt/ideal-response examples;
  • vision SFT includes image inputs;
  • DPO learns from preferred versus rejected response pairs;
  • RFT samples outputs and learns from a grader/reward;
  • distillation uses a stronger model to create/label data for a smaller model;
  • LoRA trains low-rank adapters instead of all weights.

Data

Split train/validation/test before iteration. Remove duplicates and leakage. Include hard negatives, edge cases, refusals, tool decisions, and production distribution. Every example needs provenance, rights, privacy review, and a clear expected behavior.

Current provider reality

Support changes rapidly. Gemini API tuning is currently unavailable. OpenAI's current optimization docs describe SFT, DPO, vision, and RFT methods, while current RFT documentation announces platform wind-down restrictions for new users. Verify account/model availability before designing a program.

Evaluation

Compare base prompt, RAG/tool baseline, and tuned model on a held-out set. Measure regressions, safety, latency, token savings, and drift—not training loss alone.

Feynman check

RAG gives the student a current book. Fine-tuning changes study habits. Do not retrain the student every time the policy manual changes.

Inference FoundryIndependent study material · verify version details in official project documentation