---
title: "Local fine-tuning, LoRA adapters, quantization, and Ollama"
chapter: "19"
---

# Local fine-tuning, LoRA adapters, quantization, and Ollama

Ollama is primarily an inference and model-packaging runtime. Train adapters
with tools such as Hugging Face, Unsloth, or MLX, then import compatible
Safetensors or GGUF artifacts.

## Adapter workflow

1. select a licensed base model and exact revision;
2. create clean train/validation/test datasets;
3. train LoRA/QLoRA with recorded hyperparameters and seeds;
4. evaluate the adapter and base model independently;
5. export a supported adapter format;
6. import with the same base model;
7. optionally quantize and re-evaluate;
8. package provenance, license, model card, and rollback artifact.

```text
FROM <exact-base-model>
ADAPTER /models/claim-adapter
PARAMETER num_ctx 8192
SYSTEM You extract claim facts into the supplied schema.
```

```bash
ollama create claims-extractor -f Modelfile
ollama run claims-extractor
```

The base in `FROM` must match training; otherwise output may be erratic.

## Quantization

Quantization reduces memory and may improve speed at a quality cost. Benchmark
each quantization on the real task, long context, tools, and JSON conformance.
Do not infer quality from file size or a generic leaderboard.

## Reproducibility

Record base hash, dataset hash, code/container, library versions, GPU type,
precision, epochs, learning rate, rank/alpha, prompt template, and evaluation
results. Scan model artifacts like other supply-chain dependencies.

## Serving

Pin the packaged model, pre-pull it, warm deliberately, bound concurrency, and
monitor load time, tokens/second, queue time, memory, context truncation, and
fallbacks.

## Feynman check

LoRA is a small removable lesson attached to a large textbook. Quantization
prints the textbook smaller; some fine detail may disappear.