---
title: "Tokens, transformers, context, and uncertainty"
chapter: "01"
---

# Tokens, transformers, context, and uncertainty

A language model receives tokens and predicts a probability distribution for
the next token. Repeating that operation produces text, JSON, tool calls, code,
or multimodal descriptions. It does not query a hidden database of guaranteed
facts.

## Tokens and context

Tokens are pieces of text, not words. Input instructions, retrieved documents,
tool schemas, history, images, and output all consume context and money.
Context capacity is not memory quality: filling a large window can reduce
signal, increase latency, and expose unnecessary data.

## Parameters and inference

Model parameters contain learned statistical patterns. Inference uses fixed
weights to produce an answer. Fine-tuning changes some weights. RAG changes
the input context without changing weights. Tool calling lets the model request
deterministic work from software.

## Sampling

Sampling settings alter output diversity but do not repair missing evidence.
Current provider families increasingly manage sampling and reasoning
internally. Do not copy temperature defaults between providers. Establish a
task-specific baseline and evaluate it.

## Hallucination

“Hallucination” means unsupported or fabricated content. Reduce it with:

- authoritative retrieval and citations;
- constrained structured output;
- tools for current facts and calculations;
- explicit abstention when evidence is insufficient;
- claim-level groundedness tests;
- authorization and deterministic validation outside the model.

## Reasoning

More reasoning effort can improve difficult tasks while raising latency and
cost. It cannot make an impossible or under-specified task valid. Route by task:
cheap models for bounded extraction; balanced models for most interactions;
flagship reasoning for complex, high-value decisions with strong evaluation.

## Feynman check

The model is a brilliant sentence completer with learned patterns. Explain why
that can create both an excellent analyst and a confident mistake.