Semantic, lexical, hybrid search, and reranking

Semantic search retrieves meaning, lexical search retrieves exact terms, and hybrid search combines both. Production search usually needs all three plus metadata filtering and reranking.

Retrieval pipeline

query → normalize → intent/tenant filter → lexical candidates + vector candidates → fuse → rerank → threshold → diversify → evidence

Vector search is strong for paraphrases. BM25/keyword search is strong for error codes, product IDs, names, dates, and rare exact terms. Reciprocal rank fusion combines rankings without pretending incomparable raw scores match.

Reranking

A cross-encoder or capable reranking model scores query-document pairs more precisely after broad retrieval. Retrieve perhaps 30–100 candidates, rerank, then send a small evidence set to generation. Measure added latency and cost.

Filters and authorization

Apply tenant, region, document status, effective date, and permission filters inside retrieval—not after the model sees results. Metadata is part of the security boundary.

Evaluation

Use labeled queries with relevant document IDs. Measure recall@k, precision@k, MRR, nDCG, no-result rate, latency, and permission violations. Then measure answer groundedness separately. A fluent answer cannot prove retrieval worked.

Thresholds

A similarity threshold is a rejection policy, not a magic truth line. Tune per embedding model and corpus. If no candidate qualifies, ask a narrower question or abstain rather than filling context with noise.

Feynman check

Keyword search finds the same label; semantic search finds the same idea; reranking is a careful second reader choosing which results deserve attention.