Embeddings, similarity, dimensions, and vector identity

An embedding turns content into a numeric vector whose direction represents learned semantic relationships. Similar meanings tend to be near each other in that model's vector space.

Similarity

Cosine similarity compares direction:

cos(a,b) = (a · b) / (||a|| ||b||)

Dot product is equivalent for unit-normalized vectors. Euclidean distance can work with the index and normalization chosen. Never compare raw threshold numbers across different models or distance functions.

Vector identity

Store with every vector:

• embedding provider, exact model, revision, and dimensions;
• normalization behavior;
• source document/version and chunker version;
• content hash, tenant, language, and authorization metadata;
• creation time and ingestion run.

Index and query content must use the same embedding space. Changing dimensions or model requires a parallel index, re-embedding, evaluation, traffic switch, and rollback window.

Dimension trade-off

More dimensions may improve nuance but increase storage, memory, network, and index cost. Benchmark the actual corpus at 768/1536/3072 or model-appropriate sizes. “Largest” is not a universal quality decision.

Multimodal spaces

Gemini Embedding 2 can map text, images, audio, video, and PDFs into one space. Evaluate cross-modal retrieval separately: text-to-image recall is a different task from text-to-text question answering.

Privacy

Embeddings can leak membership or meaning and remain personal data when tied to people. Apply retention, encryption, access control, deletion, and tenant isolation to vectors and source content.

Feynman check

An embedding is a map coordinate for meaning. Coordinates from two different mapmakers cannot be mixed and expected to point to the same street.