Vector stores, pgvector, HNSW, schemas, and migration

Spring AI's VectorStore API supports pgvector, Redis, Qdrant, Pinecone, Weaviate, Oracle, S3, and other implementations. Portability covers basic operations; indexing, filtering, consistency, scaling, and backups remain store-specific.

pgvector baseline

spring.ai.vectorstore.pgvector:
  index-type: HNSW
  distance-type: COSINE_DISTANCE
  dimensions: 1536
  initialize-schema: false
  schema-validation: true

Schema initialization is opt-in. Production migrations should create extensions, tables, vector dimensions, metadata indexes, and HNSW/IVFFlat indexes through normal database change control.

HNSW and IVFFlat

HNSW offers strong query speed/recall with higher memory and build cost. IVFFlat requires representative data and tuning of lists/probes. Exact search is useful for small corpora and evaluation truth. Approximate nearest-neighbor parameters trade recall for latency.

Multi-tenancy

Choose per-tenant database/schema/table/partition or shared rows with mandatory filters from threat and scale requirements. Do not trust model-generated filters. Construct filters from authenticated application context.

Migration

Use versioned collections/tables:

1. create the new index;
2. backfill from immutable source;
3. verify counts and hashes;
4. run offline retrieval evals;
5. dual-read/shadow;
6. switch an alias;
7. retain rollback;
8. delete only after the retention window.

Operations

Back up source and metadata, not only vectors. Monitor index size, ingestion lag, query latency, filter selectivity, recall samples, dead tuples, and failed deletions.

Feynman check

The vector database is a library shelf arranged by meaning. The catalog, permissions, replacement plan, and original books still matter.