02 — Vector Search

Key Concepts

Core

Embeddings with ONNX

Tokenize → run the ONNX model → mean-pool token vectors → normalize. Same model as sentence-transformers, ~30x lighter install — no PyTorch, no CUDA.

all-MiniLM-L6-v2384-dimonnxruntime

Core

Vector Search with numpy

Normalized vectors mean dot product equals cosine similarity. One matrix-vector multiply (X.dot(v)) scores every document at once.

cosine similarityargsorttop-k

Core

Chunking

A full page mixes many topics, which dilutes its embedding. Splitting into overlapping windows keeps each chunk's vector focused on one idea.

size=2000step=1000overlap

Core

minsearch VectorSearch

Same fit/search API as text search, just backed by embeddings instead of an inverted index. filter_dict still works for narrowing by keyword fields.

fitsearchfilter_dict

Tradeoffs

Keyword vs Vector Search

Keyword search nails exact terms but misses paraphrases. Vector search matches meaning but can miss code-heavy or rare-term chunks. Neither wins outright.

BM25semanticrecall gaps

Production

Hybrid Search (RRF)

Reciprocal Rank Fusion merges ranked lists by position, not raw score. A document strong in both searches can beat one that's great in only one.

1/(k+rank)k=60rank fusion

Production

sqlitesearch

Persistent vector index backed by SQLite. ANN modes (LSH, IVF, HNSW) trade a little recall for a lot of speed once datasets grow past ~10K vectors.

ANNHNSWpersistent

Production

PGVector

Postgres + the pgvector extension. Concurrent reads/writes, transactions, and an HNSW index for scale — the production-grade option of the three.

pgvector<=> operatorDocker

Single Search vs Hybrid Search

Single method

     Query
       ↓
Vector OR Text Search
       ↓
   Ranked List
       ↓
  Top Results

Hybrid (RRF)

         Query
        ↙      ↘
  Vector Search  Text Search
        ↘      ↙
      RRF Fusion
          ↓
     Top Results