Start free
← All posts

How vector search works: HNSW explained

OriginChainDB · Jul 14, 2026
vector-search embeddings hnsw engineering

TL;DR - An embedding model turns text (or images, or audio) into a point in a high-dimensional space, placed so that similar meanings land near each other. Vector search is “find the nearest points to this one.” Doing that exactly gets brutally expensive as your data grows, so production systems use approximate indexes - most famously HNSW - that answer in milliseconds while giving up a controlled, measurable amount of recall - recall@10 of 0.96 at p99 109 ms in our 100k-vector benchmark, or 0.69 at p99 37 ms on the same index with a narrower search frontier.

From meaning to geometry

An embedding model is a neural network with an unusual output: instead of a label or a sentence, it emits a fixed-length list of numbers - 384, 1024, 1536, sometimes 3072 of them. That list is a coordinate. Feed the model “How do I reset my password?” and “password recovery steps” and you get two coordinates that sit close together. Feed it “best pizza in Mumbai” and you get one far away from both.

That’s the entire trick. The model was trained so that semantic similarity becomes spatial proximity. Once meaning is geometry, search stops being about matching words and starts being about measuring distance - which is why vector search finds “password recovery steps” for the query “reset my password” even though they share almost no tokens. Keyword search can’t do that; it was never told those phrases are the same idea.

Every vector from a given model has the same dimensionality, and vectors from different models are not comparable - a 1536-dim OpenAI embedding and a 1024-dim Cohere embedding live in unrelated spaces. This is why the model you embed with is a schema-level decision, not an implementation detail (more on that at the end).

Measuring “similar”: cosine, dot product, L2

Distance needs a definition. Three show up everywhere:

The practical rule: use whatever metric the embedding model was trained for. The model’s documentation says which; fighting it costs you recall for no benefit.

Why brute force dies

Exact search is trivial to write: compare the query against every vector, keep the top k. At small scale it’s genuinely fine: below a few tens of thousands of vectors, a straight scan is fast enough that an index isn’t buying you much - we keep an exact-scan path around for exactly that case, and for verifying index recall against ground truth.

The problem is arithmetic. One million 1536-dim vectors means ~1.5 billion multiply-adds per query. Ten million means 15 billion. At useful traffic that’s not a tuning problem, it’s a physics problem - memory bandwidth alone puts a floor under your p99 that no amount of SIMD rescues. Exact search scales linearly with corpus size, and your corpus grows faster than your hardware budget.

ANN: trading a little exactness for a lot of speed

Approximate nearest neighbor (ANN) indexes accept a bounded error to escape the linear scan. The error is measured as recall@k: of the true k nearest neighbors, what fraction did the index return? Recall@10 of 0.96 means that on average 9.6 of the true top-10 made it into your answer.

For semantic search this trade is almost always right. Embeddings are themselves approximations of meaning; the 4% of neighbors an index misses are overwhelmingly the marginal ones, not the obvious hits. What matters is that the trade is tunable and measured - which brings us to HNSW.

HNSW: a highway system for vectors

HNSW (Hierarchical Navigable Small World) is the index behind most production vector search today. Two ideas compose:

Navigable small worlds. Connect each vector to a handful of its near neighbors and you get a graph you can traverse greedily: start anywhere, repeatedly hop to whichever neighbor is closest to the query, stop when no hop improves. Local links alone get stuck, so the construction also keeps some longer-range links - the “small world” property - letting greedy search cross the space in few hops.

Hierarchy. Stack several such graphs. The top layer has few nodes and long links - think highways. Each layer down is denser and shorter-range - arterials, then streets. A query enters at the top, rides the highways to roughly the right neighborhood, then descends layer by layer, refining. The result is search cost that grows roughly logarithmically with corpus size instead of linearly.

Two knobs matter in practice. M - how many links each node keeps - sets the memory/quality baseline of the graph. ef - how wide a frontier the search explores - is the live recall/latency dial: raise it for better recall, lower it for faster answers. In our benchmarks at 100k vectors, the default high-recall setting reaches recall@10 = 0.96 at p99 109 ms, and a fast mode answers at p99 37 ms where recall ≈ 0.69 is acceptable - same index, different ef. The point isn’t the specific numbers; it’s that the trade-off is explicit and yours to set. (Benchmarks →)

Quantization: shrinking the vectors themselves

The index solves “which vectors do I look at.” Quantization solves “how much does each look cost.”

The pattern across all four: spend memory where the search is fine-grained, compress where you’re only shortlisting.

The part everyone forgets: filters

Real queries are rarely “nearest 10 overall.” They’re “nearest 10 where tenant = X and status = active.” A filtered ANN query has to reconcile two machineries: the index proposes candidates by pure similarity, and the predicate disqualifies some of them. The standard mitigation - ours included - is candidate over-fetch: retrieve a deliberately larger set, apply the predicate, return the top k survivors. That holds recall up under moderately selective filters. Under needle-narrow filters - a predicate that passes one row in a million - no fixed multiplier saves you, and the honest plan inverts: filter first, then rank the survivors exactly. When you evaluate any vector system, ask how its filters behave as selectivity climbs; the answers differ more than the headline benchmarks do.

What this looks like in OriginChainDB

Vector search here isn’t a separate product to operate - it’s one query shape over the same store as your rows:

FAQ

Which metric should I pick?

The one your embedding model documents. If vectors are normalized (most text models), cosine and dot product are equivalent - pick either and stay consistent.

Higher-dim models often embed meaning more finely, but each dimension costs memory and bandwidth forever. Several modern models are trained so their vectors truncate gracefully; when yours is, benchmark the shorter length before paying for the full one.

When is brute force the right answer?

Small corpora - roughly under tens of thousands of vectors - and any case where exactness is contractual. A scan is exact and index-free, and at that scale an index isn’t buying you much.

What happens when I switch embedding models?

Every stored vector must be re-embedded - the old and new spaces are incomparable. Plan it like a schema migration: dual-write, backfill, cut over reads.

← All posts Subscribe to RSS →