IVF vector index.
An inverted file (IVF) index groups vectors around representative points called centroids. A query searches a selection of those groups instead of comparing every vector. This can reduce work, but it can also miss relevant neighbors.
- Train representative groupsUse enough stored vectors to learn the requested centroids.
- Populate the indexCheck that vectors were assigned to the installed groups.
- Probe selected groupsSet the supported nprobe value for the query.
- Compare with exact searchMeasure missed matches and query time on your test set.
The IVF pipeline.
- Train: use a representative sample to choose the centroids.
- Assign: associate each stored vector with its nearest centroid.
- Search: compare the query with a chosen number of groups.
Measure build time, memory and search quality on your data. This guide does not establish a fixed dataset capacity or build time.
IVF is not declared in the manifest.
The vector-table API configures IVF through its runtime administration calls. Do not add invented index or quantization keys to the schema manifest. The optional vector declaration below describes the dimension and distance rule.
Check the schema reference and the endpoint contract before installing an index.
# manifest.toml — a 768-dim collection. Nothing here selects an index.
[vector]
dim = 768 # required
distance = "cosine" # cosine | l2 | dot | manhattan | l1Install centroids (admin).
This example assumes an existing ivf_demo collection with at least 256 three-dimensional vectors stored using the cosine metric. One vector is not enough: training requires at least four vectors per partition, and this request asks for 64 partitions. Use the vector-write pattern to load your representative dataset first.
Use explicit endpoint settings supplied for your deployment; do not derive them by editing OC_URL. See API connection setup.
curl --fail-with-body -X POST "$OC_BASE_URL/v1/tenants/$OC_TENANT/vector/ivf_demo/train-and-install-centroids" \
-H "Authorization: Bearer $OC_TOKEN" \
-H "Content-Type: application/json" \
-d '{"partitions":64,"max_iterations":25,"seed":42}'Check trained, installed, dim, populated and any populate_skipped reason. Supported single-writer installations populate postings from stored vectors; a replicated configuration can require a separate supported population step. See the training walkthrough.
The nprobe knob.
nprobe controls how many groups a query searches. A larger value usually examines more candidates, which can improve recall at the cost of more work. Recall means the share of relevant nearest neighbors that the approximate search finds.
Compare several values against exact search on a test set. Choose the smallest value that meets your quality requirement.
curl --fail-with-body -X POST "$OC_BASE_URL/v1/tenants/$OC_TENANT/vector/ivf_demo/topk" \
-H "Authorization: Bearer $OC_TOKEN" \
-H "Content-Type: application/json" \
-d '{"query":[0.1,0.2,0.3],"dim":3,"metric":"cosine","index":"ivf","k":10,"nprobe":4}'index: "ivf" selects the trained index. Omitting it selects the default index, so nprobe would not tune the IVF search. The query dimension and metric must match the data you loaded.
When to pick IVF over HNSW.
Consider IVF when its measured memory and search behavior suit your dataset better than HNSW. HNSW is a graph-based index that follows links between nearby vectors. Neither has a universal size threshold.
Test the same queries, filters and update workload with each supported index. For compressed vectors, continue to IVF-PQ.