Group vectors for IVF search
The language tabs use earlier client interfaces. Check SDK versions and connection setup before running them.
IVF divides vectors into groups around representative points called centroids. Training these points prepares the index to search selected groups.
Before you run it: Training needs at least four vectors per partition. Reinstalling replaces the centroid matrix and postings.
First store a representative set of vectors in shop.products. The request below asks for 1,024 partitions, so its minimum training set is 4,096 vectors. A one-vector quickstart is not enough. A partition is a group of nearby vectors; postings are the entries that assign vectors to those groups.
- Load representative vectorsProvide enough vectors for the chosen partition count.
- Train and install centroidsCreate representative points for the groups.
- Check assigned postingsRead how many stored vectors received group entries.
- Inspect coverageInvestigate skipped population instead of assuming every vector is searchable.
Follow the example
- Store enough vectors for the partition count.
- Run train-and-install-centroids.
- Wait for the synchronous request.
- Read the installed-centroid summary.
Run the request
curl -X POST "https://$OC_HOST/v1/tenants/$OC_TENANT/vector/shop.products/train-and-install-centroids" \
-H "Authorization: Bearer $OC_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"partitions": 1024,
"init": "kmeans_plus_plus",
"max_iterations": 50,
"batch_size": 1024,
"convergence_threshold": 1e-5,
"seed": 42
}'Check the result
The index installs centroids and builds postings for stored vectors without ingesting them again.
{
"trained": true,
"installed": true,
"partitions": 1024,
"dim": 768,
"iterations": 37,
"converged": true,
"last_max_shift": 0.0000073,
"training_corpus_size": 100000,
"populated": 100000
}
// populate_skipped is present only when the backfill did not run.populated is the number of stored vectors that got a cell posting from this call. When it is well below training_corpus_size, or a populate_skipped string is present, the index is installed but not fully covering the corpus - the coverage report will show it.
Request options
| Field | Required | Notes |
|---|---|---|
| partitions | yes | How many Voronoi cells to train. The engine caps this at 65,536, and refuses below the k-means floor described under common mistakes. |
| init | no | "kmeans_plus_plus" or "random_sample". Defaults to "random_sample". |
| max_iterations | no | Mini-batch iteration cap. Defaults to 100. |
| batch_size | no | Mini-batch size. Defaults to 1024. |
| convergence_threshold | no | Early-stop threshold on the per-iteration maximum centroid shift. Defaults to 1e-4. |
| seed | no | PRNG seed. Pass one if you want two runs over the same corpus to produce the same partitioning. |
Inspect installed centroids
curl "https://$OC_HOST/v1/tenants/$OC_TENANT/vector/shop.products/centroids" \
-H "Authorization: Bearer $OC_TOKEN"Read the centroid summary
{
"installed": true,
"partitions": 1024,
"dim": 768,
"centroids_preview": [
[0.0124, -0.0883, 0.0451, 0.0037, -0.0192, 0.0608, -0.0271, 0.0115]
/* ... first 4 centroids, first 8 dims of each ... */
]
}The preview is deliberately truncated to the first 4 centroids and the first 8 dimensions of each, so the response stays small whatever partitions × dim is. It is an "is anything installed" check, not a way to export the matrix. A table with nothing installed answers 200 with "installed": false and zeroes - not 404.
Understand the behavior
The trainer reads up to one million stored vectors, runs mini-batch k-means to max_iterations or until the maximum centroid shift falls under convergence_threshold, and writes the matrix in a single batch. Steps before that write touch nothing, so a failure leaves the previous partitioning in place.
Installing is a full replace, and a replace re-draws the cell boundaries. That is why the install then purges the old cell postings and streams the stored corpus back through the new assignment, one page at a time - reading under a short shared lock, assigning with no lock held, appending under a short exclusive one. The walk is not capped: every stored vector gets a posting.
If you already have a centroid matrix from your own training run, POST /v1/tenants/:t/vector/:table/install-centroids takes it directly as {"centroids": [[...], [...]]} and derives partitions and dim from the shape of what you send.
Important limits and checks
- Training on too few vectors. The engine refuses with 400 when the stored count is under
partitions × 4. Past that floor k-means puts several centroids on the same point. Lowerpartitionsor ingest more before training. - Querying IVF before installing. A
"index": "ivf"query against a table with no centroids returns 503 with"error": "ivf_centroids_not_installed"and aninstall_urlpointing here. - Assuming it is quick. Training and the backfill both run synchronously inside the request. On a large corpus the call holds the connection for seconds - set a generous client timeout.
- Re-installing and expecting the old postings to survive. They do not, and should not: new centroids mean new cells. The purge and re-populate is the point, and re-running the route converges to the same end state.
- Reading
centroids_previewas the full matrix. It contains up to four centroids and up to eight dimensions from each. Smaller matrices produce a smaller preview.