9. Train and install IVF centroids

what this does

POST /v1/tenants/:t/vector/:table/train-and-install-centroids reads the vectors already stored in the table, runs mini-batch k-means over them, and installs the resulting centroid matrix as the table's IVF partitioning - in one call.

It then backfills a cell posting for every stored vector, so a table that was written before the install becomes queryable without a re-ingest. GET /v1/tenants/:t/vector/:table/centroids reads back what is installed.

when to use it

• You want to query a table with "index": "ivf". Without installed centroids that query returns 503 - IVF has no partitioning to probe.
• The corpus has grown or shifted since the last install and the cells no longer match the data.
• You are moving a table off the default HNSW path onto IVF for a corpus large enough that a cell probe beats a graph walk.

the request

curl -X POST "https://$OC_HOST/v1/tenants/$OC_TENANT/vector/shop.products/train-and-install-centroids" \
  -H "Authorization: Bearer $OC_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "partitions":            1024,
    "init":                  "kmeans_plus_plus",
    "max_iterations":        50,
    "batch_size":            1024,
    "convergence_threshold": 1e-5,
    "seed":                  42
  }'

result = db.vector.train_and_install_centroids(
    "shop.products",
    partitions=1024,
    init="kmeans_plus_plus",
    max_iterations=50,
    seed=42,
)
print(result.installed, result.iterations, result.converged)

// No typed SDK helper for this route yet - call it directly.
const res = await fetch(
  `${OC_HOST}/v1/tenants/${OC_TENANT}/vector/shop.products/train-and-install-centroids`,
  {
    method: "POST",
    headers: {
      Authorization: `Bearer ${OC_TOKEN}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify({
      partitions:     1024,
      init:           "kmeans_plus_plus",
      max_iterations: 50,
      seed:           42,
    }),
  },
);
const out = await res.json();
console.log(out.installed, out.populated);

// No typed SDK helper for this route yet - call it directly.
payload, _ := json.Marshal(map[string]any{
    "partitions":     1024,
    "init":           "kmeans_plus_plus",
    "max_iterations": 50,
    "seed":           42,
})
url := fmt.Sprintf("%s/v1/tenants/%s/vector/shop.products/train-and-install-centroids", host, tenant)
req, _ := http.NewRequestWithContext(ctx, http.MethodPost, url, bytes.NewReader(payload))
req.Header.Set("Authorization", "Bearer "+token)
req.Header.Set("Content-Type", "application/json")
resp, err := http.DefaultClient.Do(req)

what you get back

{
  "trained":              true,
  "installed":            true,
  "partitions":           1024,
  "dim":                  768,
  "iterations":           37,
  "converged":            true,
  "last_max_shift":       0.0000073,
  "training_corpus_size": 100000,
  "populated":            100000
}

// populate_skipped is present only when the backfill did not run.

populated is the number of stored vectors that got a cell posting from this call. When it is well below training_corpus_size, or a populate_skipped string is present, the index is installed but not fully covering the corpus - the coverage report will show it.

request fields

reading back what is installed

curl "https://$OC_HOST/v1/tenants/$OC_TENANT/vector/shop.products/centroids" \
  -H "Authorization: Bearer $OC_TOKEN"

preview = db.vector.centroids("shop.products")
print(preview.installed, preview.partitions, preview.dim)

// No typed SDK helper for this route yet - call it directly.
const res = await fetch(
  `${OC_HOST}/v1/tenants/${OC_TENANT}/vector/shop.products/centroids`,
  {
    method: "GET",
    headers: {
      Authorization: `Bearer ${OC_TOKEN}`,
    },
  },
);
const out = await res.json();
console.log(out.installed, out.partitions, out.dim);

// No typed SDK helper for this route yet - call it directly.
url := fmt.Sprintf("%s/v1/tenants/%s/vector/shop.products/centroids", host, tenant)
req, _ := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
req.Header.Set("Authorization", "Bearer "+token)
resp, err := http.DefaultClient.Do(req)

what the read-back returns

{
  "installed":  true,
  "partitions": 1024,
  "dim":        768,
  "centroids_preview": [
    [0.0124, -0.0883, 0.0451, 0.0037, -0.0192, 0.0608, -0.0271, 0.0115]
    /* ... first 4 centroids, first 8 dims of each ... */
  ]
}

The preview is deliberately truncated to the first 4 centroids and the first 8 dimensions of each, so the response stays small whatever partitions × dim is. It is an "is anything installed" check, not a way to export the matrix. A table with nothing installed answers 200 with "installed": false and zeroes - not 404.

how it works

The trainer reads up to one million stored vectors, runs mini-batch k-means to max_iterations or until the maximum centroid shift falls under convergence_threshold, and writes the matrix in a single batch. Steps before that write touch nothing, so a failure leaves the previous partitioning in place.

Installing is a full replace, and a replace re-draws the cell boundaries. That is why the install then purges the old cell postings and streams the stored corpus back through the new assignment, one page at a time - reading under a short shared lock, assigning with no lock held, appending under a short exclusive one. The walk is not capped: every stored vector gets a posting.

If you already have a centroid matrix from your own training run, POST /v1/tenants/:t/vector/:table/install-centroids takes it directly as {"centroids": [[...], [...]]} and derives partitions and dim from the shape of what you send.