10. IVF coverage and cell skew

what this does

GET /v1/tenants/:t/vector/:table/ivf-rebalance-status reports the shape of an IVF index: how many vectors sit in each cell, how lopsided that distribution is, how many stored vectors have a cell posting at all, and whether the engine thinks you should retrain.

It is read-only. Nothing is rebalanced as a side effect - the recommendation is a recommendation.

when to use it

• Your IVF queries return fewer or worse hits than you expect, and you need to know whether the index actually covers the data.
• After a bulk ingest, to confirm the new vectors were assigned to cells rather than merely stored.
• On a schedule, to catch cell skew building up as the corpus drifts away from the centroids you trained.

the request

curl "https://$OC_HOST/v1/tenants/$OC_TENANT/vector/shop.products/ivf-rebalance-status" \
  -H "Authorization: Bearer $OC_TOKEN"

# No typed SDK helper for this route yet - call it directly.
import httpx

r = httpx.get(
    f"{OC_HOST}/v1/tenants/{OC_TENANT}/vector/shop.products/ivf-rebalance-status",
    headers={"Authorization": f"Bearer {OC_TOKEN}"},
)
r.raise_for_status()
status = r.json()
print(status["coverage"], status["skew"], status["action"])

// No typed SDK helper for this route yet - call it directly.
const res = await fetch(
  `${OC_HOST}/v1/tenants/${OC_TENANT}/vector/shop.products/ivf-rebalance-status`,
  {
    method: "GET",
    headers: {
      Authorization: `Bearer ${OC_TOKEN}`,
    },
  },
);
const status = await res.json();
console.log(status.coverage, status.skew, status.action);

// No typed SDK helper for this route yet - call it directly.
url := fmt.Sprintf("%s/v1/tenants/%s/vector/shop.products/ivf-rebalance-status", host, tenant)
req, _ := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
req.Header.Set("Authorization", "Bearer "+token)
resp, err := http.DefaultClient.Do(req)

what you get back

{
  "total_live":     100000,
  "partitions":     1024,
  "live_per_cell":  [112, 96, 143, 87 /* ... one entry per cell ... */],
  "skew":           1.46,
  "action":         "none",
  "stored_vectors": 100000,
  "unindexed":      0,
  "coverage":       1.0
}

action is one of "none", "recommended" or "required", set from skew against the engine's thresholds.

response fields

how it works

The report walks each cell's posting list and counts entries; it does not decode the payloads behind them. That makes it cheap enough to poll, and it means "live" is a posting count rather than a verified row count - an index with stale postings from an interrupted re-install can over-report total_live. Re-training and re-installing is the cleanup, not a per-call scan.

stored_vectors, unindexed and coverage exist for one failure shape that every other field hides: a table with ten thousand vectors stored and zero of them indexed. Nothing else on this response raises a flag there - every cell is empty, so skew is 0.0 and action is "none" - while every IVF query against that table returns nothing.

Skew matters for latency rather than correctness. A query probes a fixed number of cells; if one cell holds ten times its share, the queries that land on it do ten times the work. That is what the action hint is tracking.