Vector search

Vector search finds records whose numeric representations are close to a query vector. An embedding model creates those numbers from text or other content. Store and query vectors from the same model, with the same number of values and distance rule.

Start with the three-number quickstart, then choose the metric, index and filters your application needs. The vector-table and vector-record APIs are separate interfaces.

Recall means the share of relevant matches a search finds. Approximate indexes trade some recall for less search work. The language examples below use earlier client interfaces; check the SDK directory for your installed version. Arrays marked with ... or comments are shortened for display and must be replaced with complete numeric vectors. The three-number quickstart is ready to adapt.

1. Top-k search.Copy Anchor Link

Given a query embedding, return the top k rows whose stored embeddings are closest to it.

curl -X POST "https://$OC_HOST/v1/tenants/$OC_TENANT/vector/shop.products/topk" \
  -H "Authorization: Bearer $OC_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "query":  [0.011, -0.082, 0.046, /* ... 768 floats ... */],
    "k":      10,
    "dim":    768,
    "metric": "cosine"
  }'

The HTTP result is an array of { id, score } objects in ranked order. id is the vector ID supplied when writing. If you saved a source row under the same ID, use it to read that row separately; writing a vector does not create the row.

[
  { "id": "sku-9281", "score": 0.92 },
  { "id": "sku-4017", "score": 0.88 },
  { "id": "sku-3320", "score": 0.85 },
  ...
]

Results are ordered by score, highest first. Cosine and dot return similarity scores. L2 and Manhattan return negative distances, so a score closer to zero is a closer match. You do not need to reverse the result order for L2.

• Wrong dim. If your query vector is 1536 floats but the table was set up for 768, you get a 400 reading vector has 1536 dims but collection "shop.orders" expects 768. There is no dim_mismatch code - the body carries the message, not a token.
• Wrong metric. Query with a different metric than the table was written under and you get a 409 vector_metric_mismatch, naming both metrics and a rebuild URL. If the collection declares a non-default [vector].distance, contradicting it is refused earlier, with a 400.
• Empty results. Confirm that the vector write succeeded, the query uses the same collection and index, and its metadata filter matches. The default HNSW write updates the vector and its graph together. A short result list can also reflect filtering, access rules or approximate search; it is not a count of every eligible record.

2. Filter by metadata.Copy Anchor Link

Restrict the search to vectors whose metadata matches a filter. Useful for things like "find similar products but only in the shoes category".

curl -X POST "https://$OC_HOST/v1/tenants/$OC_TENANT/vector/shop.products/topk" \
  -H "Authorization: Bearer $OC_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "query":  [/* 768 floats */],
    "k":      10,
    "dim":    768,
    "metric": "cosine",
    "filter": { "category": "running-shoes" }
  }'

Filters compare the metadata saved with each vector. The search strategy depends on the index; a selective filter may require more candidate checks or a wider search. Measure latency and result quality with the filters your application will use.

• Filtering on a field you didn't store. The filter looks at the metadata object you passed at insert time - not at the row's other columns. If you want to filter on a column, include it in metadata.
• Range and typed conditions. The simple filter map supports equality. The separate filter_expr field supports typed conditions on eligible deployments. Do not combine it with a non-empty filter; check the endpoint contract. Filtering a short result list in your application does not find candidates the search omitted.
• Fewer than k hits. The request can return fewer matches because of the available data, filter, access rules or search limits. Check the collection and metadata before treating the returned count as exhaustive.

3. Distance metric.Copy Anchor Link

Picks the math used to compare two vectors. You send the metric on each request - on the write and on the search - and the two have to use the same one. A collection can pin it once with [vector].distance on the schema, which then fills in an omitted metric and refuses one that contradicts it.

Use the metric recommended by your embedding model. The query and stored vectors must use a compatible metric and dimension.

4. Speed vs recall.Copy Anchor Link

Approximate nearest-neighbor search examines candidates rather than comparing every vector. The mode field controls how widely HNSW searches. Wider searches may find more relevant results but take longer.

# Add "mode" to the topk body. "high_recall" (default) or "fast".
curl -X POST "https://$OC_HOST/v1/tenants/$OC_TENANT/vector/shop.products/topk" \
  -H "Authorization: Bearer $OC_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "query":  [/* 768 floats */],
    "k":      10,
    "dim":    768,
    "metric": "cosine",
    "mode":   "fast"
  }'

"Recall" measures how many of the true nearest neighbors the search found. Both modes sort the returned candidates by score; neither guarantees that it found every true top-k neighbor. Compare them against known relevant matches or an exact baseline for your data.

5. Index choice.Copy Anchor Link

OriginChainDB supports several different index types for vector data. Most users should stick with the default (HNSW). The other types are for very large or memory-constrained corpora.

The index is picked per request, not on the schema - index is a field on both the write and the query body, and the manifest has no index key. See Index kinds for the accepted values.

6. Examples.Copy Anchor Link

Every operation below is shown in cURL, Python, TypeScript and Go. Where an SDK does not wrap an endpoint the tab says so and shows the raw call instead.

Insert vectors.Copy Anchor Link

id, embedding and dim are required. metadata is a free-form JSON object stored alongside the vector — it is what filtering reads later, so put anything you may want to narrow on in there at write time.

A successful write to the default HNSW index saves the embedding and graph update together; it does not require a separate index-build step. This does not combine a row write or full-text write into the same transaction. For replica and recovery behavior, read durability and recovery.

Single insert.

curl -X POST "https://$OC_HOST/v1/tenants/$OC_TENANT/vector/shop.orders/put" \
  -H "Authorization: Bearer $OC_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "id":        "01JTRX9KQ3YH8K2WMX0F5JZAB7",
    "embedding": [0.0124, -0.0883, 0.0451],
    "dim":       3,
    "metric":    "cosine",
    "metadata":  { "status": "paid", "customer": "01JTRX1H4Q9P0N2WMX0F5JZ001" }
  }'
# → 201 Created, empty body

Bulk insert.

Bulk is the right shape for ingest: it builds the graph in one pass and writes one log frame for the whole batch. dim, metric, quantization and index live on the envelope; each item carries only id, embedding and metadata. The body-size limit is lifted on this route only.

curl -X POST "https://$OC_HOST/v1/tenants/$OC_TENANT/vector/shop.orders/put_bulk" \
  -H "Authorization: Bearer $OC_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "dim":    768,
    "metric": "cosine",
    "vectors": [
      { "id": "01JTRX9KQ3YH8K2WMX0F5JZAB7", "embedding": [/* 768 floats */],
        "metadata": { "status": "paid" } },
      { "id": "01JTRX9KQ3YH8K2WMX0F5JZAB8", "embedding": [/* 768 floats */],
        "metadata": { "status": "refunded" } }
    ]
  }'

{
  "inserted":   2,
  "elapsed_ms": 14
}

• At most 100,000 vectors per call — over that the engine answers 413, not 400.
• An empty vectors array is a 200 with inserted: 0 and no write.
• Duplicate ids inside one batch are last-writer-wins. The whole batch is one atomic write.
• Single insert returns 201 Created with an empty body — don't try to parse it.

Sparse vectors.

Learned-sparse embeddings go to POST /vector/:table/put_sparse with { id, indices, values, dim, metadata? }, and are queried with POST /vector/:table/topk_sparse. indices and values must be the same length, every index must be inside dim, and non-finite values are refused. No SDK wraps the sparse endpoints today.

Query — top-k.Copy Anchor Link

query, k and dim are required; everything else has a default. The field is k — there is no top_k alias.

curl -X POST "https://$OC_HOST/v1/tenants/$OC_TENANT/vector/shop.orders/topk" \
  -H "Authorization: Bearer $OC_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "query":  [0.0124, -0.0883, 0.0451],
    "k":      5,
    "dim":    3,
    "metric": "cosine"
  }'

[
  { "id": "01JTRX9KQ3YH8K2WMX0F5JZAB7", "score": 0.9421 },
  { "id": "01JTRX9KQ3YH8K2WMX0F5JZAB8", "score": 0.9187 },
  { "id": "01JTRX9KQ3YH8K2WMX0F5JZAB9", "score": 0.8804 }
]

A bare JSON array, not an envelope — no hits wrapper and no total count. Sorted by score descending, and larger always means closer: for l2 and manhattan the distance is returned negated so the ordering convention holds across every metric.

k: 0 returns [] without touching storage. The ceiling on k is 4096 by default; above it you get a 400.

Query modes — fast vs high_recall.Copy Anchor Link

mode is the one recall/latency knob exposed on the API. It sets the search beam width and nothing else — the build-time graph parameters are unaffected.

curl -X POST "https://$OC_HOST/v1/tenants/$OC_TENANT/vector/shop.orders/topk" \
  -H "Authorization: Bearer $OC_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "query":  [/* 768 floats */],
    "k":      10,
    "dim":    768,
    "metric": "cosine",
    "mode":   "fast"
  }'

• Omitting mode gives you high_recall. That is the safe default and the right one for first-pass retrieval.
• The value is case-sensitive. An unknown value is a hard 400: unknown `mode` "FAST": expected one of "fast", "high_recall".
• mode applies to the default graph index only. On ivf and ivf_pq queries it is accepted and then ignored — those paths have no beam width. Use nprobe there instead.
• Measure recall and latency on your own corpus. A preset does not promise a particular retrieval percentage or response time.

Seeing what a query actually did.

POST /vector/:table/topk_explain returns { hits, config }. The configuration identifies the metric, dimension and selected index. HNSW includes beam settings; IVF and IVF-PQ include cell-probing settings. This endpoint runs a search, so it is not a free configuration lookup. The current endpoint follows the selected index and filter settings, and can report completeness for eligible filtered requests. See the explain endpoint for fields and deployment restrictions.

Metadata filtering.Copy Anchor Link

filter is a flat equality map against metadata saved with the vector. Every supplied key must match; this field does not accept query operators. For typed conditions, use the separate filter_expr contract where supported. Do not send both a non-empty filter and filter_expr.

For eligible filtered requests, require_complete: true returns 422 filtered_search_incomplete instead of an underfilled answer. This is not a guarantee of exact nearest-neighbor ranking. Typed filters and completeness options have caller and topology restrictions; review the top-k API.

curl -X POST "https://$OC_HOST/v1/tenants/$OC_TENANT/vector/shop.orders/topk" \
  -H "Authorization: Bearer $OC_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "query":  [/* 768 floats */],
    "k":      10,
    "dim":    768,
    "metric": "cosine",
    "filter": { "status": "paid" }
  }'

Values are compared with strict JSON equality, so 1 does not match "1" and true does not match "true". A record that is missing a key named in the filter is rejected, not treated as null.

A filtered query can return fewer than k hits.

On the default graph index, filtering is a post-filter: the engine searches for k × 4 candidates and then drops the ones that don't match. If your filter is highly selective, most of the over-fetched set is discarded and you get back fewer than k results — even when more matching vectors exist. Ask for a larger k than you need when you filter narrowly.

Filter is silently ignored on ivf_pq.

A filter sent with "index": "ivf_pq" is not applied, and the request still returns 200 with unfiltered results. This is the sharpest edge on this page: filter and IVF-PQ do not compose today. Filter on the default index, or apply the predicate yourself after the hits come back. On "index": "ivf" the filter is honoured, and applied inside each cell scan before scoring.

Index kinds.Copy Anchor Link

index appears on both the write and the query body, and the two must agree — a vector written under one index kind is not visible to a query using another. Omitting it everywhere is the common and correct choice.

Approximate, not exact.Copy Anchor Link

HNSW, IVF and IVF-PQ use approximate nearest-neighbor search. They can miss a true nearest neighbor. Increasing the search width or reranking returned candidates can improve results, but reranking cannot recover a candidate the search never returned. If your application requires exact results, verify that its chosen API and deployment explicitly support that requirement.

Building an IVF-PQ index from a preset.Copy Anchor Link

IVF-PQ has a lot of knobs — partition count, subspace count, code width. Rather than expose them, the engine ships exactly two named presets and derives every parameter from your corpus. preset is the only required field.

curl -X POST \
  "https://$OC_HOST/v1/tenants/$OC_TENANT/vector/shop.orders/create-ivf-pq-index" \
  -H "Authorization: Bearer $OC_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{ "preset": "compressed" }'

Response — the config it chose.

{
  "trained":              true,
  "installed":            true,
  "preset":               "compressed",
  "partitions":           1024,
  "pq_m":                 48,
  "pq_bits":              8,
  "keep_raw":             false,
  "dim":                  768,
  "training_corpus_size": 50000
}

The optional seed field (default 0) controls training reproducibility. The endpoint also accepts drop_raw_vectors, which defaults to false. Setting it to true with the compressed preset irreversibly discards the original vectors; retain a recoverable source before considering it. The preset selects partitions, subspace count and code width.

Build then query — no re-insert needed.

On supported configurations, this endpoint trains and populates IVF-PQ cells from stored vectors. Inspect the successful build receipt before querying with index: "ivf_pq"; an interrupted build is not a completed index. Raw vectors are retained by default, including with the compressed preset. Lower-level IVF installation has its own population and topology rules, described below.

How each preset resolves.Copy Anchor Link

Both presets pick the same partition count and 8-bit code width. They differ in how finely the vector is subdivided and whether queries use retained raw vectors for refinement. Raw-vector retention is a separate setting.

Because pq_m must divide dim and is capped at 64, the two presets converge at high dimensionality. Worked from the real formula:

At 1536 dimensions both presets reach the 64-subspace cap. That does not imply identical query behavior: high_recall permits refinement against retained raw vectors, while compressed uses the compressed representation.

Raw vectors and refinement.

keep_raw describes whether the query path can refine candidate scores using raw vectors. raw_vectors_retained describes whether those vectors remain stored. A compressed index retains them by default; only the explicit drop_raw_vectors: true option discards them. Read both fields in the build response.

Which to pick.Copy Anchor Link

• Start with the default index and measure its memory, latency and missed matches.
• Evaluate compressed when smaller query representations suit the workload. Do not count raw-vector storage as removed unless you explicitly discard it.
• Evaluate high_recall when refinement improves the returned candidates enough to justify its extra work. Refinement cannot recover candidates that the initial search missed.

Querying the built index.Copy Anchor Link

Pass "index": "ivf_pq" and, optionally, nprobe — the number of cells to visit.

curl -X POST "https://$OC_HOST/v1/tenants/$OC_TENANT/vector/shop.orders/topk" \
  -H "Authorization: Bearer $OC_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "query":  [/* 768 floats */],
    "k":      20,
    "dim":    768,
    "metric": "cosine",
    "index":  "ivf_pq",
    "nprobe": 16
  }'

• The default nprobe grows with the square root of the partition count, subject to minimum and maximum bounds. Inspect explain to see the resolved value. A larger value visits more cells and does more work.
• Valid range is 1 to 256. 0 is a 400 (nprobe must be >= 1); above 256 is a 400 naming the cap.
• nprobe is ignored on the default graph index — it only means something for ivf and ivf_pq.
• Check your installed SDK version for nprobe and refinement options. Use the HTTP endpoint directly when the client does not expose the field.

The minimum-corpus rule.Copy Anchor Link

Training refuses to run on a corpus too small to populate its partitions: you need at least four vectors per partition. Because the partition count is floored at 64, that means a practical floor of 256 vectors — and far more once 4 × √N pushes the partition count up.

{
  "error": "not enough vectors to build a Compressed index: 1024 partitions\n            need >=4*K = 4096 vectors, found 900"
}

Building on an empty table is a separate 400: no vectors stored for this table; put vectors before building an index. Note that the preset name is echoed in the error in its internal capitalised form (Compressed, HighRecall) rather than the wire form you sent.

The lower-level centroid endpoints.

Plain IVF uses separate training, installation and inspection endpoints. The worked training example explains the minimum corpus size and response fields.

Append the suffix to the collection's /vector/:table route. Training needs at least four stored vectors per requested partition. Installation replaces the existing centroid matrix. On supported single-writer paths it can populate stored-vector postings; replicated paths can report populate_skipped. Inspect populated and any skip reason before assuming that every stored vector is searchable through IVF.

Hybrid dense + sparse.

POST /vector/:table/topk_hybrid runs a dense and a sparse query together and fuses the two rankings server-side with Reciprocal Rank Fusion. Fields: dense_query, dense_dim, sparse_query_indices, sparse_query_values, sparse_dim, k, plus optional dense_metric, dense_mode, rrf_k (default 60), candidates and filter. The returned score is a fused rank score, not a distance — it is not comparable to the scores from a single-mode query. Check your SDK version for a hybrid helper, or call the HTTP endpoint directly.

Delete.Copy Anchor Link

# Single - idempotent, 200 with {"deleted": false} when the id is absent
curl -X DELETE \
  "https://$OC_HOST/v1/tenants/$OC_TENANT/vector/shop.orders/01JTRX9KQ3YH8K2WMX0F5JZAB7" \
  -H "Authorization: Bearer $OC_TOKEN"

# Bulk - at most 10,000 ids per call
curl -X POST "https://$OC_HOST/v1/tenants/$OC_TENANT/vector/shop.orders/delete-bulk" \
  -H "Authorization: Bearer $OC_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{ "ids": ["01JTRX...AB7", "01JTRX...AB8"] }'

• Single delete is idempotent: deleting an id that isn't there is a 200 with { "deleted": false }, never a 404.
• Bulk delete caps at 10,000 ids and returns { deleted_count, missing_count }.
• Both accept an optional index selector (hnsw by default) — it must match the index the vector was written under.
• Vector ids are capped at 1024 characters and may not be empty.

Limits and gotchas.Copy Anchor Link

Metric and index are per-request - and only metric is checked.

index is per-request and recorded nowhere. metric is per-request too, but it is checked: every HNSW graph is stamped with the metric it was built under, so writing with cosine and querying with l2 is refused with a 409 vector_metric_mismatch. An index built before that stamp shipped carries none and still serves the mismatch behind a 200, until the next write stamps it. Same for index: writing under the default graph index and then querying index: "ivf_pq" finds nothing at all, unless the preset build populated those cells. Pick one metric and one index kind per collection and hold them constant in your own code.

Training and populate sample at most one million vectors.

A preset build reads up to 1,000,000 vectors, both for training and for populating cells. On a collection larger than that, rows beyond the first million are not written into the IVF-PQ index by the build and will not be found by an ivf_pq query until they are written again under that index.

Back-pressure and quota.

A 429 response means the request was limited; follow Retry-After when supplied. A 402 can indicate an exhausted vector quota or a missing instance capability. Check your configuration and the response details. After a failed or interrupted bulk write, confirm which records were saved before retrying.

Quantization on the write path.

Separately from IVF-PQ, a write can carry quantization: "none" (default), "scalar", "binary" or "pq". Quantization reduces the vector payload at some cost in precision. Binary quantization supports cosine only; dot, L2 and Manhattan combinations are rejected with 422 quantization_metric_unsound. Measure complete index memory and retrieval quality before changing representation.

Where this lives in the dashboard.Copy Anchor Link

The current console guide includes a Vector input in Workbench and vector views in Data Explorer. Enter an embedding with the same dimension and metric as the stored vectors, then inspect the matching records. See Run queries in Workbench.

The console screenshots use local sample data. Live search and index-management controls depend on the instance and its reported capabilities; a visible preview control does not establish that the operation is enabled on your deployment.