Prepare data for vector search
Keep embeddings consistent and connect matches back to their source records.
Use the same embedding model for saved content and the question you search with.
- Keep the source recordGive the content a stable ID.
- Generate and write its embeddingSave the complete vector under the matching ID.
- Query with a compatible vectorGenerate the query representation with the same model.
- Read the matching sourceUse returned IDs to retrieve the original content.
Prepare the collection
- Choose the embedding model your application will use.
- Create embeddings with that model before sending them to the database.
- Give each embedding the same ID as its source record.
- Use the same model and vector length for indexing and searching.
Saving a normal row does not create an embedding or add it to the vector index. Write vector values through the vector endpoints.
Example schema with vector settings
# The row schema. Vectors do NOT live in a column - they sit in their
# own keyspace, addressed by the same id. Registering the table is what
# lets you read the row back with SQL after a vector hit gives you an id.
namespace = "shop"
table = "orders"
primary_key = ["id"]
[[columns]]
name = "id"
ty = "str"
required = true
[[columns]]
name = "customer"
ty = "str"
[[columns]]
name = "amount_cents"
ty = "i64"
[[columns]]
name = "status"
ty = "str"
[[columns]]
name = "notes"
ty = "str"
[[columns]]
name = "placed_ms"
ty = "u64"
# Optional. Declares the dimensionality the collection expects so a
# wrong-sized vector is refused with a readable message instead of a
# generic mismatch. `distance` accepts cosine | l2 | dot | manhattan | l1,
# defaults to cosine, and resolves the metric queries actually run under.
[vector]
dim = 768
distance = "cosine"The [vector] block describes the collection. dim is the number of values in each vector; distance selects how similarity is measured. A schema may also declare vector metadata through columns.params.vector_dim; that does not make ordinary row writes populate the search index.
Choose the distance measure
Use the distance measure recommended by your embedding model.
| Setting | What it compares |
|---|---|
cosine | Direction, rather than vector length. The default. |
dot | Inner product. Normalize vectors when the model requires it. |
l2 | Euclidean distance. Use this spelling, not euclidean. |
manhattan or l1 | Sum of the absolute differences. |
An unknown measure returns 400. If a non-default measure is declared, a conflicting request is also rejected. An omitted request measure uses the collection setting. A declared cosine does not reject another request measure in the current contract, so keep your application's settings consistent.
Check vector length
Every stored and query vector in a collection must have the same positive dimension. For example, a 512-value vector does not fit a collection configured for 768 values.
Example dimension error
vector has 512 dims but collection "shop.orders" expects 768.
Use the same model, or re-index the collection.Plan a model change
- Changing models can change both dimension and meaning. Rebuild the collection with the new embeddings.
- Larger vectors use more memory and calculation work. Test quality and latency on representative data.
- Keep the source record so a returned ID can be opened and shown to the user.