A colleague asks, “How do I get money back after a work trip?” The handbook calls the process “expense reimbursement.” Those phrases share an intention without sharing all their words. Semantic search uses embeddings—numeric representations produced by a model—to find content that may be related in meaning.
The difficult part is not merely storing a vector. You need a useful unit of text, a dependable link to the original document, and a way to measure whether the returned material answers the question. This guide prepares a small handbook collection for OriginChainDB’s vector workflow.
1. Pick one task and a small source set
Start with three fictional policies: travel expenses, equipment requests and annual leave. Use approved documents that your test users are allowed to read. Record an ID, title, revision and source location for each. Do not mix private employee records into the exercise.
Write five test questions before generating embeddings. Include “How do I claim a train ticket?” and “Where can I request a keyboard?” Add one question whose answer is absent. That last case is essential: the nearest available passage may still be irrelevant.
2. Split by meaning before splitting by size
A chunk is the piece of text you retrieve. Divide the travel policy into sections such as eligible costs, submission steps and deadlines. Keep a heading with the paragraph it describes. If a paragraph says “submit it within this period,” preserve enough surrounding text to explain what “it” and “this period” mean.
Store this relationship in your working dataset:
| chunk_id | document_id | revision | section |
|---|---|---|---|
| travel-r2-costs | travel | r2 | Eligible costs |
| travel-r2-steps | travel | r2 | Submission steps |
| equipment-r1-request | equipment | r1 | Request process |
These are illustrative IDs, not a universal chunking standard. Very small pieces may lose context; very large pieces can bury the answer. Evaluate both against your actual questions rather than choosing a size solely because another project uses it.
3. Keep the embedding contract explicit
Choose an embedding model and record its identifier, output dimension, preprocessing and any documented query/document instructions. Generate document and query vectors according to the same model’s requirements. Two different models can produce vectors with equal length but incompatible meanings.
Do not pad, truncate or combine vectors to make an arbitrary dimension fit unless the model explicitly supports that operation. Configure the OriginChainDB collection to match the actual output using the vector schema reference. A dimension setting is a compatibility requirement, not a promise of better answers.
4. Store the text and vector with the same ID
Create a row schema for chunk metadata and text. Use chunk_id as its primary key. Keep the full source and revision available so a result can be explained to a person. The documented vector write endpoint is:
POST /v1/tenants/:tenant/vector/:table/putIt accepts an id, dim and the complete embedding array. Use the same ID as the source row. Writing an embedding into an ordinary row body is not a substitute for the vector indexing operation. Follow the vector quickstart for the exact request shape and collection configuration.
Treat row storage and vector writes as separate steps unless you deliberately use a documented atomic operation covering both. Record which chunks have completed indexing so an interrupted import can resume without silently leaving holes.
5. Search, resolve and score the result yourself
Embed each test question, then call the documented POST /v1/tenants/:tenant/vector/:table/topk endpoint with the query vector, dimension, supported metric and a small k. Resolve every returned ID to the matching source row. Read the text: did the result contain the reimbursement steps, or merely mention travel?
Make a test sheet with question, expected document, returned IDs, usefulness and notes. Keep the model and index configuration alongside the results. A score from one model or metric should not be compared numerically with a score from an unrelated setup.
6. Plan for policy changes
When a policy changes, regenerate the affected chunk vectors, preserve revision history, and exclude superseded content from user-facing retrieval through a verified application/index lifecycle. Confirm that an old deadline cannot reappear as the current answer. If filters remove candidates, check how that affects result count and retrieval quality.
You now have a repeatable semantic-search experiment: source text, clear chunks, a consistent model, matching IDs and a small evaluation set. For the console walkthrough, watch Build Semantic Search. Use its mechanics with your own verified handbook examples rather than expecting a model to repair missing or outdated source information.
Create the source collection before generating a real embedding
A source schema and a vector collection serve different purposes even when they share a table label. The row stores inspectable text and provenance; the vector write makes an embedding searchable. A compact manifest for a real model that emits 768 dimensions could be:
namespace = "handbook_demo"
table = "chunks"
primary_key = ["id"]
[[columns]]
name = "id"
ty = "str"
required = true
[[columns]]
name = "document_id"
ty = "str"
[[columns]]
name = "revision"
ty = "str"
[[columns]]
name = "body"
ty = "str"
[vector]
dim = 768
distance = "cosine"The number 768 is an example contract, not a recommended dimension for every model. Replace it with the actual supported output of the model you select before registering a fresh collection. The vector schema reference explains this separation and its dimension setting. Do not store a three-number teaching vector in a 768-dimensional collection or present a made-up vector as a real semantic embedding.
For each chunk, first inspect the source text. Then generate the complete embedding with the chosen model and write it with the same ID through the documented vector endpoint. Store the document identity and revision in the vector metadata as well if you intend to filter on those fields. The vector reference states that metadata filtering reads the vector’s stored metadata, not arbitrary columns from its source row.
Decide whether the retrieved passage really answers the question
Consider two travel passages. One lists eligible costs; another gives submission steps. A question about claiming a train ticket might need both. Returning the word “travel” somewhere in the top results is not enough. Label the specific expected passage IDs and note whether each contains all necessary conditions.
Use a small review table with five columns: question, expected passage, returned passage, missing context and action. For “Can I claim a family member’s ticket?”, the correct result may be that the small source set does not answer the question. Record that explicitly instead of manufacturing a rule. Human review of a few realistic queries often reveals source gaps before any model change is justified.
If chunks lose a deadline, eligibility condition or exception, repair the chunk boundary or source text. If the evidence exists but is never retrieved, check whether it was indexed under the intended ID and model. If the correct passage is returned but the UI shows the wrong text, investigate source resolution. These are different failure classes and should not all be labeled “bad embeddings.”
Rehearse one policy revision end to end
Create a fictional r3 of the travel steps with a changed submission deadline. Keep the historical r2 source for audit according to your retention policy, but make the current retrieval behavior explicit. Regenerate r3’s vectors using the same embedding contract or move a model migration into its own separately evaluated collection.
Update the application-visible current revision only through a tested process. For example, prepare and verify the new retrieval set before switching an application-level revision pointer. That is an integration design you must implement; the article does not claim that registering a schema automatically provides a publish workflow. After the switch, ask the deadline question and verify both the result ID and the text rendered to the user.
Finally, test a failed embedding call, a wrong vector length, an unavailable source row and a user without permission. Keep failed chunks recoverable and excluded from claims of a completed import. The useful deliverable is a source-to-vector pipeline whose IDs, versions and answers can all be inspected, plus a small evaluation record that explains why the chosen chunking works for this handbook.