You correct a help article, but search still returns its old wording. This is a data-flow problem: the source record and the searchable representation have separate update paths. A successful row update does not necessarily mean that every retrieval view is current.
OriginChainDB documents explicit write operations for full-text and vector indexes. This guide shows how to reason about their lifecycle using a small application-managed work queue. The queue and worker are design components you implement; they are not presented here as an automatically configured product feature.
1. Give the source a stable ID and revision
Imagine a delivery guide with ID delivery-guide. Revision 1 says “dispatch in five days”; revision 2 corrects it to “dispatch in three days.” Keep the document ID stable and change the revision. Store the latest source text with that revision so a worker can establish what it is supposed to index.
Decide which fields belong in search. Internal editorial notes should not accidentally enter the searchable body. If you concatenate title and body for FTS, record that transformation so the indexing process is repeatable.
2. Record unfinished work explicitly
After accepting revision 2, the application needs to index it. A robust design records a pending indexing task through a transaction or another recoverable protocol appropriate to the application’s write path. Without such a protocol, a crash between saving the document and recording the task can leave the change invisible to the worker.
A task can contain the document ID, desired revision, target modality, state, attempt count and last error category. Do not store credentials in it. The task key should identify the logical work, for example delivery-guide:r2:fts, so a retry can be recognized instead of becoming an unrelated new task.
3. Update FTS under the same document ID
For FTS, the documented operation is:
POST /v1/tenants/:tenant/fts/:table/:field
{"doc_id":"delivery-guide","text":"Dispatch in three days."}The real request also needs the normal authorization and JSON headers. Re-indexing the same doc_id replaces that document’s earlier searchable text. Keep the table and field labels identical to those used by the search request. After success, mark the specific revision complete in the worker’s bookkeeping.
This replacement behavior is useful for retrying a known FTS document write. It does not imply that every database operation is idempotent, or that replaying an unrelated payment or stock adjustment is safe.
4. Regenerate embeddings when the meaning-bearing text changes
A vector is derived from the source text and the embedding model. Editing the text does not rewrite the old vector automatically. Generate a new embedding using the collection’s established model and preprocessing, then write the full vector with the matching source ID through the documented vector-put path.
Record the model version and input revision alongside the indexing status. Do not let a worker silently switch embedding models while keeping old vectors in the same search space. Equal dimensions alone do not establish compatibility.
5. Prevent an older task from winning a race
Suppose the revision-1 worker is slow and finishes after revision 2. If it blindly writes its older text last, search moves backward. Coordinate workers per document, check the desired revision immediately before committing work, or use a supported version-aware design. The exact synchronization mechanism belongs to your application and should be tested under concurrency.
Use a deliberate test: pause an old task, complete a new task, then resume the old one. Verify that the chosen design preserves the newest intended searchable content. A happy-path demonstration cannot prove this property.
6. Verify behavior, not just acknowledgements
Search for a word unique to revision 1 and another unique to revision 2. The current source and search result should agree. For semantic search, use evaluation questions because an embedding update is not directly visible as a word count. Confirm that the retrieved ID resolves to the intended revision before showing a result.
Track pending-task age, repeated failures and unresolved source IDs. Retrying with bounded backoff helps temporary failures; a permanent schema or dimension error needs an explicit repair path. Preserve enough information to diagnose the issue without logging sensitive source content by default.
The result is a small, explainable ingestion process: one source identity, explicit derived writes, recoverable work and a check that the latest content is actually retrievable. Consult the FTS indexing guide and vector quickstart when implementing the endpoint calls.
Make progress observable with explicit states
Give an indexing job a small state machine that operators can understand. A proposed sequence is pending, running, verified and complete; a failed attempt returns to pending only when it is safe to retry, or enters a needs-review state when the input is invalid. Store a non-sensitive error category and the intended source revision. These are application records that you design, not an automatically created OriginChainDB queue.
For delivery-guide r2, a status view might look like this:
| Source | Target | Desired revision | Confirmed revision | State |
|---|---|---|---|---|
| delivery-guide | FTS body | r2 | r2 | verified |
| delivery-guide | vector collection | r2 | r1 | pending |
This view tells an operator that one retrieval path still needs work. A single “document saved” badge cannot communicate that distinction. The user-facing application should have a declared behavior for the transition: use a verified current path, withhold an unsupported result, or show an explicit updating state according to its requirements.
Test the race with an event table
A source revision comparison is useful only if the design prevents an older task from writing between the comparison and its commit. A check performed much earlier in the job is not sufficient. Use the synchronization or conditional-write mechanism actually supported by the chosen application path, and document its critical section.
Rehearse this sequence in a disposable fixture:
| Order | Event | Required observation |
|---|---|---|
| 1 | Worker A reads r1 | It records which source it read |
| 2 | Source advances to r2 | Desired revision becomes r2 |
| 3 | Worker B indexes and verifies r2 | Current search resolves to r2 |
| 4 | Worker A resumes | It cannot replace current search with r1 |
| 5 | Search runs again | r2 remains the current evidence |
If your implementation cannot guarantee that fourth step, do not declare the race solved. One alternative is versioned search identities plus an application-managed current-revision gate; another is serializing publication per source. Each has trade-offs in cleanup, query filtering and coordination. Choose one that your team can test and operate rather than assuming a revision field creates concurrency control by itself.
Reconcile missing and uncertain writes
A timeout can occur after a remote operation was accepted. Treat the outcome as uncertain, not automatically as rejected. For a replace-style indexing request, a retry may be appropriate when it carries the same intended content and revision, but confirm the endpoint’s semantics. Do not generalize that decision to incrementing counters or creating business events.
Periodically reconcile the desired source revision with completed indexing records. Select a bounded set of incomplete jobs, compare their intended source state and repair only the work still needed. If a source has been withdrawn, the next action is different from reindexing its old content. Deletion, permissions and retention are part of the lifecycle, not exceptions to ignore after the first successful import.
Keep credentials out of job payloads and logs. Workers should obtain normal service credentials through the deployment’s approved mechanism. A useful failure record identifies the source, revision, target modality and error category without copying sensitive source text into every log line.
Verify what the reader actually receives
After an FTS replacement, check the old unique token and the new unique token. After a vector replacement, use a labeled question and inspect the resolved passage because a semantic representation cannot be checked by token absence alone. In both cases, compare the displayed source revision with the intended one.
Monitor the age of unfinished work, oldest pending revision and count of unresolved hits. Choose alert thresholds based on the application’s promise about freshness. “Indexing eventually catches up” is not a usable promise unless someone knows how long eventually can be and how failures are repaired. The end result should be a source-to-search process with observable progress, recoverable failures and a demonstrated answer to the late-worker race.