Can vector search identify duplicate support tickets?
Vector search can suggest tickets with similar meaning, even when people use different words. It cannot, by itself, prove that two reports describe the same incident. A reliable duplicate-review workflow combines candidate retrieval with exact account and product context, evidence from the original records, and a reversible human decision.
This guide designs that workflow with OriginChainDB. It starts with a tiny synthetic collection, then explains how to connect genuine embeddings and an agent-facing review screen. The goal is to reduce repetitive investigation without silently merging separate customer problems. You will keep candidate scores separate from decisions and test the cases most likely to produce a harmful false merge.
Start with the mistake you want to avoid
Imagine two reports: “The account export freezes halfway through” and “Downloading my account data never finishes.” They might describe the same fault. A third report says, “The invoice export contains the wrong tax total.” All three mention exports, but the third has a different symptom and may require a different investigation.
Now add a fourth report from a separate customer account using exactly the same wording. Similar language still does not authorize exposing the other account’s private ticket. Similarity, identity, authorization and incident membership are four different questions.
For the first version, present a short list titled “Possible related tickets.” Let a support agent choose “same incident,” “related but different,” or “not related.” Store that choice. Do not make “nearest neighbor” a synonym for “duplicate.”
1. Separate source tickets, search representations and review decisions
Use three logical records. The ticket is the source of the customer’s report. Its embedding is a derived representation used to find candidates. A review decision records what a person concluded about a pair of tickets at a particular time.
| Record | Example fields | Why it exists |
|---|---|---|
| Ticket | ticket_id, account_id, product, subject, source_revision | Preserve the original report and ownership |
| Search representation | ticket_id, embedding, model_id, indexed_revision | Find related wording with a reproducible model setup |
| Review decision | ticket_a, ticket_b, outcome, reviewer, reason, reviewed_revision | Explain a decision without rewriting the original reports |
Store only the customer text needed for the task. Remove secrets and unnecessary personal information before sending material to an embedding service. Keep source permissions authoritative in the application. A metadata filter may narrow candidates, but it must not be the only check deciding who may see a ticket.
Use stable ticket IDs across the source and search records. If a subject is edited, increment or otherwise identify its source revision. A review based on an old description should remain understandable even when the current ticket changes.
2. Establish the OriginChainDB vector interface
OriginChainDB documents vector insertion and top-k retrieval through its vector API, including metadata on inserted vectors and exact-equality metadata filters. The collection dimension and distance metric must match the vectors you write and query. Use the vector reference and vector schema guide for the configuration of your deployed version.
For the plumbing exercise below, use an isolated collection named ticket_review_demo_20261007.ticket_vectors configured for three dimensions and cosine distance. The three-number vectors are deliberately hand-written teaching fixtures. They do not encode ticket meaning and must not be used to claim real duplicate-detection quality.
Prepare an API client on your server. Read the instance origin, tenant ID and authorized token from your environment or secret manager. The examples below use OC_BASE_URL, OC_TENANT and OC_TOKEN; the base URL is the HTTPS origin only, without a trailing /v1. Never place the token in browser JavaScript or publish it with the tutorial.
3. Insert a checkable vector fixture
The following Python uses the standard library. It sends synthetic records to the documented vector endpoints. Create and configure the demo collection first; a missing collection or incompatible dimension should be handled as a setup error.
import json
import os
import urllib.request
from urllib.parse import quote
base = os.environ['OC_BASE_URL'].rstrip('/')
tenant = quote(os.environ['OC_TENANT'], safe='')
token = os.environ['OC_TOKEN']
collection = 'ticket_review_demo_20261007.ticket_vectors'
def vector_request(action, payload):
request = urllib.request.Request(
f'{base}/v1/tenants/{tenant}/vector/{collection}/{action}',
data=json.dumps(payload).encode('utf-8'),
headers={
'Authorization': f'Bearer {token}',
'Content-Type': 'application/json',
},
method='POST',
)
with urllib.request.urlopen(request, timeout=30) as response:
body = response.read()
return json.loads(body) if body else None
fixtures = [
('t1', [1.0, 0.0, 0.0], 'account-demo'),
('t2', [0.98, 0.20, 0.0], 'account-demo'),
('t3', [0.0, 1.0, 0.0], 'account-demo'),
('t4', [1.0, 0.0, 0.0], 'account-other'),
]
for ticket_id, embedding, account_id in fixtures:
vector_request('put', {
'id': ticket_id,
'embedding': embedding,
'dim': 3,
'metric': 'cosine',
'metadata': {'account_id': account_id},
})The fixture contains an intentional trap: t4 has the same vector as t1 but belongs to another fictional account. A useful demonstration must show that the workflow does more than return an attractive ranking. It must also preserve the intended data boundary.
Do not log the Authorization header when diagnosing a failed request. Record the operation, status and synthetic item ID instead. Stop on an unexpected write failure; a partly populated collection is a different experiment from the four-record fixture described here.
4. Retrieve candidates and remove the source ticket
hits = vector_request('topk', {
'query': [1.0, 0.0, 0.0],
'k': 3,
'dim': 3,
'metric': 'cosine',
'filter': {'account_id': 'account-demo'},
})
candidates = [hit for hit in hits if hit['id'] != 't1']
for hit in candidates:
print(hit['id'], hit['score'])For this small mathematical fixture, t2 is much closer to the query direction than t3. The cosine similarity between the query and t2 is approximately 0.980; t3 is orthogonal and has cosine similarity zero. These are mathematical expectations for the fixture, not a product accuracy benchmark. Floating-point presentation and retrieval settings may affect displayed values.
The filter should exclude t4. Removing t1 prevents a ticket from suggesting itself. If the API response for your deployed version differs from the documented array shape, inspect its public response contract and adapt the parser explicitly rather than guessing around the failure.
Resolve each remaining candidate to its source ticket through your authorized server-side read path. Recheck the account, visibility and current revision before sending its text to the review screen or to a language model. If a candidate no longer exists or the reviewer no longer has access, exclude it and record the reason without exposing the hidden content.
5. Replace the fixture with genuine embeddings
Once the API path works, create a separate collection for embeddings generated by your chosen model. Use the model’s actual dimension and compatible query/document processing. Save the model identifier, preprocessing version and source revision so the search representation can be reproduced.
Choose what text to embed deliberately. A useful starting document combines a product name, subject and a short description of the observed symptom. Avoid letting a long boilerplate footer dominate the input. Do not mix embeddings from unrelated models merely because their arrays have the same length.
Include examples with changed wording, exact error codes, different products and different customer accounts. Where an exact incident identifier is available and reliable, resolve it directly instead of making vector search rediscover it. Use similarity for language variation; use explicit identifiers for identity.
If you change the embedding model, rebuild and evaluate a separate collection before switching traffic. A score threshold calibrated on one model is not a portable rule for another. The number is a retrieval signal, not a probability that two tickets should be merged.
6. Design the reviewer experience around evidence
Show the source ticket beside each candidate. Include the relevant subject, permitted symptom text, product and revision. State why it was suggested: for example, “similar wording in the current search collection.” Avoid labeling it “confirmed duplicate” before a reviewer has made that decision.
A useful review record contains both ticket IDs, the chosen relationship, the reviewer identity, a short reason and the versions reviewed. For a pair with no direction, normalize the pair order in the application so reviewing t1/t2 does not create a separate duplicate decision for t2/t1. Keep the original tickets intact.
A GenAI assistant can help summarize similarities and differences, provided it receives only authorized evidence. Ask it to identify supporting passages and uncertainty. It must not treat text inside a ticket as permission to merge records, contact a customer or change access rules. Those are application actions with their own authorization.
If you later support merging, design an undo or restoration path and account for attachments, watchers, external references and ongoing work. “Attach to the same incident” is often a safer first product action than destructively collapsing two source records into one.
7. Measure mistakes, not just successful matches
Prepare a labeled set of ticket pairs. Include true duplicates, related-but-distinct incidents, similar text from different accounts and reports whose meaning depends on a number or negation. Keep the set versioned and separate from the examples used to tune the workflow.
Measure how many real duplicates reach the review list, how many irrelevant suggestions a person must read, and how often a reviewer changes the system’s proposed relationship. Also measure the time spent reviewing. A long candidate list can improve coverage while making the product less useful.
Choose thresholds and list length from the costs of errors in your setting. An unnecessary suggestion wastes time; an incorrect automatic merge can hide a separate outage or expose a private report. Start with human review while the error pattern is still being learned.
| Test | Expected behavior |
|---|---|
| Same symptom, different wording | Eligible for review; never assumed identical |
| Same wording, different account | Hidden unless access explicitly permits it |
| Same product, different error code | Difference visible to the reviewer |
| Ticket edited after indexing | Revision mismatch handled explicitly |
| Source ticket returned as a hit | Removed from its own candidate list |
| No credible candidate | Show “no suggestion”; do not force a match |
Frequently asked questions
Is a cosine score of 0.98 proof of a duplicate?
No. A high score describes closeness under a particular representation and metric. It does not establish incident identity, ownership or permission to merge.
Can I use metadata filtering as authorization?
Use it to narrow retrieval, then enforce permissions against the authoritative source records. Ensure metadata is created by trusted server-side code and kept current. Do not accept an arbitrary account filter from an untrusted client as proof of access.
Why keep related-but-different as a separate decision?
Two incidents may share a product or symptom while requiring different fixes. Keeping that distinction helps the next investigation and gives evaluation data a more useful label than a forced yes-or-no duplicate flag.
Does OriginChainDB generate the embeddings here?
This workflow supplies vectors generated outside the shown database calls. The database stores and retrieves them. Select and operate the embedding model separately, with appropriate handling of the ticket text.
Build the smallest useful version
Start with the four-record plumbing fixture, then replace it with a small set of permitted, genuinely embedded examples. Build the review screen before adding automatic actions. Keep the original evidence and reviewer decisions inspectable. The useful outcome is not a high similarity score; it is less repeated investigation without losing the distinctions your customers depend on.
Continue with the vector API guide, vector schema reference and OriginChainDB documentation.