9. Synonyms and stopwords

what this does

Two admin endpoints that configure how one FTS field treats words. A synonym map makes terms interchangeable, so a search for tv reaches a document that only ever said television. A stopword override replaces the built-in list of words the analyzer discards. Both are keyed by (schema, field) and both replace the stored record in full.

when to use it

• Vocabulary no stemmer will ever bridge: tv and television, a part number and its spelled-out name, an internal codename and the public product name.
• Recall complaints after launch. Query-time expansion reaches documents that were indexed long before the map existed, so you can fix a gap without reindexing.
• A corpus where the built-in stopword list drops a word that carries meaning in your domain - or keeps one that does not.

installing a synonym map

The body maps each canonical term to the words that should mean the same thing. Send the complete desired map every time - there is no merge.

curl -X POST "https://$OC_HOST/v1/tenants/$OC_TENANT/fts/shop.products/description/synonyms" \
  -H "Authorization: Bearer $OC_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "synonyms": {
      "television": ["tv", "telly"],
      "headphones": ["headset", "cans"]
    }
  }'

import requests

# The COMPLETE map for (schema, field) - this call replaces any previous one.
resp = requests.post(
    f"https://{OC_HOST}/v1/tenants/{OC_TENANT}"
    f"/fts/shop.products/description/synonyms",
    headers={"Authorization": f"Bearer {OC_TOKEN}"},
    json={
        "synonyms": {
            "television": ["tv", "telly"],
            "headphones": ["headset", "cans"],
        }
    },
)
assert resp.status_code == 201

// The COMPLETE map for (schema, field) - this call replaces any previous one.
const res = await fetch(
  `https://${OC_HOST}/v1/tenants/${OC_TENANT}` +
    `/fts/shop.products/description/synonyms`,
  {
    method: "POST",
    headers: {
      Authorization: `Bearer ${OC_TOKEN}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify({
      synonyms: {
        television: ["tv", "telly"],
        headphones: ["headset", "cans"],
      },
    }),
  },
);
// 201 Created, empty body
console.log(res.status);

// The COMPLETE map for (schema, field) - this call replaces any previous one.
body, _ := json.Marshal(map[string]any{
    "synonyms": map[string][]string{
        "television": {"tv", "telly"},
        "headphones": {"headset", "cans"},
    },
})

req, _ := http.NewRequestWithContext(ctx, http.MethodPost,
    "https://"+ocHost+"/v1/tenants/"+ocTenant+
        "/fts/shop.products/description/synonyms",
    bytes.NewReader(body))
req.Header.Set("Authorization", "Bearer "+ocToken)
req.Header.Set("Content-Type", "application/json")

resp, err := http.DefaultClient.Do(req)

what you get back

201 Created with an empty body. Both installers answer the same way; a malformed map is a 400. Nothing needs restarting - the records are read per request, so the next query already sees them:

curl -G "https://$OC_HOST/v1/tenants/$OC_TENANT/fts/shop.products/description" \
  -H "Authorization: Bearer $OC_TOKEN" \
  --data-urlencode "q=tv" \
  --data-urlencode "mode=bm25"

That search now ranks documents containing television as well as those containing tv, and the reverse query does the same.

installing stopwords

The array is the complete stopword list for the field. When set, it is used instead of the built-in list for the analyzer language.

curl -X POST "https://$OC_HOST/v1/tenants/$OC_TENANT/fts/shop.products/description/stopwords" \
  -H "Authorization: Bearer $OC_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{ "stopwords": ["the", "a", "with", "for"] }'

An empty array is not a no-op - it is how you turn stopword removal off for the field entirely:

curl -X POST "https://$OC_HOST/v1/tenants/$OC_TENANT/fts/shop.products/description/stopwords" \
  -H "Authorization: Bearer $OC_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{ "stopwords": [] }'

how it works

• Synonyms work from both ends. At index time every occurrence of a canonical term also writes postings under each of its synonyms. At query time a term that appears as either a canonical or a synonym expands to the whole class. That is what makes tv and television symmetric without writing the index twice.
• Expansion is capped at 32 synonyms per class to bound query-time work - a term mapping to a thousand others would otherwise issue a thousand posting-list scans.
• Stopwords apply on the analyzer-aware path. They are consulted when a document is indexed with an analyzer block, and by the query nodes that carry an analyzer, so that both sides tokenize the same way. An index write with no analyzer uses the default tokenizer and the override does not enter into it.
• Both are configuration, not content. They silently change what every later query on the field matches and how it scores, for every reader, without touching a single document - so they are charged as a schema-level change rather than a row write, and need the corresponding privilege.
• Each record is read per request, so an install takes effect on the next index or query with no restart and no reindex.