Lemmatization (engine, not yet exposed).
Lemmatization maps related word forms to a dictionary form, such as ran to run. It is implemented in the engine but is not selectable through the documented full-text HTTP API. This page explains the concept; it is not a setup procedure for a released analyzer option.
- Start with source textFor example, a sentence containing Wireless and running.
- Use supported tokenizationThe current path splits Unicode words and lowercases them.
- Search the normalized termsWireless becomes wireless; running does not automatically become run.
- Test the expected wordingCheck real searches before depending on word-form matching.
Stem vs lemma — worked example.
| Input | Example stem | Dictionary form |
|---|---|---|
| running | run | run |
| ran | ran | run |
| runs | run | run |
| better | bet | good |
| geese | gees | goose |
| mice | mice | mouse |
A stemmer removes word endings. A lemmatizer looks up a dictionary form, which can handle irregular words. These examples illustrate the distinction; neither option is selectable through this HTTP interface.
9 languages (planned).
The engine contains dictionaries for English, Spanish, French, German, Italian, Portuguese, Russian, Dutch and Swedish. Their presence in the engine does not make them query parameters or establish a release date.
What you can rely on today.
The documented text path splits words using Unicode rules and lowercases them. That lets Wireless match wireless; it does not make ran match running.
Where supported, a synonym map can group chosen words for indexing and search. Use the documented full-text runtime calls rather than inventing a language or analyzer parameter.
Where lemma will help (once exposed).
It can help people search prose with different grammatical forms of the same word. It is less useful for exact identifiers such as product codes.
Until the option is exposed, test the actual tokens and supported synonym behavior with your data. Continue with full-text search.