Stemming

Stemming expansion allows you to search for words with a common root, or stem.

For example, searching on the word “Britannia” would expand the query to include words with the stem “Britann”, such as “Britannic” and “Britanny”.

There is often some confusion in understanding the difference between stemming and lemmatization. Both have an objective of finding a common base for query words with several related derivations or inflected forms.

This page discusses:

Dependencies
Stemming vs Lemmatization
Stemming Rules
Stemming Options

Dependencies

For stemming to work at search-time, you must first create stemmed forms at indexing time. For details, see Snowball Stemmer.

Stemming vs Lemmatization

Stemming is the simpler method, as it seeks to find the root (the stem) of a word by cutting off endings. For example:

Searching on alsaciennes (women from the Alsace region of France) would also search for words with the stem alsac.
Searching on alsace would also search for words with the stem alsac.

By contrast, lemmatization uses a more complex morphological analysis to find the singular or masculine form of nouns and adjectives.

Searching on alsaciennes also searches for words with the lemma alsacien.
Searching on alsace also searches for words with the lemma alsace.
Moreover, searching on geese would also search for goose.

Stemming Rules

Depending on the language, two kinds of rule dictionaries are used:

Rules based on the "Snowball" library.
Internal CloudView rules

For stemming to work, the words must have been extracted from text at indexing time (this is the default configuration).

Stemming Options

When configuring a prefix handler’s query expansion, the following stemming options are available.


Option	Description
`max_matches` or `m`, or matches	Searches no more than N stems for a word. If the number of available stems exceeds this value, Exalead CloudView searches the N most frequent stems in the corpus. The default value is `10`.
`weight` or `w`	Relative ranking weight of the stemmed forms. This controls the w parameter of the standard term score formulas. The default value is `0.1`. For details on term score and weight, see Ranking and Sorting Search Results.