Lemmatization

Lemmatization expansion allows you to search for the masculine or singular forms of feminine or plural nouns and adjectives.

For example, searching for “geese” would also search for the lemma “goose”.

There is often some confusion in understanding the difference between lemmatization and stemming. Both have an objective of finding a common base for query words with several related derivations or inflected forms. See Stemming vs Lemmatization.

This page discusses:

See Also
Configuring Query Expansion
Stemming

Dependencies

For lemmatization to work at search-time, you must first create lemmatized forms at indexing time. For more information, see Lemmatizer

Languages Lemmatized Natively

Lemmatization is available in the following languages:

  • English (en)

  • French (fr)

  • German (de)

  • Italian (it)

  • Portuguese (pt)

  • Russian (ru)

  • Spanish (es)

Languages Lemmatized with Basis Tech Add-On

If you want to perform lemma query expansion for languages tokenized by the Basis Tech (Extended Languages) tokenizer, you must configure the Lemmatizer semantic processor, as well as follow some additional steps for tokenization.

For details, see Enable Lemmatization with Basis Tech.

Lemmatization Options

When configuring a prefix handler’s query expansion, the following lemmatization options are available.

Option

Description

weight or w

Relative ranking weight of the lemmatized forms. This controls the "w" parameter of the standard term score formulas. The default value is 0.1.

For details on term score and weight, see Ranking and Sorting Search Results

masculinization

Looks up for the masculine form of a word when a feminine form is entered. For example, entering canadienne searches on canadien.

masculine_weight or mw

When transcribing a word from the feminine to masculine form, the resulting masculine form has this weight value instead of "w". The default value is 0.01.