Approximation

Approximation is similar to spell check. The difference is that approximation only applies to the word that follows the prefix handler with which it is associated (myprefix: word). Meanwhile, spell check applies to the entire user query and has more configuration options.

The approximation module considers both the word length and the number of transformations allowed to expand the original query with additional words. Transformations mean that you replace (substitute) a letter, add a letter, transpose a letter, or delete a letter.

The approximation module searches for words that are at transformation distance 1 or 2 of the original word from the user’s query. Distances are hard-coded, so you can only control the word length that triggers distance 1 or distance 2. In other words, depending on the word length, you set either max_distance=0, 1 or 2.

See also Adding 'Did You Mean?' Spell-Check.

When configuring a prefix handler’s query expansion, the following approximation options are available.

Table 1. Approximation Options

Option	Description
flm	Specifies an additional transformation distance (think of it as a penalty) on any transformation on the first letter. This shows that in most cases, people do not make typos on the first letter. For example, "abc" and "zbc" are at: distance 1 if flm=false, distance 2 if flm=true. The default value is true.
max_matches or m or matches	Searches no more than N fuzzy matches for a word. If the number of available fuzzy forms exceeds this value, Exalead CloudView searches the N most frequent forms in the corpus. The default value is 10.
min_chars_distance1 or mcd1	Only searches for distance 1 fuzzy matches if the original word in the query is at least N characters long. This avoids too much approximation on very short words. The default value is 5.
min_chars_distance2 or mcd2	Only searches for distance 2 fuzzy matches if the original word in the query is at least N characters long. This avoids too much approximation on short words. The default value is 10.
weight_distance1 or w1 or wd1	Relative ranking weight of the fuzzy matches at distance 1. This controls the "w" parameter of the standard term score formula. For details on term score and weight, see Ranking and Sorting Search Results. The default value is 0.1.
weight_distance2 or w2 or wd2	Relative ranking weight of the fuzzy matches at distance 2. This controls the w parameter of the standard term score formula. For details on term score and weight, see Ranking and Sorting Search Results. The default value is 0.01.

Take the word “screwdriver”.

If the query expansion config expression is set to approximate{min_chars_distance1=5} only

• srewdriver => approximation works, there is 1 transformation (deletion of the “c” character).

• screwdrive => approximation works, there is 1 transformation (deletion of the last character “r”).

• screwqdriver => approximation works, there is 1 transformation (insertion of the “q” character).

• scrwedriver => approximation works, there is 1 transformation (transposition of the “w” and “e” characters).

• screqdriver => approximation works, there is 1 transformation (substitution of the “w” character by the “q” character).

• sredriver => approximation does not work as there are 2 transformations (deletion of “c” and “w”).

If we set the query expansion config expression to approximate{min_chars_distance1=,min_chars_distance2=10}

• sredriver => approximation still does not work because:

  • there are 2 transformations (deletion of “c” and “w”)

  • and there are only 9 characters in the query word whereas the minimum number to get 2 transformations is set to 10.

If we set the query expansion config expression to approximate{min_chars_distance1=,min_chars_distance2=9}

• sredriver => approximation works because:

  • there are 2 transformations (deletion of “c” and “w”)

  • and there is the minimum number of 9 characters to make these 2 transformations.

Let us say that we have created a prefix handler called approxprefix using the approximation module with its default option configuration.

We enter the query: approxprefix:correkt and get matches for documents containing the word correct. However, if we search for approxprefix:vorrect, we do not get any matches.

This behavior is normal, since in most cases, people do not make typos on the first letter. Therefore, there is an additional transformation distance by default for any transformation on the first letter (flm=true).

Since vorrect is a short word (fewer than 10 characters) and substituting “c” for the first letter “v” equals to a transformation distance of 2, the approximation module does not expand the search.

If we search for approxprefix:vorrection, the approximation module expands the search to include correction, as the search term is 10 characters long.

To disable the additional transformation distance for first letters, edit the query expansion config expression to include flm=false. In our example, we would have the following expression: approximate{flm=false}

We could also tackle this kind of issue by reducing the minimum length to trigger distance 1 and distance 2 (using the min_chars_distance1 and min_chars_distance2 options).