About Spell-Check

The transformation distance (Damereau-Levenshtein distance) is the number of changes (replace a letter, add a letter, transpose a letter, delete a letter) required to transform word A to word B. Each change represents a distance. The greater the transformation distance, the larger the difference between two words.

Exalead CloudView performs distance 1 and distance 2 spell-checking. This means spell-check suggests alternate words that have a distance of 1 (or 2, for longer words) from the original word.

For example, let us assume each transformation change (replace, add, transpose, delete) has the same distance value of 1. If you index the Exalead CloudView documentation and enable spell check, a search on “exaleed” would suggest “exalead”. “exalead” qualifies as a valid suggestion of spell check because it is at distance 1 from the original search term (an "a" replaced the third “e”), and occurs quite frequently in the documentation.

By contrast, “exaloda” (if this word really exists in the corpus) would not qualify because it is at distance 2 from the original search team: it requires substituting the “o” with an “e”, plus transposing the “d” and “a”.

Spell-check is far more effective when your corpus is indexed with phonetic and Ngram extraction. You can configure this in the Data Model, as described in Set Up Ngram and Phonetic Extraction in the Data Model (Optional).

An ngram is a sequence of N words. Its sole purpose is to improve spell checking for multiword expressions. At indexing time, 2-, 3-, and 4-word ngrams are extracted from your corpus and stored in the dictionary. Exalead CloudView is then able to compare a user’s query with these ngrams, calculate the probability that the user meant to use a different spelling, and then suggest alternate expressions.

Important: These options improve quality but also impact performance. Extracting spell check ngrams is CPU-intensive. Storing ngrams and phonetic forms significantly increases dictionary size, since it stores the various phonetic forms for tokens in addition to the tokens themselves.

You can force or prevent certain spell-check suggestions through the use of allow lists and block lists.

• Allow list: Always suggests X when a user queries for Y.

  For example, people often enter "excede" instead of "exceed". To automatically suggest "exceed" (regardless of which spelling occurs most frequently in your dictionary) when a user types "excede", add this word pair to your allow list.

• Block list: This does not block specific spell check suggestions. Instead, it blocks specific query words from correction.

  Say that you want to search for documents containing brand names, and you do want the spellcheck to suggest other names. For example, for "Informatica", by default, spellcheck suggests "information". Solution: add "Informatica" to the spellcheck block list to ensure that spellcheck never corrects this term.

Default settings have been chosen so that search performance is not impacted by the spell-checker activation.

To get a more effective spell-check:

• Check that Extract spell check ngrams is clicked in Data Model > Semantic Types > text.
  Note: The ngram dictionary is built from a sample of document texts. Only one document out of 5 takes part extraction. Therefore, you have to index a minimum number of documents to reach a reasonable level of quality for spell-check. Being based on statistical algorithms, the more documents you have in the index, the better the spell-checker works.
• Select Keep only most important suggestions in Search > Search Logics > your search logic > Query Expansion to increase suggestions relevancy. It is assessed by querying the index and checking the count of documents returned.
• If spell-check suggestions are displayed too often, set a value for Only check spelling if query returns less than in Search > Search Logics > your search logic > Query Expansion.