Specify a Tokenization Configuration for Prefix Handlers

Each search logic needs a default tokenization that determines how prefix handlers break down user queries into tokens.

This section covers how to specify a specific tokenization configuration for search-time processing. To learn how to create new tokenization configs and specify them at index-time, see Customizing the Tokenization Config.

This task shows you how to:

See Also
The Different Types of Prefix Handlers
Code Samples to Create Similarity Query Prefix Handlers

Specify Another Tokenization Config for a Search Logic

  1. In the Administration Console, create a new default tokenization config as described in Customizing the Tokenization Config.
  2. Specify this default tokenization config for:
  3. Go to Search > Search Logics > Your search logic > Query Language tab.
  4. From the Default tokenization config list, select the new tokenization config.
  5. Click Apply.

All prefix handlers use this tokenization config to tokenize queries.

Specify a Tokenization Config for a Prefix Handler

For metas that need to be tokenized differently than the default tokenization config, you can specify a different tokenization config for the prefix handlers targeting those metas.

  1. In the Administration Console, create a new “exception” tokenization config as described in Customizing the Tokenization Config.
  2. Specify this default tokenization config for the index mappings for metas that need to be tokenized differently. See Specify Another Tokenization Config for an Index Mapping.
  3. Go to Search > Search Logics > Your search logic > Query Language tab.
  4. Select the prefix handler for the appropriate meta.
  5. From the Default tokenization config list, select the “exception” tokenization config you created in step 1.
  6. Repeat for all the prefix handlers that need a different tokenization.
  7. Click Apply.