Chunk Operations

This page discusses:

See Also
Appendix - Configure Document Processors
Chunker

Copy Context Chunks

Copies all document chunks from the context specified in Input from, and creates new document chunks with the same score, language, and part, in the context specified in Output to.

You can apply matching conditions to this processor to refine its behavior. For example, if you have a multivalued field having the following values: order-1, order-2 and order-A, and want to numeric orders only (that is, everything but not order-A), you can set a condition with a value that equals the order-d+ regular expression.

Multi-Context Encoder

Creates a DocumentChunk containing the ContextName and the textual value of the DocumentChunks matching 'inputContexts'.

This processor can be used, for example, to store arbitrary (key, value) pairs into one single index field.

The serialization format is the following:

  • "ContextName1"="TextContent1"

  • "ContextName2"="TextContent2"

  • ...

The double-quote character in name and value is escaped with a backslash.

Note: This storing method is inefficient and must be used with caution.

New Chunk

Creates a new DocumentChunk with 'outputContext' as ContextName, and textual content specified in 'value'.

Remove Contexts

Removes all DocumentChunks with a ContextName matching 'inputContexts'.

Rename Context for Chunks

Renames each DocumentChunk with ContextName matching 'inputContext' with a ContextName 'outputContext'.

Rename Unmapped Contexts

Changes the ContextName for all DocumentChunks associated with a ContextName that does not have a Mapping Configuration.

This avoids extensive renaming using RenameContext.

Replace Values

Compares all DocumentChunks for a given inputContext with the specified KeyValue map.

When the DocumentChunk value is an exact match, the specified string replaces it.

You can use this processor, for example, to normalize different spelling for document metadata.

NOTE: The specified KeyValue map must be an exact match with the complete DocumentChunk.

To replace only a substring of a DocumentChunk, use ReplaceRegexp.

Input: All DocumentChunks associated with the specified 'inputContext' ContextNames.

Output: Same as input.

Value Selector

Takes the input contexts in the specified order, and as soon as one is found, copies the content to the output context and stops.