Normalization

This page discusses:

See Also
Appendix - Configure Document Processors
Normalizer

Date Formatter

If a document chunk matches either:

The Date Formatter generates three additional document chunks, each with its own context name, using the following naming convention:

  • $inputContext$dateTimeOutputContext (default format: %Y/%m/%d-%H:%M:%S)

  • $inputContext$dateOutputContext (default format: %Y/%m/%d)

  • $inputContext$timeOutputContext (default format: %H:%M:%S)

    Note: You can also define specific output formats in the config XML.

Automatically Recognized Input Formats

If no input format is specified, the Date Formatter automatically recognizes dates in the following formats.

Note: Timezones are ignored.

  • RFC 822 and 2822

  • ISO 8601 and RFC 3339

  • Other (‘day’ and ‘month’ values are only recognized if written in English):

    • "day, DD month YYYY hh:mm:ss"

    • "day, DD month YYYY hh:mm:ss timezone"

    • "day, DD month YY hh:mm:ss"

    • "day, DD month YY hh:mm:ss timezone"

    • "day month DD YY hh:mm:ss"

    • "day month DD hh:mm:ss timezone YYYY"

    • "DD month YYYY hh:mm:ss"

    • "YYYY/MM/DD hh:mm:ss"

    • "YYYY/MM/DD-hh:mm:ss"

    • "MM/DD/YYYY hh:mm:ss"

    • "MM-DD-YYYY hh:mm:ss"

    • "MM/DD/YYYY"

    • "MM-DD-YYYY"

Output Formats

The default output formats are:

  • date-time: %m/%d/%Y %H:%M:%S

  • date: %m/%d/%Y

  • time: %H:%M:%S

    Note: You can also define specific output formats in the XML configuration, using UNIX date syntax.

Numerical Formatter

Creates valid numerical chunks from various number formats.

Public URL Processor

Creates 4 DocumentChunks, each associated with a different ContextName, for each input DocumentChunk associated with the ContextName 'inputContext':

  • 'treeOutputContext'

  • 'leafOutputContext'

  • 'urlOutputContext'

  • 'urlCategoryOutputContext'

Units of Measurement Normalizer

Detects the unit symbol if specified in the input value and operates a conversion according to the index unit symbol when required. Then creates a new meta-data with the normalized value.

You must define the following properties:

  • Input from – Specify the name of your data model measurement property.

  • Measurement type name – Specify the measurement type defined in your data model measurement property.

  • Unit Symbol – Specify the Unit symbol to use in the output value.

  • Suffix to add to the meta_name – (default "_um") suffix name to add to the meta name to create output meta value.

For example, if you have a volume index field with a measurement type set to volume and a unit symbol set to millimeter:

Let us say that Document1 has a volume meta-data containing the value 50cl and Document2 has a volume meta-data containing the value 1000. Then the output for Document1 is a new volume_um meta containing the value 500 and for Document2 a volume_um meta with the value 1000.