Custom Configuration

You can customize the Crawler connector to override the standard crawler configuration using the <DATADIR>/config/CrawlConfig.xml file.

You configure the crawler customization by adding a CustomCrawlConfig object in the Crawler object.

You can customize each crawler individually through the following customization method.

The Crawler connector is in Exascript language. For more information, contact Professional services.

Note: You can rewrite the entire crawler pipeline or only parts of it. The missing parts are taken from the default one. Therefore, you can override a single processor of the crawler pipeline.