Maximize Performances

A default configuration (4 threads for folders, 4 threads for documents) is normally sufficient for most cases, the limiting factor being the filesystem/disks I/O. You can however consider tuning the following points to maximize the filesystem connector performances.

This task shows you how to:

See Also
Advanced Configuration Parameters

Maximize Crawl Throughput

In the Advanced tab, you can change the value of the No. pipeline document thread and No. pipeline folder thread options.

Specify sufficient values for these options, to process more document/folder pipelines in parallel. For example, 32 for big configurations with a lot of I/O, or even higher if you can afford expensive RAID+SSD.

Add a Metadata Compaction Preprocessor

With small files, when PushAPI performance is blocking, add a metadata compaction preprocessor to boost network I/O and remove contentions.

  1. Go to Advanced > PushAPI filters
  2. Click Add item.
  3. In PushAPI filter type, select Metadata Compaction PushAPI Filter.