About Exalead CloudView Sizing

While developing your search application, you most likely used a small corpus to test indexing. Your focus was on getting relevant results that display according to the specifications for the application.

Once you start the testing phase of your application, indexing with a real corpus, your focus turns to performance, and the sizing you need to support it.

Sizing is a complex topic with so many variables that it is impossible to provide hard & fast rules.

We can, however, explain the main sizing considerations in a project, and the type of resource they impact.

Important: This section does not replace an NETVIBES professional services engagement. Sizing is complex, with many factors to consider. Consult an NETVIBES sizing expert before undertaking sizing.

This page discusses:

How Project Requirements Impact Sizing
Disk Requirements
RAM Sizing Formula

How Project Requirements Impact Sizing


This requirement	Impacts
Project description
Do you need exhaustive search? (It guarantees that it searches and retrieves every match, even if it takes longer).	CPU
What is an acceptable response time? Typically, for dashboards it is 4 or 5 sec; for intranet search it is in milliseconds.	CPU & RAM
Scope: what is the expected lifecycle for this app? Can this app be used for further projects?	Number of machines, or type of hardware
How many documents can be indexed?	Mainly disk space. May also impact RAM & CPU
How many data sources, and what kind?	CPU & RAM
How many users + what is the estimated QPS (Queries Per Second)?	CPU & RAM
Index freshness - every minute/every hour/every day?	CPU, RAM & disk performance
What is the expected throughput (how many incremental changes to your corpus)?	CPU, RAM & disk performance
Does it need to be an HA deployment?	Number of machines

See also the Before Going Live in the Exalead CloudView Administration Guide for a list of issues to be aware of when going into production.

Disk Requirements

See the General System Requirements in the Exalead CloudView Installation Guide.

RAM Sizing Formula

You can estimate your total RAM requirements like this:

RAM for processes + RAM for RAM-based fields + RAM for document cache.

The table below explains this in more detail.


Memory	is used for	and requires
RAM for processes	running Exalead CloudView processes.	Fixed at 8 GB
RAM for RAM-based fields	storing metas used for faceting, sorting, grouping and in virtual expressions in memory.	Highly variable: `2* (avg size of all RAM-based meta * # docs)`
RAM for document cache	caching indexed documents in memory.	20% of the size of your index. To estimate index size Enterprise search: `20 kb * # docs` Analytical dashboards or eCommerce: `3 kb * # docs`

To put all this together:

For enterprise search

Total RAM = 8 GB + 2 (avg size of all RAM-based meta * # docs) + 0.2(20 kb * # docs)

For search-based applications (analytical dashboards) or eCommerce

Total RAM = 8 GB + 2 (avg size of all RAM-based meta * # docs) + 0.2(3 kb * # docs)

To see the RAM usage per field, use the cvdebug command-line tool located in <DATADIR>/bin and start the following command:

cvdebug index dump-attribute-group-column-infos