UC-2: Enriching Child Documents with Parent Document Metas

Flattening data allows you to build powerful queries in Exalead CloudView, and the Consolidation Server is the right tool to achieve this kind of operation.

In the provided coffee sample, trade records contain, for each year and each country, a volume of exchanges for each type of trade (import, export, re-export). However, the ICO membership status is only present on the country record. For a relational database, you could write an SQL join query to retrieve trade only for the countries that are members of the ICO. For an index engine, it is more efficient to move down this information directly to the trade record at indexing time.

This task shows you how to:

Before you begin: We assume that UC-1 has been completed.

Step 1 - Define the Source Connector for Trades

  1. In the Administration Console, go to Index > Connectors and click Add connector.
    1. In Name, enter trades.
    2. For Type, select the JDBC connector.
    3. For Push to PAPI server, select the Consolidation server cbx0 instance.
    4. Click Accept.
  2. For Store documents in data model class, choose the trade class.
  3. In Connection parameters:
    1. For Driver, enter org.sqlite.JDBC
    2. For Connection string, enter jdbc:sqlite://<INPUTDIR>/coffee.db
    3. Click Test connection. The database connector automatically connects to the database.
  4. In Query parameters:
    1. For Synchronization mode, select Full synchronization
    2. For Initial query, enter select country_id, type, volume, year from trade
  5. Click Retrieve fields.
  6. Define the country_id, type, and year fields as primary keys.
    1. Click the country_id field to expand it.
    2. Select Use as primary key.
    3. Repeat the operation for the type and year fields.
  7. Click Apply.

Step 2 - Configure Consolidation

Configure the Transformation Processor

  1. Go to Index > Consolidation
  2. Add a new transformation processor:
    1. Select Groovy as format
    2. For Name, enter Trades
    3. Click Accept
  3. For Source connector, select trades
  4. Replace the default code by the following one: 

    // Process all nodes
process("") {
 // Link trade records to nodes having the “country” type with a link based on the trade type
 // (i.e.; Import / Export / reExport) as arc label
 it.addArcFrom(it.metas.getValue("type"), "country_id=" + it.metas.getValue("country_id") + "&");
}

With this processor, we have achieved to link trades to their related countries.

Configure the Aggregation Processors

  1. Add an aggregation processor:
    1. Select Groovy as format
    2. For Name, enter Trades_UC_2
    3. Click Accept
  2. Replace the default code by the following one: 

    // Process nodes having the “trade” type
process("trade") {
 log.info "trade found : " + it.metas.year + "_" + it.metas.country_id + "_" + it.metas.type ;

 // Find the ICO status member from nodes.
 // It is now possible to use a dynamic path based on node meta
 for (path in (match( it, "-" + it.metas.getValue("type") + "[country]" )) ) {
 // Retrieve the last path element
 last = path.last();
 log.info "Country found : " + last.getUri();

 // Get the “membership” meta value from nodes having the “country” type
 it.metas.membership = last.metas.getValue("ico_status");
 }
}

  3. Save and apply the configuration.

Step 3 - Scan Source Connectors and Check What Is Indexed

  1. Go to the Home page and under the connectors list, click Scan for the trades JDBC connector.

    Trades are indexed.

  2. Open the Mashup UI application search page: http://<HOSTNAME>:<BASEPORT>/mashup-ui/page/search
  3. Check that trade documents have an ICO Membership facet available.



The following graphic shows what we achieved on the object graph.