SEARCH

Input

The SEARCH operator is a source so it has no input.

Output

By default, the output of this operator is a stream with multiple elements. The type of the elements in the stream depends on the configuration.

When the Matching Annotations are enabled, the type of element in the output stream becomes a Tuple (see below).

Configuration

The operator outputs the selected Index Unit elements that match a predicate (Boolean condition). You configure the predicate using an expression. The implementation uses indexing data structures, so the expression is not executed for every document in the Index.

Example

We select the Person type and define person as variable name.

person.age > 30 AND person.city = "New York City"

The return type of this expression must be a Boolean.

The operator outputs every Item of the Person type (including the Employee type) with an age higher than 30 and living in "New York City".

Matching Annotations and Text Scoring

Matching Annotations are metadata indicating how the Item has matched the expression.

There are two types of Matching Annotations: "implicit" and "explicit".

You declare explicit annotations in the expression, for example:

person.age > 30 AND ( (person.city = "New York City"){city=1} OR (person.city = "Miami"){city=2} )

The Items that match the query are annotated with city=1 or city=2 depending on the city.

The query engine computes implicit annotations internally, without explicit declaration in the expression. It is an implicit annotation named score, that generates the text scoring.

If you add matching annotations (ma), the output type of the Operator changes to Tuple. The output displays a Tuple containing an Item and the different matching annotations that have been listed.

In the example above, to enable text scoring, we have 2 matching annotations, city and score. The output type is: Tuple<item:Item<Person>, ma:Tuple<city:Integer, score:Integer>>

Builtin Matching Annotation

You can use matching annotations to know the frequency of terms in text.

Example 1

Input:

SEARCH
  expression:   hasTerm(item.text, "car")
  matching annotations: ["score"]

Output:

Tuple<item: Item<core.Item>, ma: Tuple<score: Integer>>

For example:

{
  item: {
    ...
    text: "I love car"
  },
  ma: {
    score: 16
  }
},
{
  item: {
    ...
    text: "I love my car and cars are eveything for me"
  },
  ma: {
    score: 32
  }
},

Example 2

Input:

SEARCH
  expression:   hasTerm(item.text, "car"){boost= 10} OR hasTerm(item.text, "literature")
  matching annotations: ["score", "boost"]

Output:

Tuple<item: Item<core.Item>, ma: Tuple<score: Integer>>

For example:

{
  item: {
    ...
    text: "I love car"
  },
  ma: {
    score: 160
  }
},
{
  item: {
    ...
    text: "I love car because cars are eveything for me"
  },
  ma: {
    score: 320
  }
},
{
  item: {
    ...
    text: "I love litterature"
  },
  ma: {
    score: 42
  }
},

In this example, the scores of the item.text containing car have been multiplied by 10, as specified with {boost= 10} in the expression.

Other Use of Matching Annotations

You can use matching annotations for other purposes. They are typically useful when used as "side variables". For example, to know what worked and what did not work.

Input:

SEARCH
  expression:   hasTerm(item.text, "red"){matchRed= 1} OR hasTerm(item.text, "magenta"){matchMagenta= 1}
  matching annotations: ["matchMagenta", "matchRed"]

Output:

Tuple<item: Item<core.Item>, ma: Tuple<matchMagenta: Integer>>

For example:

{
  item: {
    ...
    text: "these tulips are red"
  },
  ma: {
    matchMagenta: 0,
    matchRed: 1
  }
},
{
  item: {
    ...
    text: "The ink cartridge color is magenta"
  },
  ma: {
    matchMagenta: 1,
    matchRed: 0
  }
},
{
  item: {
    ...
    text: "John cannot make the difference between red and magenta"
  },
  ma: {
    matchMagenta: 1,
    matchRed: 1
  }
},