Where Clause Guidelines

These tips help you include where clauses within a query.

This page discusses:

Select Fields in Well-structured Queries

Queries that perform well always include at least 1 field that can be converted to SQL to limit the results returned from the database server. Criteria that cannot be converted to SQL are evaluated against this candidate result set in memory and the final result set is displayed. The smaller the candidate result set, the better. Well-structured queries use several criteria that are “SQL convertible” so that the work performed in memory is kept to a minimum.

The only query terms considered for SQL conversion are relational expressions that use this syntax: <left_operand> <operator> <right_operand>.

SQL convertible fields were formerly referred to as “indexed” fields, before the advent of indexed attributes.

The query optimizer processes a query in the following manner:

  1. The where clause is first factored into the “OR” form: A || B || C || D. This is done using the boolean logical equivalences. For example:
    (A || B) && C 

    becomes

    (A && C) || (B && C) and !(A && B) == !A || !B 

    Each of the OR’d terms will be processed independently.

  2. The query is parsed to identify all fields that are SQL convertible (that is, which terms can be converted into SQL and be included in the select commands issued to the database server).
  3. The query determines whether the set of SQL convertible terms are primarily related to business objects or relationships to decide if the initial selects are requested against an lxBO table (businessobjects) or lxRO table (relationships).
  4. The commands are issued to the database server to select rows from lxBO or lxRO including these SQL convertible terms. This step returns a candidate result set which satisfies the SQL convertible terms (associated to either objects or relationships).
  5. The remaining criteria (non-SQL convertible terms) is processed against the candidate objects (or relationships). This step requires additional db selects to get the data specified by those terms, and evaluating the expressions in memory within the process, and producing a final result set.
  6. Any selects associated with the query are processed that could involve further db selects to get additional data about the final result set.

Steps 4-6 are repeated for each vault identified by the Vault field of the query.

The most optimal queries use sufficiently specific SQL convertible terms to result in the smallest possible candidate result set. This strategy provides the following advantages:

  • Using SQL fully leverages the database server’s optimization and indexing
  • Minimizing the candidate result set means there are less objects to postprocess to get a final result set, which requires both less time and less in-process memory.

The following selectables are SQL convertible. All queries should include at least one of these criteria:

SQL Convertible Field Means
name Specify as NAMEPATTERN in name field
type Specify as TYPEPATTERN in type field
owner Specify as USERPATTERN in owner field
organization == ORGPATTERNObject's organization owner
project == PROJECTPATTERNObject's project owner
Policy == POLICYPATTERN Governing policy of object
Format == FORMATNAME Format of files checked into object
Originated > DATE Date object was created
Modified > DATE Date object was last modified
Current == STATEPATTERN Current state of object
Id == IDObject ID (Format: 1234.5678.910.1112)
Physicalid == PIDUses UUID format: 32 hex chars
Majorid == MIDUses UUID format: 32 hex chars
Versionid == VIDUses UUID format: 32 hex chars
Logicalid == LIDUses UUID format: 32 hex chars
Isbestsofar == TRUEStatus of bestsofar
Locker == USERNAMEThe username of the person that locked the object
attribute[] == VALUE Value of attribute on object
to[].attribute[] == VALUE Value of attribute on relationship connected to object
from[].attribute[] == VALUE Value of attribute on relationship connected from object
reserved == TRUEBoolean indicating reserved status of business object or connection
reservedby == USERNAMEA non-empty string or context-user
Reservedstart > DATEDate and timestamp of when the object is reserved
to[RELTYPE]==TRUETo find objects with relationships of the RELTYPE pointing to them
from[RELTYPE]==TRUETo find objects with relationships of the RELTYPE pointing from them
revision==firstTo find objects that are the first revision of their minor revision family
revision==lastTo find objects that are the last revision of their minor revision family
member[SETNAME]==TRUECheck if object is member of set SETNAME
toset[SETNAME,RELTYPE]==TRUECheck if object is pointed to from any object in set
fromset[SETNAME,RELTYPE]==TRUECheck if is from object pointing to any object in set.
type.kindof{TTYPE]==TRUECheck if object's type derives from TTYPE
interface==Check if object has interface applied
interface[INTTYPE]==TRUECheck if object has interface applied
interface.kindof[INTTYPE]==TRUECheck if object has interface applied that is derived from INTTYPE
ownerhip.organization == ORGNAMECheck secondary ownership organization
ownership.project == PROJECTNAMECheck secondary ownership project
format.file.store == STORENAME Store containing files checked into object
format.file.location == LOCATIONNAME Location containing files checked into object
format.file.modified > DATE Modified date on files checked into object
format.file.lastsync > DATE Last sync date on files checked into object
format.file.lastaccess > DATE Last accessed date on files checked into object

You can use SQL convertible fields with any relational operator (==, !~=, <> and so on) applied to a constant value that can include wildcards. For example,

name ~= 'A*B'

Returns all objects whose name begins with “A” and ends with “B”. Use of select fields in where clauses that are not in the list above will likely result in non-optimal queries since non-SQL convertible fields have to be evaluated on the client.

It is best not to use any word that can be a selectable for a business object or connection as a value in the Where clause of a query because it will be evaluated as a select clause rather than being taken literally. If it is unavoidable, see Using Const for Reserved Words.

Clauses That Are Not SQL convertible

Any selectables not on the above list are not SQL convertible when used in a where clause. For example, the following are not SQL convertible:

description 
grantor (and grantee, granteesignature) 
state[] 
revisions[] 
previous (and next) 
type.kindof

“Revision” is not SQL convertible when included in the where clause, but is SQL convertible when included in the business object specification part of the query.

Selectables that are not SQL convertible cannot be built into the SQL commands that get objects and connections from the database server, so these clauses are not used to limit the number of objects that are retrieved from the server. Qualifying these clauses is very sensitive to the number of objects retrieved, since for each such object, additional SQL calls have to be constructed to retrieve the values of these fields, and data has to be stored in the clients memory. Given these realities:

Important: Never execute a query that only has fields that are not SQL convertible. Always make sure that a query containing fields that are not SQL convertible also has some SQL convertible fields to limit the number of objects retrieved.

Here are a few examples of common queries that are bound to be slow. In all of the examples, the type (PART) is specified, but a production database can have huge numbers of PARTs, so the type specification is not very limiting:

PART * * where 'description ~~ “*a word*”' 
PART * * where 'state[StateName].satisfied == TRUE' 
PART * * where 'revision == last'

In the above queries, it would be best to include an attribute that substantially limits the number of PARTs that the system would have to examine.

The number of objects retrieved from the database server (that is, the size of the candidate result set) by the SQL convertible fields is the biggest factor in query performance and client memory requirements. The length (in characters) of the where clause or number of expressions within the where clause are immaterial. Actually, a longer where clause is desirable if much of it is comprised of SQL convertible fields that limit the number of objects retrieved:

PART * * where ' (description ~~ “*a word*”) AND (attribute[Keyword] ~~ “abc*”)
 AND (attribute[Keyword] ~~ “123*”) '