The row filter evaluates whether to include rows depending on a logical expression that evaluates to either true or false.
The Filter Row step filters rows that meet either any (logical Or) or all (logical And) conditions in one or more groups of expressions. groups expressions that filter rows fields of group of field expressions, group expressions, or any combination of field and group expressions evaluated by the And operator or by the Or operator. A binary operator (such as Equal or Greater than) in a field expression compares values in a column against either a literal value or the value in another column. A unary operator (such as Null or Empty) in a field expression checks the value itself without any comparison. A group expression consists any series of field or group expressions evaluated with either And or Or.
Basic Filter
The Basic Filter settings allow you to combine groups of field statements in a single
level expression. Each statement evaluates a single column value in a record. You can
create one or more groups of statements in the expression. Each group uses row filter
operators to combine field statements into a single expression that returns
True or False. Multiple groups are always combined so
the outcome of every group must be true for a record to be allowed.
For example, if you create groups named A through D, a
record is allowed if every group returns a True value.
A AND B AND C AND D
In other words, the basic filter combines groups in a single level of AND expressions.
Every group must return True to allow a record.
To create a basic filter expression:
- Click Add Condition to add a field condition within a group.
- Click Add Group Condition to add another group.
- Within each group you can choose Match All to match all
conditions within the group (the logical
AND), or you can choose Match Any to match any condition within the group (the logicalOR). - When there is more than one group, they are all combined within one level in the Main Group.
- Groups must always match all (
ANDexpression).
You can always start configuring this step with Basic Filter settings and anytime later click Switch to advanced to continue configuring the step with Advanced Filter settings. A basic filter expression can be expressed by the advance filter expression, so you do not lose anything when you switch from basic to advanced settings.
Advanced Filter
The Advanced Filter settings allows you to create hierarchical expressions with which to
evaluate and filter records. Both field statements and group expressions may be combined
with either Match All (logical AND) or
Match Any (logical OR) Each level in a hierarchy
behaves like a parenthetic expression in a larger expression, so if you have groups
A through G, you can combine groups and subgroups in
complex expressions, such as:
(A AND B) OR (C AND (D OR E)) AND (F OR G)
Whether you convert a Basic Filter to an Advanced Filter or start with Advanced Filter, it always starts with a single Group 1 within the Main Group. You can add field statements within this group just to create a filter that is no different from the basic filter. In the Advanced Filter, unlike the Basic Filter, you can add groups within the Main Group or any other group.
- To add a group within any group, click the Add button on the line that circumscribes the group, then click + Add Group Condition.
- To add a field statement within a group, click the Add button on the line that circumscribes the group, then click + Add Condition.
- To specify that all conditions must be matched within a group (logical
AND), click Match All next to the group name. - To specify that any condition within a group may be matched (logical
OR), click Match Any next to the group name.
Note that if you click Switch to basic, you will lose an advanced filter expression. An advanced filter expression cannot be expressed as a basic filter expression, all conditions are deleted when you switch to the basic filter.
Save
Click this button to close settings and save changes to the transformation settings.
Preview
Click this button to preview the results of the transformation settings.
Cancel
Click this button to close settings for this transformations without saving any changes.
Row Filter operators
Binary Row Filter operators compare column values either a literal value or with the value in another column of the same data type. Unary Row Filter operators perform a test on the column value itself without any comparison.
Numeric field operators
Use numeric operators to evaluate values in Integer, Long, Float, Date, DateTime, and Time fields.
| Operator | Binary/Unary | Expression true when |
|---|---|---|
| Equal | Binary | Field value is equal to a literal or field value. |
| Not equal | Binary | Field value is not equal to a literal or field value. |
| Null | Unary | Field value is null. |
| Not null | Unary | Field value is not null. |
| Between | Tertiary | Field value is between two (lesser and greater) literal or field values. |
| Not between | Tertiary | Field value is not between two (lesser and greater) literal or field values. |
| Greater than | Binary | Field value is greater than a literal or field value. |
| Greater than or equal | Binary | Field value is greater than or equal to a literal or field value. |
| Less than | Binary | Field value is less than a literal or field value. |
| Less than or equal | Binary | Field value is less than or equal to a literal or field value. |
String field operators
Use string operators to evaluate values in String fields.
| Operator | Binary/Unary | Expression true when |
|---|---|---|
| Equal | Binary | Field string value matches a literal or field value. |
| Not equal | Binary | Field string value does not match a literal or field value. |
| In | Binary | Field string value matches any one of two or more literal values. (Type and press Enter to add each literal value to the box.) |
| Not in | Binary | Field string value does not match any one of two or more literal values. (Type and press Enter to add each literal value to the box.) |
| Empty | Unary | Field string value is empty. |
| Not empty | Unary | Field string value is not empty. |
| Contains | Binary | Field string value contains a literal or field value. |
| Not contains | Binary | Field string value does not contain a literal or field value. |
| Start with | Binary | Field string value starts with a literal or field value. |
| Not start with | Binary | Field string value does not start with a literal or field value. |
| End with | Binary | Field string value ends with a literal or field value. |
| Not end with | Binary | Field string value does not end with a literal or field value. |
Logical field operators
Use logical operators to evaluate values in Boolean fields.
| Operator | Binary/Unary | Expression true when |
|---|---|---|
| Equal | Binary | Field value is equal either to a literal True or False value or to the value in another Boolean field. |
| Null | Unary | Field value is null. |
| Not null | Unary | Field value is not null. |