Table lookup

Data Integrity Suite
Product
Spatial_Analytics
Data_Integration
Data_Enrichment
Data_Governance
Precisely_Data_Integrity_Suite
geo_addressing_1
Data_Observability
Data_Quality
dis_core_foundation
Services
Spatial Analytics
Data Integration
Data Enrichment
Data Governance
Geo Addressing
Data Observability
Data Quality
Core Foundation
ft:title
Data Integrity Suite
ft:locale
en-US
PublicationType
pt_product_guide
copyrightfirst
2000
copyrightlast
2025
Type: Enrich step

Table lookup is a transformation step to search matching fields in a reference dataset (also called as Lookup Dataset) and add fields to the pipeline based on the matching fields.

The matching criteria is defined by mapping the input data field (Pipeline Field) with the lookup dataset field (Table Field). You can add multiple fields to map. The lookup dataset is used to retrieve matching information and add additional fields to the input dataset.

Table lookup is a convenient way to add data fields to your existing dataset. Use matching fields, in both input and lookup datasets, and choose desired fields in output configuration to add them to the output data. To use this step,
  1. Select the Table Lookup step from the Enrich type step.
  2. Choose Match Fields.
  3. Select the Output configuration fields that you want to include in the output.
The step preview displays fields to be included in the output dataset when a match is identified. If the Pipeline Field does not match with the Table Field, the selected output field is added as a blank column in the output dataset.
Important:
  • Lookup Dataset must be a cataloged asset.
  • In case the mapped field has two different values, the system selects the first value as the reference. For example, the mapped field DeptCode has two values for 101 as HR and HumanResource . In this case, the first value selected as reference will be HR, considering the lookup dataset is sorted as low to high.
  • The dataset lookup supports all data types and is case-sensitive.

Usage example:

Consider a sample dataset (input dataset) as given below.
EmpId EmployeeName Office_Address Age Designation Dept_ID
4234855 John Smith Jr 5600 Cottle Rd Bldg 24 33 SoftwareDeveloper 101
4234930 William Bill 1400 S Gloster St 38 TechnologyArchitect 102
4231180 Mark Waugh 7 Barnett Plaza 47 Cybersecurity 103

Lookup dataset (reference dataset):

Consider the table below as the lookup dataset.
DeptCode DeptName
101 HumanResources
102 InformationTechnology

Match fields:

The fields mapped are as follows.
Pipeline Field Table Field
Dept_ID DeptCode

Step preview:

We have selected Dept_ID and DeptCode as match fields. The step preview displays the mapped pipeline fields and the selected fields in output configuration. Here, the Pipeline Field is Dept_ID and the field selected in Output configuration is DeptName.
Step Preview
Dept_ID DeptName
101 HumanResources
102 InformationTechnology

Output dataset:

When you select Save, the output dataset includes the input fields along with the selected fields from the output configuration. In the Output configuration, we have selected the output field as—DeptName.
EmpId EmployeeName Office_Address Age Designation Dept_ID DeptName
(Input field) (Input field) (Input field) (Input field) (Input field) (Input field/pipeline field) (Field selected in output configuration)
4234855 John Smith Jr 5600 Cottle Rd Bldg 24 33 SoftwareDeveloper 101 HumanResources
4234930 William Bill 1400 S Gloster St 38 TechnologyArchitect 102 InformationTechnology

Step properties

Specify Properties as below for transformation step.

Lookup Dataset: Select the cataloged reference dataset used to lookup mapped fields from the data catalog. In case the data sample is not available, you can upload or generate a new sample dataset.

Match Fields: Select the mapping fields from the input and lookup dataset.
  • Pipeline Field: Select the field from input dataset.
  • Table Field: Select the field from reference dataset or lookup dataset.

Add Field to Map: Adds additional fields to map pipeline and table fields.

Sort lookup dataset: Arranges the items in the selected match field. You can select the sort order as low to high or high to low.

Output configuration

Select fields that you want to include in the pipeline from the lookup dataset. You can either choose to include all fields or specific fields from the lookup dataset. When you save the transformation step, the selected lookup dataset fields under output configuration are added to the pipeline.

Output fields

The output dataset contains input dataset fields and the selected lookup dataset fields (under output configuration). The dataset field added to the output is blank when the match is not identified.