Data Quality transformation step settings allow you to define how data is processed in a pipeline. Each transformation step performs a specific operation on your data. The transformation steps are organized into nine categories:
| Category | Transform | Description |
|---|---|---|
| Structure | Split | Use Split to create multiple outputs in a Data Quality pipeline. |
| Union | Union combines data into a single row, appending mapped columns and eliminating duplicates. | |
| Join | Join relates different datasets in a Data Quality pipeline to enrich or compile records. | |
| Output | Configure and run each output branch in the Output step. | |
| General | Copy Field | Duplicates a column into new columns. |
| Filter Field | Deletes specific columns from a table. | |
| Filter Row | Filters rows based on a logical condition. | |
| Evaluate Rule | Assesses records against a predefined rule. | |
| Execute Formula | Calculates and adds results to a new column. | |
| Generate Key | Creates a unique identifier for each record. | |
| Custom Coding | Addresses complex scenarios in Data Quality Suite. | |
| LLM Transform | This enables AI-driven modifications such as categorization, translation, and structured data extraction on pipeline data. | |
| Make API Call | The Make API Call step enables seamless integration of external APIs into your data pipelines. | |
| Rename Field | Changes the name of a column. | |
| Split Field | Divides a column into two columns. | |
| String | Cleanse Data | The Cleanse Data step allows multiple string-cleansing operations in one transformation within your data quality pipeline. |
| Addressing | Geocode Address | The Geocode step converts addresses into latitude and longitude coordinates. |
| Identify Country | The Identify Country step determines the country from address fields. | |
| Verify Address | The Verify Address step processes and corrects address data using postal references. | |
| Parsing | Parse Email | This step breaks down personal and business names into components like given name, surname, and titles. |
| Parse Name | This step breaks down personal and business names into components like given name, surname, and titles. | |
| Parse Phone Number | This step parses, formats, and validates international phone numbers. | |
| Standardization | Standardize Date | Use the Standardize Date step to unify different date formats in a data column. |
| Standardize Field | The Standardize Field step performs standard conversions on semantically typed fields. | |
| Enrich | Enrich | The Enrich step offers detailed insights into weather, geological, and other natural risks for asset owners, enabling them to assess, decide, mitigate risks, control costs, and boost profits through data utilization. |
| Table Lookup | Table lookup is a transformation step that searches for matches in a reference dataset to enhance the data pipeline. | |
| Matching and Consolidation | Match and Group | This step matches and groups dataset records by entity values. |
| Consolidate Matches | The Consolidate Matches step merges inconsistent duplicates into a single consistent record or a new golden record. | |
| Spatial | Search at Point | The Search at Point step enriches data by searching a spatial dataset, currently exclusive to the Snowflake connection. |