Use the Data Quality Normalization and Standardization Agent to clean and prepare your data by making field values consistent and standardized across your dataset.
Overview
The Data Quality Normalization and Standardization Agent helps you clean and prepare your data by making field values consistent and standardized across your dataset.
This agent can perform the following tasks:
- Normalize fields (for example, country names, codes, city names)
- Standardize date formats
- Standardize and verify address fields
- Apply custom coding for advanced normalization (for example, metro/non-metro classification)
- Ensure data correctness against business rules
Using this agent improves data quality, prevents mismatches, and supports accurate analysis and downstream processing.
Accessing the Gio™ AI Assistant
- Open your Data Quality Pipeline in the Data Integrity Suite.
- Select or create a pipeline (for example, Property_Policy_Holders).
- In the pipeline canvas, click the Gio™ AI Assistant icon.
- Choose a conversation starter or type your request (for example, "I want to standardize the Date, Country, state, addresses fields in this dataset.").
Configuration and usage
Follow these steps to configure and use the agent:
Step 1: Start the agent
In the Gio™ AI Assistant panel, select one of the following options:
- Normalize Fields: For making field values consistent (for example, fixing typos, standardizing codes).
- Standardize Fields: For enforcing a single, correct format (for example, date formats).
- Enrich Spatial Data: For adding spatial information (optional).
Step 2: Enter your request
- Type your request in the input box (for example, "I want to standardize the Date, Country, state, addresses fields in this dataset.").
- The agent analyzes your dataset and suggests recommended steps.
Step 3: Review recommended steps
The agent displays a Recommended Steps flowchart and a detailed Steps and Reasoning table.
Example steps include:
- Standardize Field: Converts country values to 3-letter ISO codes, flags unmapped values.
- Standardize Date: Parses and outputs dates in a single
format (
YYYY-MM-DD), flags invalid or out-of-range values. - Verify Address: Standardizes and validates address fields for consistency and accuracy.
- Custom Coding: Normalizes city names, classifies metro/non-metro, flags unmatched entries.
Each step includes a reasoning column explaining its purpose.
Step 4: Add steps to pipeline
- Click Add Steps to insert the recommended steps into your pipeline.
- A confirmation message appears: "Step has been successfully added to the pipeline."
Step 5 (Optional): Add more steps
You can ask the Gio™ AI Assistant to add further steps, such as:
- "Help me to add a step to ensure correctness of the data against business rules."
The agent will create a new suggestion, including the extra steps and the earlier recommendation (for example, additional standardization or validation).
Example use cases
- Country Normalization: Converts "INDIA", "india", "IND" to "IND" (ISO3), flags unmapped values.
- Date Standardization: Converts "15/03/1985", "22-07-90",
"1988/11/08" to
YYYY-MM-DD. - Address Verification: Standardizes and validates address fields for matching and enrichment.
- Business Rule Validation: Adds steps to check data correctness against defined business rules.