The Data Integrity Suite helps you
explore and validate data across your datasources, datasets, and
fields. You can visualize score indicators and drill into details to
ensure your data meets quality and governance
standards.
Getting started:
- Go to Catalog and choose one of the
main tabs:
- Datasources: View and manage your datasources.
- Datasets: Explore the tables and data collections within your datasources.
- Fields: Examine the individual columns or attributes within your datasets.
- The Catalog offers two primary views:
Card view: Displays assets as tile cards with:
- Asset name
- Asset type
- Asset location
- Field count
- Score badges (Quality and Governance)
- Status indicator (such as, Certified, Draft, etc)
List view: Presents assets in a tabular format with sortable columns:
- Asset name (Datasource, dataset or field)
- Quality Score
- Governance Score
- Status
- Asset type
- Asset location (Hierarchical path: Database > Schema > Dataset > Field - for dataset and fields)
- Regardless of the view you choose, you can access detailed
information about each asset:
- In Card View: Click a card to open a quick detail panel on the right side of your screen. This panel provides a comprehensive view of the asset's description, type, full location, business terms, and a list of its fields.
- In List View: Score columns display
numeric percentages alongside color-coded
indicators. Colors map to score bands:
- Green: Good (e.g., 90-100%)
- Orange: Average or Caution (e.g., 80-90%)
- Red: Poor (e.g., 0-80%)
- The list view also include columns for Status (e.g., Certified, Draft, Under Review) and Datasource Type (e.g., Snowflake, Databricks, Amazon S3).
Locating assets with precision
To help you find the exact data
asset you need, especially in complex environments with many
similarly named items, there are two options:
- Full hierarchical asset paths: You will see the
complete path for each asset, showing its lineage from
the top level down to the specific item. This eliminates
ambiguity when multiple assets share the same name.
- Format:
Database > Schema > Dataset > Field(or relevant hierarchy for datasources or datasets). - Example: Instead of just seeing "Customer
Data," you might see:
- DWH > core_warehouse > curated > customer data
- HumanResources > customer_insights > customer data
- Display locations: This full path is displayed in search results, autocomplete suggestions, both Card and List views within the Catalog tabs, and in asset selection interfaces (like when creating a Quality Rule).
- Format:
- Asset filtering: Use the filters extensively, especially when dealing with common asset names, to quickly isolate the data you need. This allows you to focus precisely on the data relevant to your current task, significantly speeding up selection and reducing the chance of errors. To know more about filtering assets, refer to the following:
- Review selections: When selecting multiple assets, always review the Selected Datasets or Selected Fields panel on the right. This allows you to confirm your choices or remove any assets before confirming an action.
These features are integrated into critical areas of the Data Integrity Suite:
- Quality rule creation: When selecting fields or datasets for your quality rules, you'll see the full asset paths and can use database filters to pinpoint the exact data. The card view on the Create Quality Rule page also includes check-boxes that let you select multiple datasets.
- Pipeline configuration: When setting up pipeline inputs, lookup tables, or runtime parameters, leverage the hierarchical paths and filters to ensure you select the exact data assets required.
- General catalog browsing: Whether in the Datasources, Datasets, or Fields tabs, the enhanced display and filtering capabilities make it easier to navigate and understand your organization's data landscape.