Catalog allows you to assess the quality and trustworthiness of cataloged assets. When viewing datasources, datasets, or fields in Catalog, you can view the Quality Score indicator. The scores provide an aggregate measure of factors such as freshness, completeness, validity, accuracy and more. Higher scores indicate assets of higher quality that can be trusted for critical operations.
View quality scores
To view quality scores for a cataloged asset:
- Navigate to Catalog from the main navigation menu.
- Select the Datasources, Datasets or Fields tab depending on the type of asset you want to view the scores for.
- Quality scores are visible in both the card and list view for each
cataloged asset.
- In the card view, the score is shown prominently next to the asset name.
- In the list view, a Quality Score column has been added showing the scores for each row.
You can quickly scan for high-quality datasets or fields that meet your needs or identify lowers scoring assets that may need enrichment or replacement.
Access scoring page for an asset
The Scoring page lists the Quality Score Dimensions. It displays when you go to the main navigation menu and select Catalog. You can then select Datasources, Datasets or Fields tab to see their respective Scoring page.
To view the scores at the Datasource level:
- Navigate to .
- Click the datasource name from the list for which you want to view scores.
- Select Scoring tab. The scoring page displays the score along with the dimensions for each of the associated scores.
To view the scores at the Dataset level:
- Navigate to .
- Click the dataset name from the list for which you want to view scores.
- Select Scoring tab. The scoring page displays the score along with the dimensions for each of the associated scores.
- The dimension score displays the associated rules and the fields based on which the scores have been generated. The scores are generated for each associated rule and the respective fields.
- Clicking on the rule navigates to the scoring page which displays the rule score and rule definition.
- Clicking the field further displays the score for the respective fields.
To view the scores at the field level:
- Navigate to .
- Click the field name from the list for which you want to view scores.
- Select Scoring tab. The scoring page displays the score along with the dimensions for the rule that are associated with the respective fields.
Quality score dimensions
All the quality scores generated in the Data Integrity Suite are associated with a quality score dimension. These are the parameters based on which the aggregate score is generated.
- Completeness: Addresses whether all required data is present. Rules in this dimension assess whether essential information is available and not missing. For example, rules checking for null or blank values. In a field, if 98 out of 100 records are not null or blank values, then the completeness score is 98%.
- Validity: Checks if data conforms to predefined formats, standards, or constraints. Rules in this dimension ensure that data adheres to specific rules or guidelines. Depending on the rule condition, if it is 98% is the confidence of semantic type, then the validity score is 98%.
- Field Score: Provides the aggregate score of quality score dimensions.
Dimension Score
This parameter displays the scores associated with each of the dimensions. Dimension scores can be viewed at the Datasource, dataset and field level.
To view the dimension scores at the Datasource level:
- Navigate to .
- Select the required datasource from the list.
- Select the respective dimension to view the associated scores.
- On selecting a dimension, further details are displayed that include the name of the schema, the aggregate score and the date when the schema was last evaluated.
To view the dimension scores at the Dataset level:
- Navigate to .
- Select the required dataset from the list.
- Select the respective dimension to view the associated scores.
- On selecting a dimension, further details are displayed that include the name of the associated field, the aggregate score and the date when the field was last evaluated.
To view the dimension scores at the Field level:
- Navigate to .
- Select the required field from the list.
- Navigate to the Scoring tab.
- Select the respective dimension to view the associated scores.
- On selecting a respective dimension, further details are displayed that include the name of the rule, the aggregate score and the date when the rule was last evaluated.