The Precisely Data Catalog is a powerful service for data-driven decision-making. It is a centralized repository of data assets and metadata that can be reused across different modules in Data Integrity Suite for replication, cleansing, enrichment, and observation. It helps you organize and manage your data assets in easy-to-understand formats that are accessible to all users. A business-ready catalog enables users to search for data, trace its origins, and understand its journey through the data supply chain without the complexity of technical terminology.
Catalog configuration guide
- Access catalog: Log in to the Precisely Data Integrity Suite and navigate to the Data Catalog service.
- Connect datasources: Use the provided connectors to link your various data sources. Ensure all relevant datasources are connected for comprehensive metadata collection.
- Configure metadata collection: Set up automated metadata collection. Define the frequency and scope of metadata collection according to your needs.
- Integrate technical metadata: Add technical metadata to ensure data accuracy and consistency.
- Define catalog rules: Establish and enforce rules to standardize metadata practices.
- Manage and monitor catalog jobs: Consistently manage and monitor catalog jobs and address any issues or errors promptly.
- Understand asset scores: View and evaluate asset scores to support informed decision-making.
- Sort and filter assets: Utilize sorting and filtering tools to improve the discoverability of your data assets.
- Organize data elements: Effectively organize datasources, datasets, and fields within the catalog.
Catalog assets overview
To view all your cataloged assets, go to Catalog. By default, information is displayed in card and list views, sorted alphabetically by asset name. For detailed insights into specific assets in the Data Integrity Suite, you can drill down further within each tab:
- Datasources: Items representing external or internal data sources. View connections with details such as datasource name, number of datasets, and fields present.
- Datasets: Logical groupings of data extracted from sources.
View dataset name, location, associated datasource, and number of
fields. Hover over the Location name to see the
asset path in the format
Datasource>Schema>Dataset. You can also upload or generate sample data for a dataset to create a Data Quality pipeline. - Fields: Individual data attributes within datasets. View
field name, associated datasource name, and field type. Hover over the
Dataset name to see the asset path in the
format
Datasource>Schema>Dataset>Field. - Technical assets: View technical asset types, descriptions, and asset count for each type. Click an asset type name to create or manage technical assets for that type.
- Jobs: View jobs history to see the status and details of the cataloging process, and troubleshoot any issues that occur.
Example: A SalesDB datasource monitors sales transactions and comprises three datasets featuring fields such as SaleID, ProductID, Quantity, and SaleDate. Among these, the SalesRecords dataset, which contains four fields, is essential for organizing pertinent information for validation and reporting. Each dataset includes the SaleID field, which acts as a unique identifier for every sale, crucial for preventing duplication and ensuring accurate data relationships. Additionally, business assets such as the Sales Report offer a comprehensive overview of operations, while technical assets such as the Data Warehouse function as a central repository for integrated data. Users can implement rules such as the Sales Amount Check to ensure that sales amounts are not negative, and manage tasks through jobs such as Job ID 002, which is currently in progress for data validation within the SalesDB datasource. Collectively, these elements work together to uphold data integrity and facilitate informed decision-making.
Required permissions for Catalog features
The following table provides a high-level overview of the actions you can perform from the assigned permissions in the Catalog section. It outlines the permissions needed for actions such as viewing, managing, or deleting assets and related capabilities within this service.
| Feature | Permissions | Actions |
|---|---|---|
| Catalog | View | View the details of all the fields and datasets cataloged, technical assets, and cataloged jobs |
| Profile | View profiling details associated with the datasets and fields | |
| Metrics | View scores and measurements about your data | |
| Catalog Meta Model | View | View the assets cataloged in the workspace. |
| Manage | Edit the details of the cataloged assets |
Metadata management
Metadata management in the Data Integrity Suite involves the systematic handling of metadata to ensure data quality, compliance, and efficient data usage. It encompasses the collection, management, and discovery of metadata across various enterprise systems.
Catalog captures metadata from connected datasources, including hosts, connections, schemas, and datasets; so you can discover, browse, and understand your data assets. Metadata is collected automatically from datasources and curated by users for reuse.
The Data Integrity Suite includes three core metadata management features:
- Data Catalog: An inventory of enterprise data assets that helps you discover available data, see relationships between assets, and understand their quality.
- Business glossary: Stores definitions of business terms, KPIs, business dimensions, and other metadata linked to specific data assets.
- Data Lineage: Shows how data flows through your ecosystem, including source systems, destination systems, and derived data assets.
You can also catalog database views metadata. When a connection is cataloged, you'll see database schema tables and views, and can harvest metadata from both to understand lineage between them.
By integrating metadata across data replication, data quality, and data observability scenarios, the Data Integrity Suite simplifies metadata collection and enables automated data mapping, performance monitoring, and lineage tracking.
Catalog spatial data
Spatial data must be cataloged before performing spatial analytics, such as using spatial operations in Data Integrity Suite pipelines. For example, you may have a spatial table representing store catchment areas boundaries, which you can utilize in a pipeline to assign a store ID to each customer record. Data Integrity Suite catalog process supports cataloging of spatial data tables in Snowflake. Both Snowflake spatial data types are supported:
- GEOGRAPHY data type, modeling Earth as a perfect sphere with coordinates in latitude and longitude using the WGS84 spatial reference system (EPSG:4326).
- GEOMETRY data type with coordinates projected to a flat surface as x and y coordinates pairs, often in meters. This data type can support any spatial reference systems defined by the EPSG (European Petroleum Survey Group). For more information, see https://epsg.org/home.html.