Leveraging your data starts with getting all of your important information in a single place where your data consumers can access it. With Data Integrity Suite, you can connect to critical datasources and catalog metadata seamlessly.
A datasource is a repository where data is stored and from which Data Integrity Suite retrieves and manages data. It can include databases, files, cloud storage, or APIs. The term connectors is often used interchangeably with datasources, as it refers to the components that establish the connection between applications and these data repositories.
A datasource contains datasets (collections of related data), fields (individual attributes or columns within those datasets), and connections (interfaces that allow the suite to access and interact with the datasource).
Datasources page
The Datasources page provides a centralized location to manage all your datasources. To access it, go to Configuration > Datasources in the main navigation menu.
For each datasource listed, the following details are available:
- Datasource: Name given to the datasource.
- Catalog Summary: Cataloging status of all connections in the
datasource:
- Failed with <number of errors> errors: An error occurred during cataloging. Click the status link to view details.
- Partially Discovered: Some tables within the schema are cataloged, but not all.
- Discovering: Cataloging is initializing.
- <number> Dataset: Schema is fully discovered and all metadata is available in the catalog.
- Not Discovered: Schema has not yet been cataloged.
- Datasets: Number of datasets associated with the datasource.
- Fields: Number of fields in the datasource.
- Connections: Number of connections in the datasource.
Use + Add Datasource to add a new datasource. You can also use the search bar to find a datasource by name or type. From the datasource list, you can open Details or Remove a datasource.
Connections page
A connection is the interface between Data Integrity Suite and a supported source or target database. It defines the credentials and other information required to access external data. After you establish a connection, you can catalog it to allow Data Integrity Suite services to access and catalog the associated database or data warehouse.
To view connections for a datasource, go to Configuration > Datasources and select a datasource. For each connection, the following details are available:
- Connection: Name of the data connection.
- Catalog Summary: Current cataloging status, updated automatically every 30 seconds:
- Failed with <number of errors> errors: An error occurred. Click the status link to view details.
- Partially Cataloged: Some tables have been cataloged, but not all.
- Cataloging: Cataloging is in progress.
- Starting: Resources needed to catalog are being initialized.
- Cataloged: Schema is fully cataloged and all metadata is available.
- Not Cataloged: Schema has not yet been cataloged.
- Datasets: Number of datasets associated with the connection.
- Fields: Number of fields associated with the connection.
- Host or URL: Database address or hostname/port. For Kafka connections, brokers are listed as a comma-separated list.
- Schedule: Indicates whether a catalog schedule exists for this connection.
Use + Add to add a new connection. You can also use the search bar to find a connection by name, hostname, or type. From the connection list, you can also Test, Duplicate, or Remove a connection.
Once a datasource or connection is established, the related datasets and fields are accessible from the Catalog section. A configured datasource is also available for data quality and observability tasks — enabling you to profile data, detect anomalies, monitor data freshness, and maintain robust data integrity.