Integrating Precisely Data Integrity Suite with Databricks streamlines the management of data connections and enhances data integration capabilities. You can effectively unlock the full potential of both platforms, ensuring data integrity and streamlined data processes. This integration works for both AWS and Azure cloud platforms and offers several advantages.
To get started with the integration and establish connectivity between Databricks and Data Integrity Suite:
- Log into your Databricks account and navigate to Marketplace in the navigation menu.
- Scroll down to Partner Connect Integrations section and click View all.
- Type in Precisely in the Search by partner name field to see the available options.
- Within the Partner Connect menu, click on the Precisely Data Integrity Suite option to initiate the integration.
- For a new connection, select the Catalog you want to write to and click Next.
- The portal automatically selects a SQL warehouse for data management and querying. You can select a different warehouse from the available options.
- Click Start to initiate the run.
- Select the required schema from the list and click Add.
- Click Next to view the details on the resources that will be created by Databricks.
- Click Next to confirm the e-mail address and the associated connection details.
- Click Connect to Precisely Data Integrity Suite. This will take you to Data Integrity Suite sign up page.
- Precisely will check if you already have a workspace within Data Integrity Suite.
- New user: If you do not have a workspace, a new workspace will be automatically created.
- Existing user: If you already have a workspace, you will be redirected to the workspace.
- Once the integration is complete, your workspace will automatically establish a datasource with associated Databricks connection. To view and manage this connection in Data Integrity Suite, navigate to
Note:
- If a check mark appears on the partner tile in Databricks workspace, it indicates that a Workspace Administrator has previously set up this connection through Partner Connect.
- You must have the Workspace Administrator role or privilege to establish this integration.
- Once the connection is established, all schemas under the selected catalog in Databricks will be automatically synced and cataloged in Data Integrity Suite.
- Users with Datasource Manager role can edit or delete the connection. However, deleting a connection removes it, but any cataloged schemas will remain until manually removed.
- If an existing established partner connection is deleted in Databricks and an Admin tries to reconnect, the workspace will be returned and a new connection will be created in Data Integrity Suite.
Benefits of integration
The integration between Precisely and Databricks offers several advantages:
- Seamless access to Data Integrity Suite workspaces and its capabilities from Databricks.
- Simplified connection process, allowing for seamless data flows. This eliminates the need for complex manual processes, reduces the risk of errors, and saves time, thereby increasing overall operational efficiency.
- Precisely offers powerful features for users to enhance data validation, cleansing, and profiling activities within their workflows, leading to higher quality data and more reliable analytics outcomes.
- With advanced governance capabilities, the integration helps in better managing who has access to what data, under what conditions. This not only helps in protecting sensitive information but also in tracking data lineage, thereby enhancing overall data security.