This section provides you the details on various fields required to configure Databricks in Data Integrity Suite.
| Field | Description |
|---|---|
| Name | Specifies the name of data connection. This is a mandatory field. Example: SalesDataConnection |
| Description | Allows you to describe the purpose of this data connection. Example: Connection to the sales data warehouse for quarterly reporting. |
| Agent to access data | Select the agent from the dropdown that connects to the database. It helps securely access data from your network. |
| Host URL | Specifies the endpoint address of your
Databricks workspace. Ensure it is correctly
entered to establish a successful connection.
This is a mandatory field.
Example:
|
Operational Lineage |
Operational Lineage: Toggle to enable operational lineage harvesting on this connection. When enabled, the Data Catalog service harvests fresh lineage from your Databricks workspace on each catalog run. This requires Advanced Connectivity Bundle (ACB) entitlement. For more information, see Operational lineage harvesting overview. Lineage Mode: Select your harvesting mode (visible only when runtime lineage is enabled):
Lineage Window (Days): Enter the number of days to look back (visible only when "Specify Runtime Lineage Window" is selected). Minimum value is 1 day. Suggested values: 7, 30, 60, 90, 180, 365. |
|
Advanced Options |
Allows you to set the connector log level and define additional
parameters for the Databricks connection. Operational lineage query timeout: Runtime lineage harvesting queries have a default timeout of 3600 seconds (1 hour). If needed, you can override this value using a debug parameter: Key:
Value: Timeout duration in seconds. For example,
specify |
| For more information, refer; | |