| Authentication
method |
Select the authentication method to use for
connecting to Databricks. Two options are available:
- Regular: Uses standard token
authentication.
- Key Vault: Uses credentials stored
securely in a key vault for enhanced security.
|
| When key vault
authentication method is used, configure the following fields |
| Key vault setup |
Specifies the method used to authenticate with
Databricks which can be retrieved from a key vault:
- Key vault: Specifies the preconfigured
key vault to be used for retrieving the secret credentials.
- Authentication token secret path:
Specifies the path or identifier within the selected key vault
where the secret access key is stored. Example:
secret/aws/s3/accesskey.
Add key vault: Specifies the option to
configure and register a new key vault if one is not already
available in the list. For configuration steps, refer to key vault
setup.
|
| Processing Environment for
Cataloging |
Specifies the environment used for managing and
organizing data catalogs. It provides two options:
- Databricks Cluster: Select this option if
you are using a Databricks cluster for data processing and
cataloging. If this option is selected, also specify the
relevant Databricks cluster. Example: Cluster-1
- Catalog: Specifies name of the
catalog where data is stored and managed. The default
value is hive_metastore.
- Databricks SQL Warehouse: Choose this
option if you are utilizing a Databricks SQL Warehouse for data
management and querying, also specify the relevant Databricks
SQL warehouse. Example: SQLWarehouse-1.
- Catalog: Specifies name of the
catalog where data is stored and managed. The default
value is hive_metastore.
- Databricks Notebooks: Executes cataloging
operations using Databricks Notebooks, which are collaborative,
interactive environments within Databricks
|