Configure pipeline engines

Data Integrity Suite
Product
Spatial_Analytics
Data_Integration
Data_Enrichment
Data_Governance
Precisely_Data_Integrity_Suite
geo_addressing_1
Data_Observability
Data_Quality
dis_core_foundation
Services
Spatial Analytics
Data Integration
Data Enrichment
Data Governance
Geo Addressing
Data Observability
Data Quality
Core Foundation
ft:title
Data Integrity Suite
ft:locale
en-US
PublicationType
pt_product_guide
copyrightfirst
2000
copyrightlast
2025

Create a pipeline engine to be able to run your Pipeline. It is selected when you are setting up the Pipeline run configuration. The supported connection types for Pipeline Engines are Databricks, Google Dataproc, and Snowflake. This procedure describes how to add a pipeline engine to run pipeline configurations.

Settings for a pipeline engine are specific for a connection type.
  1. Click the Pipeline Engines tab.
  2. Click + Create Pipeline Engine.
    This expands the Create New Pipeline Engine settings panel.
  3. In the Pipeline engine name field, enter the name to identify the pipeline engine.
  4. In the Type dropdown, select the datasource type on which the engine will operate.
  5. Configure options for the selected datasource type.
  6. Click Create.

Edit existing pipeline engine

A pipeline engine is specified or created when you create a run configuration. This procedure describes how to edit settings for an existing pipeline engine.

  1. Click the Pipeline Engines tab.
  2. In the Name column, click the ellipsis for the pipeline engine you want to edit, then click Edit.
    This expands the Edit Pipeline Engine settings panel.
  3. Configure Edit Pipeline Engine settings.
    For more information, see Pipeline Engine settings.
  4. Click Save.

Delete pipeline engine

You can delete an existing pipeline engine from a pipeline if it is not currently used in any run configuration.

  1. Click the Pipeline Engines tab.
  2. In the Name column, click the ellipsis for the pipeline engine you want to delete, then click Delete.
  3. In the Delete Pipeline Engine prompt, click Delete.