Kafka - Precisely Data Integrity Suite

Data Integrity Suite
Product
Spatial_Analytics
Data_Integration
Data_Enrichment
Data_Governance
Precisely_Data_Integrity_Suite
geo_addressing_1
Data_Observability
Data_Quality
dis_core_foundation
Services
Spatial Analytics
Data Integration
Data Enrichment
Data Governance
Geo Addressing
Data Observability
Data Quality
Core Foundation
ft:title
Data Integrity Suite
ft:locale
en-US
PublicationType
pt_product_guide
copyrightfirst
2000
copyrightlast
2026

Set up Kafka as a data connection for continuous replication pipelines in the Data Integrity Suite.

Before you configure Kafka for continuous replication, ensure you have the following:

  • Kafka broker hostname or IP address
  • Kafka broker port number
  • Kafka vendor or provider information
  • Security protocol details (if applicable)
  • Schema registry connection (if using schema registry)
  • An agent configured to access Kafka

Kafka connection enables continuous replication and mainframe replication, facilitating real-time data transfer between source and target systems. Kafka acts as a messaging backbone for data integration pipelines.

  1. In the Data Integrity Suite, navigate to Replication connections.
  2. Click Add connection or select an existing Kafka connection to edit.
  3. Complete all Kafka configuration fields:
    Table 1. Kafka Configuration Fields
    Field Description
    Connection Specifies the name of data connection.
    Description Allows you to describe the purpose of the data connection.
    Connection type Select a connection from the dropdown list. When creating a new connection from a project, only connection types that can be used in that project type are present.
    Kafka vendor Specifies the list of vendor or provider of the Kafka messaging system.
    Configure connection for Select the replication mode. By default, the option 'Both' is selected. To configure the connection specifically for continuous replication, select the option 'Continuous replication'.
    Replication engine Specifies the replication engine to determine how data is replicated from the source to the target system.
    Note:
    • For continuous replication, only continuous replication engine can be configured.
    • For mainframe replication, only mainframe replication engine can be configured.
    • When a new replication engine is added, it appears as the default option from the dropdown menu
    Add Engine Opens the Add runtime engine window. When a new replication engine is added, it appears as the default option from the dropdown menu.
    Add Broker Adds another row to specify broker details such as Broker hostname or IP address.
    Note: Click (-) to remove the desired broker.
    Broker hostname or IP address Specifies the hostname or IP address associated with a Kafka broker.
    Port number Mention the port number to access the Kafka broker.
    Security Protocol Specifies the user security protocol for the connection. The available options in the dropdown are: Plaintext, SASL Plain, SSL, SASL SSL, and Kerberos. The fields displayed will vary depending on the selected security protocol.
    Use Schema Registry By default, the option is checked to ensure data consistency and compatibility in Kafka for replication connection.
    Schema registry connection Lists the schema registry connections configured with the same runtime engine as the Kafka connection configuration.
    Add Connection Allows to add a new schema registry data connection.
    Additional configuration parameters Allows you to specify extra settings that can customize the behavior of the Kafka client beyond the standard configuration options. These parameters enable you to fine-tune the connection to meet specific needs or requirements.
  4. Click Test Connection to verify that the connection is configured correctly.
  5. Click Save to save the Kafka continuous replication connection.

You've successfully configured Kafka for continuous replication. The connection is now available for use in continuous replication pipelines. You can now create or edit replication pipelines that use this Kafka connection as a source or target for continuous replication.