Migrating projects to support new features - Precisely Data Integrity Suite

Data Integrity Suite
Product
Spatial_Analytics
Data_Integration
Data_Enrichment
Data_Governance
Precisely_Data_Integrity_Suite
geo_addressing_1
Data_Observability
Data_Quality
dis_core_foundation
Services
Spatial Analytics
Data Integration
Data Enrichment
Data Governance
Geo Addressing
Data Observability
Data Quality
Core Foundation
ft:title
Data Integrity Suite
ft:locale
en-US
PublicationType
pt_product_guide
copyrightfirst
2000
copyrightlast
2026

This guide outlines the steps required to migrate existing projects when upgrading from versions earlier than Connect CDC 6.1.00.00 and Connect Hub 1.16.0. Projects created on or upgraded from these versions do not require migration.

Preview: This feature is currently in Preview for select workspaces and might be subject to change before general availability.

Prerequisites

Before beginning the migration, ensure that the dynamic-runtime-manager service is running on your agent.

Run the following command:

./discli agent status | grep 'dynamic-runtime-manager'

If the service is not running, make sure that you are on the latest operator version (follow operator update help section).

After ensuring the operator is on the latest version, run the following command to update the dynamic-runtime-manager certificates:

kubectl exec -it -n precisely-agent-operator-ns <operator_pod> -- sh -c '/home/scripts/ensure_dynamic_runtime_manager_certs.sh'

You can obtain the operator pod name using:

kubectl get pods –n precisely-agent-operator-ns
Note: Running the above script could restart the following Data Integrity components:
  • connect-hub – Monitors replication activity
  • connect-cdc – Handles continuous replication
  • cloud-applier – Applies data to cloud targets
  • sqdata-management – Supports mainframe replication
Ensure that restarting these services does not impact active operations in your environment.

Service updates

Before proceeding with project migration:

  • Upgrade connect-cdc and connect-hub to the latest supported versions by referring to the upgrade section.

If your environment includes BigQuery targets, configure the BigQuery region before proceeding (see Setting BigQuery region below).

Reset resources requests for tuned services (if applicable)

If you have previously applied resource tuning to the connect-cdc or cloud-applier pods, reset the Kubernetes requests values to their defaults before migrating projects. This ensures migrated projects can reserve the required resources.

For connect-cdc

Edit the StatefulSet using the appropriate command for your installation.

Agents installed in June 2026 or later

/usr/local/bin/k3s kubectl edit statefulset connect-cdc

Agents installed between February and June 2026

/usr/local/bin/k3s kubectl edit statefulset -n precisely-connect-cdc-ns connect-cdc

Agents installed before February 2026

sudo KUBECONFIG=/home/k3s/.kube/k3s.yaml /usr/local/bin/k3s kubectl edit statefulset -n precisely-connect-cdc-ns connect-cdc

In the resources section, change only the requests values back to their defaults.

requests:
    cpu: 50m
    memory: 500Mi

Do not modify any custom limits values.

For cloud-applier

Edit the StatefulSet using the appropriate command for your installation.

Agents installed in June 2026 or later

 /usr/local/bin/k3s kubectl edit statefulset connect-cloud-applier

Agents installed between February and June 2026

/usr/local/bin/k3s kubectl edit statefulset -n precisely-cloud-applier-ns connect-cloud-applier

Agents installed before February 2026

sudo KUBECONFIG=/home/k3s/.kube/k3s.yaml /usr/local/bin/k3s kubectl edit statefulset -n precisely-cloud-applier-ns connect-cloud-applier

In the resources section, change only the requests values back to their defaults.

requests:
    cpu: 100m
    memory: 2Mi

Do not modify any custom limits values.

Project migration procedure

Perform the following steps for each project labeled for migration:

  1. Log in to your workspace.
  2. Open the target project.
  3. Stop LogReaders (if applicable):
    • Select all pipelines.
    • Open the options menu (⋮) and click Stop All LogReaders.
    • Confirm the action.
  4. Allow pipeline backlogs to drain completely.
  5. Stop pipelines:
    • For cloud targets (for example, BigQuery or Snowflake): Wait an additional duration equivalent to the pipeline batch interval before stopping pipelines.
    • For non-cloud targets: Stop pipelines immediately.
  6. Verify backlog clearance (cloud targets only):
    • Follow the backlog verification steps in the next section.
  7. Stop all pipelines:
    • Select all pipelines and click Stop.
    • Confirm the action when prompted.
  8. Stop the project.
  9. Initiate migration:
    • Click Migrate at the top of the project page.
    • Select Apply to begin migration.
Note: After all projects have been migrated, verify the CPU and memory allocations for the connect-cdc and cloud-applierStatefulSets. If resource requests were reset before migration, ensure they remain at the default values unless additional tuning is required for your environment. Custom limits values do not need to be changed.

Verifying Cloud Applier Kafka backlog

To ensure that all data has been processed before migration, verify the Kafka backlog for Cloud Applier pipelines.

  1. Download the cloud_applier_backlog.sh script to your agent VM. Click here to get the script.
  2. Make the script executable: 
    chmod +x cloud_applier_backlog.sh
  3. Execute the script: 
    ./cloud_applier_backlog.sh

The script outputs a ranked summary of Kafka consumer lag, sorted by highest backlog.

Filtering results

To check a specific project:

./cloud_applier_backlog.sh | grep <project_name>
  • If the project does not appear, there is no backlog.
Note: Backlog results are most accurate when pipelines are stopped. Active pipelines may temporarily show residual Kafka control messages that are cleared upon shutdown.

Configure the BigQuery dataset region

If your environment uses BigQuery targets, configure the BigQuery dataset region before migrating projects. For instructions, see Configure BigQuery region.