Fix schema and data type errors after a data type mapping change - Precisely Data Integrity Suite

Data Integrity Suite
Product
Spatial_Analytics
Data_Integration
Data_Enrichment
Data_Governance
Precisely_Data_Integrity_Suite
geo_addressing_1
Data_Observability
Data_Quality
dis_core_foundation
Services
Spatial Analytics
Data Integration
Data Enrichment
Data Governance
Geo Addressing
Data Observability
Data Quality
Core Foundation
ft:title
Data Integrity Suite
ft:locale
en-US
PublicationType
pt_product_guide
copyrightfirst
2000
copyrightlast
2026

This article applies only to pipelines created before the data type mapping change. Pipelines created after the mapping update are not affected and require no action. You might encounter one of the following issues:

Schema Validation Error

When running a pipeline, you may receive an error similar to: Schema of source dataset and sample used in pipeline did not match. This typically occurs when using Append, Truncate, or Drop modes.

Data Type Validation Errors

Some transformation steps may display unexpected data type errors.

Step Example Error
Copy Column Expected Decimal, received Integer
Rename Column Correct decimal values are displayed, but the data type remains Integer
Evaluate Rule Expected Integer, received Decimal

Cause

After the data type mapping update, the source dataset and the sample stored in an existing pipeline can become out of sync.

Although re-cataloging updates the dataset metadata, older pipelines may still reference previous sample or cached data type information. This outdated metadata can cause schema validation or data type validation failures until the pipeline is refreshed.

Resolution

Scenario 1: Schema Mismatch During Pipeline Execution

If you receive a schema mismatch error:

  1. Re-catalog the dataset.
  2. Regenerate the sample.
  3. Run the pipeline again.

The pipeline should now execute successfully.

Scenario 2: Data Type Errors in Transformation Steps

After re-cataloging the dataset and regenerating the sample, some transformation steps may still retain outdated data type metadata. To refresh the metadata, temporarily select a different column or field, then switch back to the original value and save the step. These steps are still using the old data type:
  • Copy Column
    1. Right-click the step and select Edit.
    2. In Copy from, select a different column.
    3. Select the original column again.
    4. Click Save.
  • Rename Column
    1. Right-click the step and select Edit.
    2. In Column, select a different column.
    3. Select the original column again.
    4. Click Save.
  • Evaluate Rule
    1. Right-click the step and select Edit.
    2. Select Map Fields.
    3. Temporarily select a different field for any input.
    4. Select the original field again. Refreshing a single mapped field is sufficient.
    5. Click Apply.
    6. Click Save.

Other Transformation Steps (like matching step)

For any other affected transformation step:

  1. Open the step for editing.
  2. Temporarily select a different column or field.
  3. Re-select the original column or field.
  4. Save the step.

Summary

To resolve schema and data type issues caused by the data type mapping change:

  1. Re-catalog the dataset.
  2. Regenerate the sample.
  3. If a transformation step still displays a data type error, refresh the affected field by selecting a different value, switching back to the original value, and saving the step.

This updates the pipeline metadata and ensures the step recognizes the current data type definitions.