Troubleshoot quality errors - Precisely Data Integrity Suite

Data Integrity Suite
Product
Spatial_Analytics
Data_Integration
Data_Enrichment
Data_Governance
Precisely_Data_Integrity_Suite
geo_addressing_1
Data_Observability
Data_Quality
dis_core_foundation
Services
Spatial Analytics
Data Integration
Data Enrichment
Data Governance
Geo Addressing
Data Observability
Data Quality
Core Foundation
ft:title
Data Integrity Suite
ft:locale
en-US
PublicationType
pt_product_guide
copyrightfirst
2000
copyrightlast
2026

Quality refers to data accuracy, completeness, consistency, and reliability. Troubleshooting quality issues focuses on identifying and correcting data errors, anomalies, and inconsistencies. This can include profiling data, applying validation rules, and using quality metrics. Learn how to resolve common quality issues when you use Data Integrity Suite.

Snowflake queries are canceled automatically

After prolonged running of Snowflake queries, automatic query cancellation can occur. While running a pipeline, client sessions can detach from long-running queries for various reasons, such as network issues, timeouts, or service restarts. This detachment can lead Snowflake to cancel long-running queries automatically. The quality pipeline or job fails with a UI status of "Canceled".
  • Error message: SQL execution cancelled
  • Workaround: To prevent this issue, in Snowflake, set the parameter ABORT_DETACHED_QUERY to FALSE for the user running the pipeline. For instance, if the user is DJOHN, use this command to set the parameter to false: alter user DJOHN set ABORT_DETACHED_QUERY=FALSE

    By default, this parameter is set to FALSE in Snowflake. An admin typically enables or disables it at the account level. If it is set to TRUE, override it and set it to FALSE to avoid cancellation of long-running queries.

Existing pipeline does not open

While on the Pipeline list page, opening an existing pipeline fails. The pipeline cannot find the sample data that was used to build the pipeline. The sample data cannot be found because it was deleted or expired. The existing quality pipeline fails to open and shows an error message.

Error message: Unable to find sample file

A pipeline job fails when any of the following conditions exist:

  • The source dataset schema does not match the pipeline input schema.
  • The target dataset schema does not match the pipeline output schema.
  • The connection specified in the run configuration is deleted. The connection must exist when the job runs.
  • The pipeline engine specified in the run configuration is deleted. The pipeline engine must exist when the job runs.
Workaround: You must re-upload or regenerate the sample data that was used by the existing pipeline. This will allow you to open the existing pipeline and retain the steps in the newly generated sample for the existing pipeline.
Warning: Verify that all referenced connections and pipeline engines exist before running a job to prevent failures.

Snowflake pipeline run fails

While running a quality pipeline with a run configuration that uses the Snowflake pipeline engine, the run can fail with an error. The status is displayed as Failed on the Quality > Job page. Click the row of the failed pipeline to view the Details panel. The error message is displayed when you click Show details at the bottom next to the failed status.
  • Error message: Unload to named, user, and table stages has been disallowed.
  • Workaround: To prevent this issue, in Snowflake, set the parameter PREVENT_UNLOAD_TO_INTERNAL_STAGES to FALSE for the user running the pipeline. For instance, if the user is DJOHN, use this command to set the parameter to false::alter user DJOHN set PREVENT_UNLOAD_TO_INTERNAL_STAGES=FALSE

    By default, this parameter is set to FALSE in Snowflake. An admin typically enables or disables it at the account level. If it is set to TRUE, override it and set it to FALSE to avoid pipeline failures during runs.

Spark job out-of-memory on Precisely Agent

Pipeline job runs can throw an Out of Memory error. When pipeline jobs run with the Spark connector on Precisely Agent, an unexpected Out of Memory error can occur and cause job failure.

Workaround: Add the following keys within the Spark properties to the pipeline engine to provide additional buffer memory, which helps mitigate Out of Memory exception scenarios. Set the buffer memory value in the Spark connector to at least 0.2.

Spark provides several configuration options to fine-tune memory settings. These properties control memory overhead beyond heap size. This is important for handling off-heap memory, task run overhead, and other internal memory usage.

For example, if the executor memory allocated is 3GB, setting a buffer memory value of 0.4 in the Spark property allocates an additional 1.2GB of memory, supporting successful job runs.

Keys to be added within the Spark properties section to the pipeline engine:

spark.driver.memoryOverhead
spark.driver.memoryOverheadFactor
spark.executor.memoryOverhead
spark.executor.memoryOverheadFactor

Rule run failure messages

Review common rule run failure messages and their explanations.

When a rule run fails, an error message appears with the run status. Use the following table to understand common error messages and likely causes.
Error message Explanation
Handler type is invalid This error means Quality attempted to evaluate a rule, but the specified handler (function or action) was incorrect, missing, or unrecognized. Verify that the handler name and configuration are correct in your setup.
Invalid strategy This indicates that Quality received a strategy (method or run approach) that is unsupported or unrecognized. Double-check that the strategy name is valid and available in your configuration.
Unable to convert to JSON Quality tried to read or generate JSON data, but the format was invalid or improperly structured, which prevented conversion. Review your input files or request payload for correct JSON formatting.
No strategy accepts execution request This means Quality could not find a valid strategy (method) to process the current request. Ensure that your rule or workflow specifies a supported strategy.
No asset found corresponding to asset group This indicates that Quality could not locate any asset linked to the specified asset group, so no processing could occur for that group. Confirm that the asset group exists and contains appropriate assets.
Error occurred during client service invocation This error occurs when Quality tries to access or call a client service and the operation fails. Check the client service configuration and network connectivity.
Start task parsing error Quality failed to interpret or read the task details accurately, so the task could not be started. Verify that task definitions are complete and correctly formatted.
One or more params is missing in request body This means the request to Quality lacked required parameters and could not be processed. Make sure all required parameters are included in your API call or payload.
Metrics Publish Failure Quality attempted to record or send metrics data, but the operation failed. Check metric publishing settings and ensure external systems are reachable if metrics are sent externally.
Entity not found while creating pipeline This error indicates Quality could not locate a required entity (such as a resource, object, or configuration) needed to create a pipeline. Review your pipeline setup and ensure all referenced entities exist.
Schema drift dataset id is not available Quality could not find the dataset ID needed to check schema drift, so the operation cannot proceed. Make sure the dataset ID is provided and accessible.
Error during metric collection in metric Engine This error occurs when Quality encounters issues while collecting or recording metrics through its metric engine. Confirm that metric collection settings are correct and all required services are operational.
The pipeline engines are not available This indicates that Quality has no pipeline engines available for evaluating quality rules. Check engine status and resource allocation in your environment.
Request to run pipeline failed Quality attempted to start or run a pipeline, but the request was unsuccessful, and the pipeline did not run. Review pipeline configuration and ensure required resources are accessible.
Error while creating the pipeline Quality encountered an issue while setting up or configuring a pipeline, resulting in failed creation. Investigate pipeline setup details and verify all parameters are correct.
Error during pipeline run This means Quality experienced a problem during evaluation of pipeline tasks or stages, which prevented successful completion. Possible causes include missing or invalid input data, unavailable resources, or runtime exceptions. Check the generated logs for more details.
Error during cancel pipeline run Quality tried to stop or cancel a running pipeline, but the operation failed. This can happen if the pipeline already completed, the system could not access the running instance, or an internal error prevented cancellation.
Job cancelled This means a pipeline job in Quality was running but was intentionally stopped before completion. Reasons can include manual cancellation by a user, automatic cancellation due to system rules (such as time limits or resource constraints), or upstream errors in the pipeline.
PIPELINE_START_STATE_MISSING_ERROR This indicates that Quality cannot start the pipeline because the initial state is missing or undefined. The engine needs a valid starting point to evaluate pipeline tasks, so ensure this is configured in your workflow.

Pipeline validation errors

Use this section when a quality pipeline shows Invalid status or step-level warning, error, or data issue symbols.

Inspect step and pipeline validation errors

Use this procedure to inspect warning, error, and data issue details on individual transformation steps and across the full pipeline.
  1. Go to Quality > Pipelines, and find the pipeline that shows Invalid in the Status column.
  2. In the Name column for the invalid pipeline, click the ellipsis, then click Edit to display the pipeline and its sample data in the pipeline editor.
  3. On the pipeline panel, look for steps that show a warning, error, or data issue symbol on the step.
    • To view errors, warnings, or data issues for a transformation step, click the warning, error, or data issue symbol on the step to open a pop-up box with the step name and descriptions of warnings, errors, or data issues for that step.
    • To review all warnings, errors, and data errors across the pipeline, click the box in the top-right corner of the pipeline pane that shows error, warning, and data error counts.
    This expands a panel that identifies and describes errors, warnings, and data errors for all steps in a pipeline. For data issues, you can browse forward and backward to highlight corresponding cells in the sample data table.

Resolve pipeline step data errors

Recognize and resolve data errors that occur during pipeline validation.

Issues such as verification failures, invalid types or formats, or inability to parse data can cause data errors. A data error symbol displays on any step that generates a data error. When you click a step with a data error, red bars mark the columns and rows in the table that contain the errors. Fields that contain errors are highlighted.
  1. Look for data error symbols on pipeline steps.
  2. Click the step with a data error to view the affected data.
  3. Hover the pointer over a highlighted cell to view the cause of the data error.
  4. Proceed iteratively through successive edits and validations to eliminate all errors and warnings from a pipeline.
  5. Add additional transformation steps if necessary to fix data errors.
Data errors are identified and resolved, allowing the pipeline to execute successfully.