The rule run status indicates the current state of rule execution for all rules in the Data Integrity Suite. The status is essential for tracking the progress and effectiveness of rule execution. It allows you to identify errors or failures that occur during rule execution and troubleshoot them.
- Completed : Indicates that the rule was successfully evaluated.
- In progress : Indicates that the process of rule evaluation is initiated.
- Partially completed : Indicates that rule was partially evaluated.
- Failed : Indicates that the rule evaluation failed.
- On the main navigation menu, click .
- Click on the run status associated with each of the rules to view further details.
- The details displayed vary depending on the value displayed in the run status.
Completed
The completed status indicates that all the datasets or fields that were part of the rule were evaluated successfully and there were no exceptions during rule evaluation. Clicking on the status of execution displays the number of fields or datasets that were evaluated with the following details:
- Dataset or Field : Displays the name of the datasets or fields considered for rule evaluation.
- Score : Displays the score generated for each of the datasets or fields.
- Last evaluated : This column displays the timestamp when the rule was last evaluated.
In progress
In progress indicates that the rule execution was initiated but is still in progress. Clicking on the status of rule execution displays the run status along with the details of the associated datasets or fields.
- Completed : The completed tab displays the details of the fields that were evaluated successfully.
- Failed : Displays a count of the fields that were not evaluated along with an error summary of all the associated datasets or fields. All the errors are grouped by datasets. Clicking view details provides further details on all the associated datasets or fields
- In-progress : Displays a count of fields or datasets that are still under evaluation along with their details. All the fields are grouped by datasets in this tab.
Partially completed
Partially completed indicates that the rule was partially evaluated. It signifies that a few datasets or fields were evaluated successfully and a few others failed during rule execution. Clicking on the status of execution displays the run status along with the timestamp when the rule was last evaluated.
The partially completed run status displays the details in two tabs.
- Completed : The completed tab displays the number of datasets or fields that were evaluated successfully out of the total number of available assets. The details include the name of the dataset or field, the generated score and the timestamp when the dataset or field was last evaluated.
- Failed : The failed tab displays the number of datasets or fields that failed during rule evaluation. The error summary provides details on all the errors generated. Clicking view details in the error displays the cause for failure along with the details of the associated dataset and the connection.
Failed
The failed status indicates that none of the fields or datasets that were part of the rule were evaluated successfully. Clicking on the failed status in the rules page displays the run status of the rule and the date when the rule was last evaluated.
- It displays a count of the number of fields or datasets that were not evaluated during rule execution.
- The error summary section displays all the errors generated along with the details of the associated datasets or fields.
- Clicking view details provides further details on the cause for rule failure. Additionally, the details of the associated dataset and catalog are displayed in the error details.
When a rule evaluation fails, an associated error message appears alongside the rule run status. This message explains why the rule evaluation didn't succeed. Below are some of the most common error messages you may encounter in the Data Integrity Suite. Keep in mind that the specific error messages displayed can vary depending on which datasource is being evaluated by the rule.
| Error Message | Explanation |
|---|---|
| Handler type is invalid | This error means Data Integrity Suite attempted to evaluate a rule, but the specified handler (the function or action) was either incorrect, missing, or not recognized by the application. Verify that the handler name and configuration are correct in your setup. |
| Invalid strategy | This indicates that Data Integrity Suite received a strategy (a method or approach for execution) that is not supported or recognized. Double-check that the strategy name is valid and available in your configuration. |
| Unable to convert to JSON | Data Integrity Suite tried to read or generate JSON data, but the format was invalid or improperly structured, preventing conversion. Review your input files or request payload for correct JSON formatting. |
| No strategy accepts execution request | This means the Data Integrity Suite couldn't find a valid strategy (method) to process the current request. Ensure that your rule or workflow specifies a supported strategy. |
| No asset found corresponding to asset group | This indicates that Data Integrity Suite couldn't locate any asset linked to the specified asset group, so no processing could occur for that group. Confirm that the asset group exists and contains appropriate assets. |
| Error occurred during client service invocation | This error happens when Data Integrity Suite tries to access or call a client service and the operation fails. Check the client service configuration and network connectivity. |
| Start task parsing error | Data Integrity Suite failed to interpret or read the task details accurately, so the task could not be initiated. Verify that task definitions are complete and formatted correctly. |
| One or more params is missing in request body | This means the request to Data Integrity Suite lacked required parameters, and could not be processed. Make sure all necessary parameters are included in your API call or payload. |
| Metrics Publish Failure | Data Integrity Suite attempted to record or send metrics data, but the operation failed. Check metric publishing settings and ensure external systems are reachable if metrics are sent externally. |
| Entity not found while creating pipeline | This error indicates Data Integrity Suite couldn't locate a required entity (such as a resource, object, or configuration) needed to create a pipeline. Review your pipeline setup and ensure all referenced entities exist. |
| Schema drift dataset id is not available | Data Integrity Suite couldn't find the dataset ID needed to check for schema drift, so the operation cannot proceed. Make sure the dataset ID is provided and accessible. |
| Error during metric collection in metric Engine | This error occurs when Data Integrity Suite encounters issues while collecting or recording metrics through its metric engine. Confirm that metric collection settings are correct, and all required services are operational. |
| The pipeline engines are not available | This indicates that Data Integrity Suite has no pipeline engines available for evaluating data quality rules. Check engine status and resource allocation in your environment. |
| Request to run pipeline failed | Data Integrity Suite attempted to start or run a pipeline, but the request was unsuccessful, and the pipeline did not run. Review pipeline configuration and ensure required resources are accessible. |
| Error while creating the pipeline | Data Integrity Suite encountered an issue while setting up or configuring a pipeline, resulting in a failed creation. Investigate pipeline setup details and verify all parameters are correct. |
| Error during pipeline run | This means Data Integrity Suite experienced a problem during the evaluation of pipeline tasks or stages, preventing successful completion. Possible causes include missing or invalid input data, unavailable resources, or runtime exceptions. Check the logs generated for more details. |
| Error during cancel pipeline run | Data Integrity Suite tried to stop or cancel a running pipeline, but the operation failed. This could happen if the pipeline had already completed, the system couldn't access the running instance, or there was an internal error preventing cancellation. |
| Job cancelled | This means a pipeline job in Data Integrity Suite was running but was intentionally stopped before completion. Reasons may include manual cancellation by a user, automatic cancellation due to system rules (such as time limits or resource constraints), or upstream errors in the pipeline. |
| PIPELINE_START_STATE_MISSING_ERROR | This indicates that Data Integrity Suite cannot start the pipeline because the initial or starting state is missing or undefined. The engine needs a valid starting point to begin evaluating pipeline tasks, so ensure this is configured in your workflow. |