This section provides detailed instructions on how to access and interpret the insights and scheduling information for a connection within Data Integrity Suite.
View insights of a connection
- Find the connection you want to view the details for.
- Click the connection name and select Insights tab to view insights details. The section includes following details:
- Pipeline Engine: Specifies the engine used for the selected connection from the dropdown menu.
- Profile datasets: Toggle the switch On to profile your data, including statistics, distributions, and semantic types.
Note: When Profiling datasets for On-Prem datasources, it is
recommended to have adequate executor memory for the default rules in the Catalog to
be executed successfully. It is recommended to at least have 1.5 GB of executor
memory to execute a dataset containing 50 columns. Whenever the count of the number
of columns is increased in the dataset, the corresponding executor memory has to be
increased for effective rule evaluation.
Refer to the table below for Executor Memory details:
| Number of columns | Executor Memory |
|---|---|
| 50 | 2 GB |
| 100 | 3 GB |
| 150 | 5 GB |
View schedule of a connection
- Find the connection you want to view the details for.
- Click the connection name and select Schedule tab to view scheduling details. The section includes following details:
- Occurrence: Specifies the scheduler frequency, which may be daily, weekly, or monthly.
- Schedule Summary: Specifies the summary of the selected scheduler frequency.
Edit schedule of a connection
-
Find the connection for which you want to edit the scheduling details.
- Click the connection name and select Schedule tab.
- Click Edit Schedule, to open the side panel. Adjust the scheduler settings according to your requirements.
- Click Save to save the changes, or Cancel to discard the settings.