Data volume is defined by the number of rows in the data. Unexpected changes such as the addition or deletion of rows in the data, can be a cause for concern. You can configure a volume rule to generate alerts for these unexpected changes.
Configure a confidence-based alert
Confidence is a measure of how certain we are about the detected change in volume. You can configure confidence-based volume alerts to generate an alert when the certainty of volume change is within your specified range.
- After you select the data assets to observe on the Create Observer page, click Next.
- In the Volume row, click the gear icon in the Configure column.
- Select the confidence level for generating an alert:
- Drag the lower value slider and higher value slider to set a range of percentages or
- Enter the lower and higher value in the text box to set the range of percentages.
- Enter the minimum change in value to generate alerts. A change in value is calculated form the current and previous run. For textual data, the change in specified in number of characters, for all other datatypes it is calculated as a percentage change.
- Click Save.
For example, set the default yellow range of 60% to 80% to get warning alerts when the confidence of volume change is detected in that range. You will get critical alerts when the certainty above 80%. And you will get no alerts when the certainty is below 60%.
- The alerts will not be generated for lower values if the percentage overlap, the alerts will be generated only for values beyond the higher range.
- It will set to a default range if your range overlaps at 100%.
For example, alerts will not be generated below 80% for the range selected as 80% to 80%. Alerts will be generated only above 80%. If you set the range as 100% to 100%, the range will revert to default range as 60% to 80%.
Configure a threshold-based alert
Threshold-based alerts are customized limits you set to trigger alerts when the volume of data changes by a certain percentage. For example, a threshold range set at 3% to 8% of data volume means that alerts will generate if the volume of data increases or decreases by 3% to 8%.
Use these steps as you create an Observer to configure a volume rule with a threshold-based alert. You can also edit existing Observer rules.
- After you select the data assets to observe on the Create Observer page, click Next.
- In the Volume row, click the gear icon in the Configure column.
- Click the Threshold-Based Alerts radio button.
- Select the value against which you want to calculate volume change:
- Moving Average: Uses the specified data points,
which are the number of profile runs to detect any changes in the data
volume. You indicate how many profile runs you want to include in the
moving average.The default number of profiles for moving average is
7.
For example, if the data points are set to 7, the moving average will be calculated after 7 profile runs, to detect the change in data volume.
- Last Profile Run: Uses the last profile run to detect any changes in the data volume.
- Moving Average: Uses the specified data points,
which are the number of profile runs to detect any changes in the data
volume. You indicate how many profile runs you want to include in the
moving average.The default number of profiles for moving average is
7.
- Enter the lower and higher value in the Alert Levels text boxes to indicate what data volume percent change should generate an alert. The default range is set as 3% to 8 %.Warning alerts are generated if the change in data volume is between 3% and 8%. A change in data volume greater than 8% will generate a critical alert. No alerts will generate for a change in data volume that is less than 3%.
- Click Save. Tip: The moving average can filter noise in your alerts. It identifies the change in data volume based on the average of collected data points. And you can set the data points based on the frequency of the schedule.
Examples:
- If the scheduler is set to daily, you can set the data points to a maximum number. Because the scheduler is set to run daily, it will analyze the data after every last run and generate an alert when there is a significant change in the data volume.
- If the scheduler is not set to run frequently, you can set the data points to a minimum number. Because the scheduler is running less frequently, it will take longer to collect more data points.
The last profile run uses the latest run to identify the data volume change. This reduces noise in the alerts.