Alerts are generated based on the observer rules you configure. When you create an Observer, you define rules that monitor your data assets for unexpected changes. The system evaluates these rules during each profile run and generates alerts when anomalies are detected based on your configured thresholds and conditions.
- Alert type: The system generates four types of alerts based on your observer rules: Freshness, Volume, Data drift, and Schema drift.
- Alert level: Each alert is classified as Critical or Warning based on the severity level you configure in your observer rules.
- Confidence percentage: The system calculates a confidence level for each alert, showing the likelihood that the detected change is a genuine anomaly.
- Significance thresholds: Configure minimum change values that must be exceeded before an alert triggers. This reduces false positives and ensures only meaningful changes generate alerts. The system compares current profile run data against historical patterns and previous runs to identify deviations exceeding your thresholds.
Confidence based alert generation
- By default, critical alerts trigger when certainty exceeds 80%. You can adjust warning (yellow) alerts from 20% to 100%.
- Below 20%, confidence is too low and generates excessive alert noise.
- For example, set warnings at 60% to 80% to alert when failure confidence falls in that range. Critical alerts trigger above 80%, and no alerts generate below 60%.
- Alerts don't generate for lower values if percentages overlap. Alerts generate only for values beyond the higher range.
- If your range overlaps at 100%, it reverts to the default 60% to 80% range. For example, if you set the range to 80% to 80%, alerts generate only above 80%. If you set 100% to 100%, the range reverts to the default 60% to 80%.
Rule-specific behavior: See the individual rule topics (Freshness, Volume, Data Drift) for how confidence based alerts apply to each rule type.
Threshold based alert generation
- threshold based alerts always generate as critical alerts.
- You define custom thresholds that trigger alerts when breached.
- For Volume rules, you can use either a moving average (based on multiple profile runs) or the last profile run (based solely on the most recent run).
- The system compares current profile run data against your configured thresholds to determine if an alert should trigger.
Rule-specific behavior: See the individual rule topics (Freshness, Volume) for how threshold based alerts apply to each rule type and specific configuration options.