An Observer is configured with a set of rules that you define and apply to a data asset. The Observer monitors your selected data assets and provides alerts based on the Observer rules you choose.
For example, you could create an Observer, if written out, would say: “On a daily basis, check whether any columns have been added to or deleted from the table called Customer_Cleansed_Table. If you find such a change, create a critical alert and send that alert to Admin@ITDepartment.com.”
An Observer includes the schedule for observation, the dataset you want to observe, the rules you want to apply, when you want to generate an alert, the severity of the alert, and the individuals who should receive notification of any alert.
The observers page
Access the Observers page by clicking on the main navigation menu. On this page, you can create a new Observer, view, edit or delete an existing Observer.
Consider these key fields and areas on the Observers page.
- Search: Type the complete or partial name of an Observer, to see results that match your text string.
- Name: Shows the name of an Observer. Click the ellipsis icon to Edit or Delete an observer. You can click the observer name to view associated alerts and assets as well as profile information.
-
Scheduler: Shows the status of the Observer schedule.
- Enabled: The Observer is scheduled. Click to view Observer run history.
- Suspended: The scheduler stopped due to ten consecutive failures. To investigate the issue or enable the schedule, click the Suspended status.
- Unavailable: The scheduler service is unavailable and cannot provide scheduled Observer run history.
-
Alerts: Shows alerts and warnings (if any) for the corresponding
Observer.
- Red : Critical alert. The digit represents the number of critical alerts.
- Yellow : Warning alert. The digit represents the number of warning alerts.
- Green : No alert. A zero indicates there are no alerts.
- Created By: Shows the name of the user who created the Observer.
- A bell icon without strikethrough: You are subscribed to email notifications
- A bell icon with strikethrough: You are unsubscribed to email notifications
- A dimmed bell icon with or without strikethrough: Email notifications are not turned ON for this Observer, or the maximum subscription limit of 25 emails has been reached.
- Create Observer: Allows you to create a new Observer.
Steps to perform end-to-end data observation:
- Identify the data source you want to observe.
- Set up and catalog a connection for the data source.
-
Create an Observer.
- Choose the cataloged data assets to observe.
- Define Observer rules.
- Identify the types of rules to apply (freshness, volume, data drift, schema drift).
- Select parameters for each rule.
- Choose a schedule for running the Observer.
- Identify who receives alerts.
- Run the Observer (automatic).
- Receive alerts via email based on your rules.
- View alert details.
- Act on alerts to resolve issues.