Determine the prerequisites and the process of deciding how and when an observer should be created. This would help monitor data by ensuring accuracy, consistency and integrity.
Prerequisites
Before you create an observer, get familiar with the data you want to observe. Consider the patterns of your data and the types of anomalies that indicate potential problems. For example, would a 5% change in data volume be a concern to you, or would it take a change greater than 30% to require your attention.When you understand your data, you can choose which alert types are relevant to your data.
- Confirm that the data asset you want to observe is cataloged. You must set up a successful data connection and catalog your data asset before you can observe it.
- Determine if the data asset you want to observe is already used in another Observer. A data asset can only be used in one Observer.
- Make sure you have the necessary permissions to access and observe the data asset you want to observe, including the tables and fields.
- Review your data asset for tables or columns with single or double quotes in their names, like ‘TableName’ or “ColumnName”. Quoted database objects can cause Observer runs to fail. Consider renaming affected objects without quotes.
- Plan out which Observer rule configurations you want to use based on the unique data, use cases, and requirements of your organization.
- Determine how frequently you want the Observer to run, daily or weekly, based on the nature of your data assets, how often you need to analyze them, and any associated costs.
- Decide whether you want to receive alert notifications or Observer fail notifications via email.
- Ensure you are in compliance with data privacy regulations when observing certain types of data, such as personal data.