Scheduling ensures that the configured Observers are executed at predefined times, automatically triggering alerts when any anomalies are detected. This functionality enhances data reliability by ensuring Observers run consistently as scheduled, while providing timely notifications to users through alerts whenever an anomaly is identified.
Considerations for scheduling
How frequently you want an Observer to run depends on your needs and the resources available to you. You can experiment with different settings to find an optimal schedule that meets your requirements.
- The size of your data: If you have a large data set, running an Observer daily may consume a lot of resources. A weekly schedule might be more efficient.
- The type of data: If you are monitoring critical data, you may want to run the Observer more frequently to detect anomalies as early as possible.
- The speciļ¬c use case: You may want to run the Observer on specific days or at specific times. For example, if you are monitoring sales data, you may want to schedule the Observer to run on Monday morning to analyze the previous week's sales data.
- The cost: Balance the benefits of more frequent monitoring with usage-based pricing models popular with cloud infrastructure vendors. Increased frequency may also result in more alerts you need to review.
Limitations of observing tables and fields
Consider these factors when observing tables and columns:
- Data Access: Not all users have access to observe all tables and fields. Make sure that users have the right permissions to observe the data they need.
- Data Volume: Observing large datasets can be resource-intensive and may require additional processing power or storage. It will also take more time to complete a successful run on large datasets.
- Data Privacy: Observing certain types of data, such as personal data, may require additional security measures to ensure compliance with data privacy regulations.
- Data Quality: Observing tables and fields with poor data quality can lead to inaccurate or unreliable results.
- Data Governance: Observing tables and fields may be restricted by the data governance policies of your organization.
Observer settings
| Occurrence | Description |
|---|---|
| Daily | Specifies settings for the scheduler to run every day. at: The time at which scheduling starts (hh:mm). Note: The scheduler uses
a 12-hour time format and shows the default system time during
selection.
Example: If the schedule is selected for daily at <hh:mm>, then profiling or observer will run daily at <hh:mm>. |
| Weekly | Specifies settings for the scheduler to run every seven days. On: The day(s) in the week that profiling or observer will run. at: The time
at which scheduling starts (hh:mm).
Note: The scheduler uses
a 12-hour time format and shows the default system time during
selection.
Example: If the schedule is selected on <selected days> and at <hh:mm>, then profiling or observer will run on the <selected days at <hh:mm> for every week. |
| Monthly | Specifies settings for the scheduler to run every
month. TheSelect dayof every month at<time>: The profiling or observer will run on the every selected date of the month. Example: If the schedule is selected as The12of every month at 11:00 AM, then profiling or observer will run on every 12th of the month at 11:00 AM. The<first or last><day>of every month at <time>: The profiling or observer will run on the selected date of the month. Example: If the schedule is selected for The firstWednesday of every month at 11:00 AM, then profiling or observer will run on the first Wednesday of every month at 11:00 AM. Note: The scheduler uses a 12-hour time format and shows the
default system time during selection.
|