Scheduling automates the execution of data profiling tasks on a set schedule, such as daily, weekly, or monthly. It ensures regular monitoring of data quality, structure, and integrity by automatically running profiling processes at the specified intervals, helping maintain consistent oversight of datasets.Scheduling helps you to identify trends and generate alerts for data anomalies. It is a final setp in profile creation and is optional.
Enable profile scheduler
To schedule a data profile:
- Select the Scheduler status for the respective profile.
- Click the Enable Schedule toggle button to turn it ON.
- Review the Occurrence of scheduling.
- Click Close
- You are redirected to the Profilers page. The profile is saved and will run as per the selected schedule.
- Click Save & Run to save and run the profiling instantly. The profile will run as per the selected schedule settings.
Edit profiling scheduler
Edit the profile schedule to change the cadence of the profile run.
To edit a data profiling schedule:
- On the main navigation menu, click tab and click the ellipsis icon corresponding to the profile that you want to edit the schedule for and click Edit.
- On the Source section, click Next.
- On the Profile section, click Next.
- In the Finalize section, you can perform any one of the action below.
- Click the Enable Schedule toggle button to turn it ON (if the schedule is OFF).
- Edit the already setup schedule.
- Click the Enable Schedule toggle button to turn it OFF (if the schedule is ON).
-
Select the Occurrence of scheduling.
-
Click Save. The profile is saved and will run as per the selected schedule.
-
Click Save & Run to save and run the profiling instantly.
Disable profiling scheduler
You can stop the profile schedule if you don't want to run the profile on a regular cadence.
- On the main navigation menu, click tab.
- In the Scheduler column, click the corresponding Enabled status for which you want to disable the scheduler.
- Click the toggle button under Enable Schedule.
- The profile scheduler will be disabled.
Profiler scheduler settings
Profiler scheduler settings typically define when and how the profiler should run. It runs based on the rules configured and checks the data for statistics such as completeness, uniqueness and frequency distribution.
| Occurrence | Description |
|---|---|
| Daily | Specifies settings for the scheduler to run every
day. at: The time at which scheduling starts (hh:mm). Note: The scheduler uses a 12-hour time format and shows the
default system time during
selection.
Example: If the schedule is selected for daily At<hh:mm>, then profiling or observer will run daily at <hh:mm>. |
| Weekly | Specifies settings for the scheduler to run every seven
days. On: The day(s) in the week that profiling or observer will run. at: The time at which
scheduling starts (hh:mm).
Note: The
scheduler uses a 12-hour time format and shows the
default system time during
selection.
Example: If the schedule is selected On<selected days> and at<hh:mm>, then profiling or observer will run on the <selected days at <hh:mm> for every week. |
| Monthly | Specifies settings for the scheduler to run every
month. TheSelect dayof every month at<time>: The profiling or observer will run on the every selected date of the month. Example: If the schedule is selected as The12of every month at 11:00 AM, then profiling or observer will run on every 12th of the month at 11:00 AM. The<first or last><day>of every month at <time>: The profiling or observer will run on the selected date of the month. Example: If the schedule is selected for The firstWednesday of every month at 11:00 AM, then profiling or observer will run on the first Wednesday of every month at 11:00 AM. Note: The scheduler uses a 12-hour time format and shows the
default system time during selection.
|