Scheduling

When initially working with Airflow, it is common to get confused with how all the scheduling parameters work together. In this lesson, we’ll explore the differences between the various parameters.

When creating a DAG, we can specify the start_date and a schedule_interval as parameters to the constructor. Let’s see an example:

dag = DAG(
    'Example9',
    default_args=default_args,
    description='Example DAG 9',
    schedule_interval='@daily',
    start_date=datetime(2020, 9, 5))

In the Example9 DAG, we set start_date to 5th Sept 2020, and the schedule_interval is set to @daily. Note that @daily is an alias for the 0 0 * * * crontab expression. There are other aliases for commonly used schedules, such as @weekly, @monthly, and @yearly, which all translate to crontab expressions under the hood. You can provide a crontab expression for the schedule_interval parameter for complex schedules. A good resource to work with crontab expressions is crontab.guru. Remember that Airflow works with UTC by default but can ...