airflow.tar.gz

LiveJobApp

Apache Airflow has become the dominant and ubiquitous Big Data workflow management system, leaving Oozie and other competitors miles behind in terms of features and functionality. 

In this course, you will learn the fundamentals of Apache Airflow starting with DAGs, DagRuns, Tasks and Task Instances - the building blocks of this popular technology. You’ll then move on to understanding the architecture, creating operators and sensors, and understanding XCom.

In the latter half, you’ll learn about branching, scheduling, and working with the UI. By the end of this course, you will have a new, highly coveted, and transferable skill to add to your resume.

An Introduction to Apache Airflow

Tasks get triggered when tasks upstream to them succeed. This is the default trigger behavior for tasks, but there are different variations that Airflow offers. Consider the following workflow:

This lesson explains how the trigger behavior of tasks changes with the different settings for the trigger_rule parameter. Additionally, the lesson also covers unexpected task failures.

__default

Task Triggers and Failures