SubDAGs
This lesson explains how complex workflows can be simplified and modularized using SubDAGs.
We'll cover the following
It is common to abstract and modularize when writing software. We create classes and libraries for other teams and developers to consume. In the Airflow world, as workflows get complex and multiple teams work on pipelines, we may need to abstract away some portions of the pipeline, which are wholly contained or maintained by other teams using SubDAGs.
SubDAGs
Consider the workflow below.
Get hands-on with 1200+ tech skills courses.