The Complete Pipeline

We now have the following components of our pipeline:

• The pipeline core

  • Argument parsing

  • Artifacts and their versioning

  • Logging

• The ML library

  • The dataset module

  • The model module

  • Report generation

We need two more pieces of code before we can run the complete pipeline. Both of these conform to the factory design pattern.

The factory design pattern

In software engineering, a design pattern is a particular way to solve a frequently encountered problem. This will become clear when we discuss the factory design pattern and how it applies to datasets and models in our pipeline.

We’ve seen the abstract base class Dataset, from which we derived IrisDataset. We can use this class directly in our code, as shown below.