Gain insights into ML pipeline development, delve into best practices, discover advanced Python concepts, and explore testing methodologies to elevate your software engineering skills and career prospects.

Building a Machine Learning Pipeline from Scratch.png

docker_setup.tar.gz

omegaconf-code-widget

omegaconf-spa-widget

omegaconf-spa-widget-copy

omegaconf-code-widget-copy

omegaconf_example

dataset_model_examples

logging_example

docstring_example

pipeline_iris_classification

pipeline_autompg_regression

reporting

pipeline_packaging

inference

unit_testing

pipeline_unit_testing

experiment_tracking

argparse_example

Machine learning (ML) has matured into a mainstream development activity, and data scientists are expected to be able to write production-grade training pipelines. This course will provide you with a foundation in ML pipeline development guided by best practices in software engineering.

You’ll start by learning about code organization, style, and conceptual ideas, such as topological sorting of directed acyclic graphs. You’ll dive into the hands-on development of an ML pipeline. You’ll learn some advanced Python concepts and useful libraries. Finally, you’ll learn about testing methodologies and how to package your code into a distributable library.

By the end of this course, you’ll understand what makes a great ML pipeline work, better appreciate software engineering processes, improve your Python knowledge, and learn how to build ML training pipelines. This knowledge will make you more attractive to potential employers and improve your chances of thriving as a new data scientist.

Building a Machine Learning Pipeline from Scratch

**(True or False)** An object is a concrete realization of a class.

(True or False) An object is a concrete realization of a class.

**(True or False)** In the folllowing code, `FooDataset` is a child class of `Dataset`.
```python
class Dataset:
    def load(self):
        # ...

    def clean(self):
        # ...

    def save(self):
        # ...


class FooDatast:
    def clean(self):
        # ...
```

(True or False) In the folllowing code, <code>FooDataset</code> is a child class of <code>Dataset</code>.
<pre><code class="language-python">class Dataset:
 def load(self):
 # ...

 def clean(self):
 # ...

 def save(self):
 # ...


class FooDatast:
 def clean(self):
 # ...
</code></pre>

**(True or False)** There is nothing wrong with the following code:
```python
from abc import ABC, abstractmethod


class Dataset(ABC):
    @abstractmethod
    def load(self):
        # ...

    @abstractmethod
    def clean(self):
        # ...

    @abstractmethod
    def save(self):
        # ...


if __name__ == "__main__":
    dataset = Dataset()
```

(True or False) There is nothing wrong with the following code:
<pre><code class="language-python">from abc import ABC, abstractmethod


class Dataset(ABC):
 @abstractmethod
 def load(self):
 # ...

 @abstractmethod
 def clean(self):
 # ...

 @abstractmethod
 def save(self):
 # ...


if __name__ == &quot;__main__&quot;:
 dataset = Dataset()
</code></pre>

**(True or False)** A class can have no more than one mixin parent.

(True or False) A class can have no more than one mixin parent.

There is no problem, and it will run fine.

It will throw an error because `feed` is defined twice.

It will throw an error because the order of `Animal` and `AnimalMixin` is incorrect in the definition of `Deer`.

It will throw an error because `Deer` is an abstract class and can’t be instantiated.

Is there any problem with the following code?
```python
from abc import ABC, abstractmethod

class Animal(ABC):
    @abstractmethod
    def run(self) -> None:
        pass

    @abstractmethod
    def feed(self) -> None:
        pass

class AnimalMixin:
    def feed(self) -> None:
        print("Feeding...")

class Deer(Animal, AnimalMixin):
    def run(self) -> None:
        print("Running...")

if __name__ == "__main__":
    bambi = Deer()
    bambi.feed()
    bambi.run()
```

Is there any problem with the following code?
<pre><code class="language-python">from abc import ABC, abstractmethod

class Animal(ABC):
 @abstractmethod
 def run(self) -&gt; None:
 pass

 @abstractmethod
 def feed(self) -&gt; None:
 pass

class AnimalMixin:
 def feed(self) -&gt; None:
 print(&quot;Feeding...&quot;)

class Deer(Animal, AnimalMixin):
 def run(self) -&gt; None:
 print(&quot;Running...&quot;)

if __name__ == &quot;__main__&quot;:
 bambi = Deer()
 bambi.feed()
 bambi.run()
</code></pre>

Test yourself on the concepts learned in this chapter.

Introduction

Getting Started

Structuring the ML Pipeline

Directed Acyclic Graphs (DAGs)

The ML Library

The Pipeline Core

Extending the Pipeline

Testing

Deployment

Other Considerations

Wrapping Up

Appendix

Final Assessment

Quiz: The ML Library