Explore how to ensure reliability in ML models. Gain insights into software testing, ML-specific techniques, runtime checks, and monitoring tools to build robust ML systems effectively.

MyAwesomeDocker.tar.gz

MyAwesomeDocker

TorchJob

spa_run

spa_run_python

mypy_example

Ensuring the reliability and robustness of machine learning models is essential to building successful ML-powered applications.

This course begins with a thorough introduction to software testing essentials, particularly use cases within the machine learning context. You’ll learn about topics related to software testing, including unit and integration testing and more advanced testing techniques. Next, you’ll learn the best practices in software testing and dive into ML-specific testing techniques, such as behavioral and smoke tests. Lastly, you’ll cover the aspects of ML software reliability outside of testing, including runtime checks and type hinting.

By the end of this course, you'll be equipped with the knowledge and skills to ensure the reliability and robustness of your machine learning systems. You’ll be able to apply software engineering principles to your ML processes, create and execute efficient testing approaches, and utilize monitoring tools to identify and resolve problems in your ML systems.

Reliable Machine Learning

# Overview

Mocking and patching are techniques used to isolate certain parts of a system under test and control their behavior. They are useful because they allow us to test the behavior of our code under different scenarios and verify that it behaves correctly in those scenarios.

**Mocking** involves creating a fake or mock version of an object or module used in our system under test. This mock object is designed to mimic the behavior of the real object, but it can be controlled by the test to return specific values or to raise exceptions.

**Patching** involves replacing the behavior of a real object or module with a mock version at runtime during the execution of a test. This allows us to test how our code behaves when it interacts with a specific version of an object or module rather than relying on the real object or module.

Both mocking and patching are useful tools for testing complex systems as they allow us to isolate and control specific parts of the system to test the behavior of the code under different conditions. They are often used with unit and integration testing to ensure that our code is correct and reliable.

# Example

When developing a complex integration test, we may face multiple problems caused by the environment. For example, an external service may respond slowly or be unavailable. This is where mocking is helpful.

Let’s look at an example of using mocks and patches below.


Learn about mocks and patches and when to use them.

Introduction to Reliable ML

Software Testing

Best and Worst Practices

ML-Specific Tests

ML Software Reliability outside of Tests

Wrapping Up

Appendix

Mocks and Patches

Overview