Explore how to ensure reliability in ML models. Gain insights into software testing, ML-specific techniques, runtime checks, and monitoring tools to build robust ML systems effectively.

MyAwesomeDocker.tar.gz

MyAwesomeDocker

TorchJob

spa_run

spa_run_python

mypy_example

Ensuring the reliability and robustness of machine learning models is essential to building successful ML-powered applications.

This course begins with a thorough introduction to software testing essentials, particularly use cases within the machine learning context. You’ll learn about topics related to software testing, including unit and integration testing and more advanced testing techniques. Next, you’ll learn the best practices in software testing and dive into ML-specific testing techniques, such as behavioral and smoke tests. Lastly, you’ll cover the aspects of ML software reliability outside of testing, including runtime checks and type hinting.

By the end of this course, you'll be equipped with the knowledge and skills to ensure the reliability and robustness of your machine learning systems. You’ll be able to apply software engineering principles to your ML processes, create and execute efficient testing approaches, and utilize monitoring tools to identify and resolve problems in your ML systems.

Reliable Machine Learning

## Overview

When to add new tests? At which stage of the project and development cycle? The standard approach assumes that new tests should be introduced after finding a defect. 

Each new production defect highlights the lack of existing tests, so we need to add one to verify that the fault is solved and will not occur again.

Another case is when we may need to add new tests right before we start to refactor our code. Covering our code with tests, we freeze its behavior with all its current use cases and guarantee that everything works before refactoring. 

It's especially applicable when we're dealing with legacy code. In this case, writing tests makes sure that we understand how current components of the existing codebase work.

# Overview

When to add new tests? At which stage of the project and development cycle? The standard approach assumes that new tests should be introduced after finding a defect. 

Each new production defect highlights the lack of existing tests, so we need to add one to verify that the fault is solved and will not occur again.

Another case is when we may need to add new tests right before we start to refactor our code. Covering our code with tests, we freeze its behavior with all its current use cases and guarantee that everything works before refactoring. 

It's especially applicable when we're dealing with legacy code. In this case, writing tests makes sure that we understand how current components of the existing codebase work.

Learn when to add new tests and get familiar with test-driven development.

Introduction to Reliable ML

Software Testing

Best and Worst Practices

ML-Specific Tests

ML Software Reliability outside of Tests

Wrapping Up

Appendix

Test-Driven Development

Overview