Explore how to ensure reliability in ML models. Gain insights into software testing, ML-specific techniques, runtime checks, and monitoring tools to build robust ML systems effectively.

MyAwesomeDocker.tar.gz

MyAwesomeDocker

TorchJob

spa_run

spa_run_python

mypy_example

Ensuring the reliability and robustness of machine learning models is essential to building successful ML-powered applications.

This course begins with a thorough introduction to software testing essentials, particularly use cases within the machine learning context. You’ll learn about topics related to software testing, including unit and integration testing and more advanced testing techniques. Next, you’ll learn the best practices in software testing and dive into ML-specific testing techniques, such as behavioral and smoke tests. Lastly, you’ll cover the aspects of ML software reliability outside of testing, including runtime checks and type hinting.

By the end of this course, you'll be equipped with the knowledge and skills to ensure the reliability and robustness of your machine learning systems. You’ll be able to apply software engineering principles to your ML processes, create and execute efficient testing approaches, and utilize monitoring tools to identify and resolve problems in your ML systems.

Reliable Machine Learning

# Overview

In this lesson, we discuss logging and debugging in machine learning applications. Logging always goes toe to toe with debugging because both are essential parts of troubleshooting.

**Logging** helps us understand what's going on with our program after it has launched. We can investigate logs either during the application run or retrospectively after the problem is already captured.

**Debugging** is a set of techniques that helps us effectively detect, localize, and fix troubles that have occurred in our code.



# Logging

After the problem is identified (it could be an alert from monitoring in the best case or a customer complaint in the worst), it's time to understand what happened. Such investigation will go much smoother if there are enough informative logs. 

Logging is a cornerstone of a monitoring strategy. If we're not sure whether we should log or not, we should log. We can always remove logs that we later realize are unnecessary, but can't retroactively add them back. Don't log if it doesn't add new context. So try to inject some specific details into the logging message.

Let's look at the example below. First, we'll look at a log with less context for the task performed. 

Learn how to understand what's going on with your code (logging) and how to investigate problems in it (debugging).

Introduction to Reliable ML

Software Testing

Best and Worst Practices

ML-Specific Tests

ML Software Reliability outside of Tests

Wrapping Up

Appendix

Logging and Debugging

Overview

Logging