Metrics

Let's look at the online and offline metrics used to judge the performance of the recommendation system.

We'll cover the following

In this lesson, you will look at different metrics that you can use to gauge the performance of the movie/show recommendation system.

Types of metrics

Like any other optimization problem, there are two types of metrics to measure the success of a movie/show recommendation system:

  1. Online metrics

    Online metrics are used to see the system’s performance through online evaluations on live data during an A/B test.

  2. Offline metrics

    Offline metrics are used in offline evaluations, which simulate the model’s performance in the production environment.

We might train multiple models and tune and test them offline with the held-out test data (historical interaction of users with recommended media). If its performance gain is worth the engineering effort to bring it into a production environment, the best performing model will then be selected for an online A/B test on live data.

Level up your interview prep. Join Educative to access 70+ hands-on prep courses.