Candidate Generation and Ranking Model
Explore the two-stage process of candidate generation and ranking in video recommendation systems. Understand how feature engineering and models like matrix factorization and neural networks combine to deliver scalable and personalized video suggestions by estimating watch likelihood.
3. Multi-stage models
Architecture diagram for the video recommendation system
There are two stages, candidate generation, and ranking. The reason for two stages is to make the system scale.
It’s a common pattern that you will see in many ML systems.
We will explore the two stages in the section below.
- The candidate model will find the relevant videos based on user watch history and the type of videos the user has watched.
- The ranking model will optimize for the view likelihood, i.e., videos that have high a watch possibility should be ranked high. It’s a natural fit for the logistic regression algorithm.
Candidate generation model
Feature engineering
- Each user has a list of video watches (videos, minutes_watched).
Training data
- For generating training data, we can make a user-video watch space. We can start by selecting a period of data like last month, last 6 months, etc. This should find a balance between training time