Gain insights into ML system design, state-of-the-art techniques, and best practices for scalable production. Learn from top researchers and stand out in your next ML interview.

Machine Learning System Design is an important component of any ML interview. The ability to address problems, identify requirements, and discuss tradeoffs helps you stand out among hundreds of other candidates. Readers of this course able to get offers from Snapchat, Facebook, Coupang, Stitchfix and LinkedIn. 

This course will help you understand the state of the practice on model techniques along with best practices in applying ML models in production at scale. Once you finished the course you can learn more use-cases at: http://mlengineer.io/

Once you're done with the course, you will be able to apply and leverage knowledge from top researchers at tech companies. You will have up to date knowledge in model techniques from hundreds of the latest research and industry papers. There is even a chance that the interviewers will be surprised at the depth of your knowledge.

I really found the quizzes very helpful for testing my ML understanding. Also, the resources shared helped me a lot for revising concepts for my interview preparation. This course will definitely help engineers crack Machine Learning Engineering and Data Science interviews

Senior Data Scientist at Amazon

I have been using your github repo to prep for my interviews and got an offer with NVIDIA with their data science team. Thanks again for your help!

Data Scientist at NVIDIA

I really like what you've built, it'll help a lot of engineers.

MLE at Facebook

It's well organized and the illustrations are well done. Being able to visualize and walk through the steps in order is really helpful in system design. The hints for quizzes is a nice addition, a hint is given if you get stuck in a real interview, mimicking how a real interview 

DS at Fortune 500

Andrew

I just heard back from the recruiter that I passed the Google L5 HC. Thank you very much for sharing the resources on GitHub and for the course on educative.io!

Google Machine Learning Engineer, L5

I got the offer from Intuit. Thanks so much, it would not have been possible without your help.

Senior Machine Learning Engineer, Intuit

I got Google, Facebook, Apple, Tesla, Cruise offer for Senior ML engineer. I thought the course is super helpful. 

Senior Machine Learning Engineer at Cruise

Machine Learning System Design

There are two stages, candidate generation, and ranking. The reason for two stages is to make the system scale.

> It's a common pattern that you will see in many ML systems. 

We will explore the two stages in the section below.

- The candidate model will find the relevant videos based on user watch history and the type of videos the user has watched.
- The ranking model will optimize for the view likelihood, i.e., videos that have high a watch possibility should be ranked high. It’s a natural fit for the logistic regression algorithm. 

## Candidate generation model

### Feature engineering

- Each user has a list of video watches (videos, minutes_watched). 

### Training data

- For generating training data, we can make a user-video watch space. 
We can start by selecting a period of data like last month, last 6 months, etc. This should find a balance between training time and model accuracy. 
 

### Model

- The candidate generation can be done by [Matrix factorization](https://en.wikipedia.org/wiki/Matrix_factorization_(recommender_systems)#:~:text=Matrix%20factorization%20is%20a%20class,two%20lower%20dimensionality%20rectangular%20matrices.). The purpose of candidate generation is to generate “somewhat” relevant content to users based on their watched history. The candidate list needs to be big enough to capture potential matches for the model to perform well with desired latency. 

- One solution is to use [collaborative algorithms](https://en.wikipedia.org/wiki/Collaborative_filtering) because the inference time is fast, and it can capture the similarity between user taste in the user-video space. 

> In practice, for large scale system (Facebook, Google), we don't use Collaborative Filtering and prefer low latency method to get candidate. One example is to leverage Inverted Index (commonly used in Lucene, Elastic Search). Another powerful technique can be found [FAISS](https://engineering.fb.com/2017/03/29/data-infrastructure/faiss-a-library-for-efficient-similarity-search/) or Google [ScaNN](https://ai.googleblog.com/2020/07/announcing-scann-efficient-vector.html).

## Ranking model

During inference, the ranking model receives a list of video candidates given by the Candidate Generation model. For each candidate, the ranking model estimates the probability of that video being watched.  It then sorts the video candidates based on that probability and returns the list to the upstream process. 


### Feature engineering
|**Features**   |**Feature engineering**   |
|:-: |:-: |
|<center>Watched video IDs </center>   |<center>Video embedding </center>  |
|<center>Historical search query</center>   |<center>Text embedding</center>   |
|<center>Location</center>   |<center>Geolocation embedding</center>   |
|<center>User associated features: age, gender</center>   |<center>Normalization or Standardization</center>   |
|<center>Previous impression</center>   |<center>Normalization or Standardization</center>   |
|<center>Time related features</center>   |<center>Month, week_of_year, holiday, day_of_week, hour_of_day.</center>    |

Read about [embedding](https://www.educative.io/courses/machine-learning-system-design/feature-selection-and-feature-engineering#4-embedding).

### Training data 
- We can use User Watched History data.  Normally, the ratio between watched vs. not-watched is 2/98. So, for the majority of the time, the user does not watch a video. 





### Model 
At the beginning, it's important that we started with a simple model, as we can add complexity later.

- A fully connected neural network is simple yet powerful for representing non-linear relationships, and it can handle big data. 

- We start with a fully connected neural network with **sigmoid** activation at the last layer. The reason for this is that the Sigmoid function returns value in the range [0, 1]; therefore it's a natural fit for estimating probability.

> For deep learning architecture, we can use **relu**, (Rectified Linear Unit), as an activation function for hidden layers. It's very effective in practice. 

- The loss function can be cross-entropy loss.


Learn about candidate generation and ranking of videos based on user preferences.

Machine Learning Primer

Video Recommendation

Feed Ranking

Ad Click Prediction

Rental Search Ranking

Estimate Food Delivery Time

Conclusion

Machine Learning Knowledge

Machine Learning Model Diagnosis

Candidate Generation and Ranking Model

3. Multi-stage models