Occam’s Razor
Get introduced to Occam’s Razor and compare bias-variance trade-off.
We'll cover the following
What is Occam’s Razor?
Occam’s Razor is credited to the English philosopher William of Ockham. It states that out of two or more competing theories, the simpler theory is to be preferred.
In this lesson, we’ll see this theory from the lens of machine learning. Occam’s Razor is intuitive and is easily understood when applied to real-life concepts. Similarly, we can say that a simple model should be preferred in machine learning.
What is a simple model?
A machine learning model is a parametric function, and the number of parameters is a quantification of its complexity. The number of parameters is directly proportional to the complexity of the model.
For example, let’s take a task for which we have two models and . This is shown in the figure below:
Get hands-on with 1400+ tech skills courses.