Probabilistic Models

In this chapter, we will revise a linear model and linear regression to include a description of uncertainty in the data. This will show us how modern probabilistic machine learning can be formulated. We first follow a simple stochastic generalization of the linear regression example to introduce the formalism. This leads to the important maximum likelihood principle on which we will base our learning. We later generalize this idea to nonlinear problems in higher dimensions and relate this to Bayes’ nets. After this, we will discuss how such a probabilistic approach is related to deep learning.

Prediction and probability of values

We again consider supervised learning where examples of input-output relations are given, and our goal is to make a model that can make predictions of previously unseen data. The main difference is that we do not only want to make a prediction of a value, but we would also like to know how probable different values are.

Let’s consider an example from robotics where we want to model how far a terrestrial robot is moving when the wheels are turning for a given number of seconds after activating the corresponding motors with a certain power. Figure 1A below shows a Lego Mindstorm robot that has two motorized wheels and an ultrasonic distance motor attached to it. We want to model (or predict) how far this robot moves when both motors are driven for a certain amount of time.