Overview of Probabilistic Models

Explore generative and discriminative models for classification and regression within the realm of probabilistic modeling.

In the context of supervised learning, it’s more advantageous to predict the distribution of the target variable It is the variable that we want to predict using the rest of the dataset. rather than a specific fixed value. In fact, numerous classification and regression models implicitly calculate the underlying distribution and generate a fixed target by sampling from this distribution.

Note: A probability distribution is a mathematical function that describes the likelihood of different outcomes occurring in a random experiment. It provides a way to model and quantify the uncertainty associated with various events or values.

The most common approach involves sampling the value with the highest probability, although alternative sampling methods also exist. However, certain models don’t lend themselves easily to estimating a probability distributionIt refers to the process of determining the shape and characteristics of a probability distribution based on available data. for the target variable. In situations where obtaining samples of the target variable proves difficult, we turn to conventional machine learning models as an alternative solution.

Press + to interact

Bayes’ rule

We can express the computation of the target probability distribution of a target variable yy given an input feature vector x\bold{x}, represented as p(yx)p(y|\bold{x}), using Bayes’ rule in the following manner:

p(yx)=p(xy)p(y)p(x)p(y|\bold x) = \frac{p(\bold x|y)p(y)}{p(\bold x)}

Here, p(yx)p(y|\bold x) represents the probability of event yy occurring given that event x\bold x has occurred. This is called the posterior probability. p(xy)p(\bold x|y) is the probability of event x\bold x occurring given that event yy has occurred and is called the likelihood. p(y)p(y) is the prior probability of event yy occurring before any new evidence is taken into account, and p(x)p(\bold x) is the probability of event x\bold x ...