Inference in graphical models

We use graphical models to represent the relation between complex variables with the help of a graph structure. Inference in graphical models is the process that requires the observed variables to compute and find the probabilities of unknown or unobserved variables in the model. As you might know, graphical models represent a structure between the variables, where nodes act as variables and edges represent the dependencies among those variables. So inference in graphical models helps efficiently compute the probabilistic behavior of the variables.

To perform inference in graphical models, several methods are illustrated below.

In this Answer, we will discuss these individually.

Marginal inference

Marginal inference means finding the marginal probabilities of a variable or subsets of the variables. It only considers the probability distribution of the variables of interest and neglects other variables.

The mathematical equation for the marginal probability is calculated by taking the sum of specific variables and fixing the other variables. Consider the graph shown below.

There are three variables $A$, $B$ and $C$.

In this example $A$ and $B$ influences $C$ . However, $A$ and $B$ do not have a direct influence on each other. To calculate the marginal probability of variable $C$, we sum out the unwanted variables $A$ and $B$from the joint probability distribution. The equation of the marginal inference for the graph is below.

$P(C=True) = Σ_A Σ_B P(C=True|A,B) * P(A) * P(B)$

This equation gives us the probability of $C$ given $A$ and $B$, because $C$ is dependent on $A$ and $B$.

Maximum a posteriori inference

Maximum a posteriori (MAP) is a commonly used statistical technique used in graphical models to find the inference. With the help of observed evidence, it gives us the most probable inference for the variables. In MAP, the main goal is to infer the values of the variables that maximize the likelihood of the observed variables.

Consider an example where $W$ represents the variables of interest and $X$ represents the observed evidence. In mathematical terms, it is represented as:

$MAP(W | X) = argmax P(W | X)$

This inference technique is commonly used in classifying objects after observing the features of the classes.

Note: MAP inference assumes that a variable is independent and identically distributed and the observed evidence is accurate and complete.

Conditional probability queries

We compute the variables' probability distribution in a conditional probability query, given the other variables' observations. To understand this concept in detail, let us consider an example in which variable $A$ influences $B$ and $B$ influences $C$.

Following are the steps to perform conditional probability in the graph given above.

• The joint probability of all the variables is given by $P(A, B, C)$

• In this case, $A$ is set to true, and the variable$B$is summed out to obtain the conditional probability of $C$ , given the observations on $A$.

The resulting equation of conditional probability queries is:

$P(C | A=true) = \frac{Σ_B P(C=true, B, A=true)}{ P(A=true)}$

Joint probability queries

In this technique, we compute the probability of multiple variables simultaneously present in the graph. It captures the dependencies of the variables. Consider the graph as we have in conditional probability queries. The equation given below calculates the graph's joint probability.

$P(A, B, C) = P(A) * P(B|A) * P(C|B)$

In joint probability, we multiply the probability according to the relation and dependencies specified in the graph.

Time series and dynamic inference

In time series and dynamic, the state of the variable is dependent over time. It considers the dynamic relationship between variables at different time steps and predicts the future based on the observation. The hidden Markov model is the commonly used graphical model used for time series and dynamic inference.

Take an example of weather forecasting, where we predict the weather based on temperature over time. Our hidden state at each step is the weather condition (sunny, cloudy, rainy) at every time step, and the observed variable is the temperature reading.

As shown in the graph below, the observed variable $O$ is correspondence to the state $S$ and every hidden state is dependent on the previous hidden state.

In this way, we can infer the weather situation based on the temperature's history. Dynamic inference is required where sequential prediction is required, such as speech recognition, natural language processing, and financial market analysis.

Conclusion

Inference in graphical models helps us in making probabilistic decisions. It calculates the probabilities of variables in the graph in the presence of previous evidence or observations. Most importantly, it gives us valuable insights into the complex data.

Now time for a quiz!