Goodness of Fit and Information Criteria

Motivation

When developing a time series model, we want to ensure that the model fits the training data. The extent to which it fits the data is what we call the goodness of fit of the model. Calculating a goodness-of-fit metric also allows us to compare and rank models among an array of possible specifications. There are many ways to test whether or not a model fits its training data in a satisfactory way. In this lesson, we will study two famous goodness-of-fit metrics: the Akaike information criterion (AIC) and the Bayesian information criterion (BIC).

In this context, the term “information” refers to the notion that these criteria involve a trade-off between the complexity of the model and the amount of information gained by fitting it to the data. In other words, models with more parameters tend to fit their training data better but at the risk of overfitting it. Therefore, both the AIC and the BIC try to strike a balance between explaining as much variation in the training data as possible and not having a lot of parameters.

Akaike information criterion

The Akaike information criterion (AIC) is defined as follows: