ARMA and ARIMA Processes

AR($p$) and MA($q$) models are practical specifications to model time series, but they are hardly applied in isolation. It is more common to use them as building blocks of slightly more complex processes: the ARIMA and the ARMA. In this lesson, we will dive into the characteristics of these processes.

Definition

ARMA and ARIMA processes are almost identical. Whether we use one or the other is a matter of our data’s order of integration. Let’s see why.

ARMA

The acronym ARMA stands for Autoregressive Moving Average. In a way, the acronym delivers exactly what it promises: the ARMA process is just a combination of an AR($p$) and an MA($q$):

Therefore, when modeling a time series, $y_t$, as an ARMA($p$,$q$), we assume that up to $p$ lags of the series have an impact on the present value of $y_t$. Similarly, we also assume that up to $q$ past, random shocks also have a direct impact on $y_t$. To see how the two parts of the model come together, let’s take the example of an ARMA($1$,$1$). Mathematically, we would define it as:

Just like in the case of the AR($p$) model, modeling a time series as an ARMA($p$,$q$) will only work as long as the data is stationary. The reason is also the same. Otherwise, the process will either explode (for $\phi > 1$) or wander randomly (for $\phi =1$). This takes us to the other type of model that we are introducing today, the ARIMA.

ARIMA

Much like its ARMA cousin, the acronym ARIMA stands for Autoregressive Integrated Moving Average. The difference between ARMA and ARIMA processes, as we can see from the definition, is integration. ARIMA processes are used to model integrated data as a combination of an AR($p$) and an MA($q$) component.

We can think of modeling a time series as an ARIMA process as a sequence of two steps.

1. We differentiate the series as many times it is necessary to make it stationary. This is a parameter of the model. It is called the order of integration. We usually refer to it as $d$.

2. We estimate the AR($$ ...