Multiple Regression

Understanding multiple regression

Multiple regression is an extension of simple linear regression. It is defined as a linear regression with multiple features. It is the most widely used machine learning technique. This technique is useful across many domains as we cannot depend on one single variable to do effective modeling. Now, let’s formulate an equation for multiple linear regression. We have a feature set X, which consists of p features.

We can present the equation as

$$y_i= w_0 +w_1x_{i1}+ w_2x_{i2}+ w_3x_{i3} + ... + w_px_{ip}+ Error$$

Instead of using the features directly, we can use transform features.

$$y_i= w_0 +w_1T(x_{i1})+ w_2T(x_{i2})+ w_3T(x_{i3}) + ... + w_kT(x_{ik})+ Error$$

Here, $T(x_{ij})$ is the $i_{th}$ record and $j_{th}$ transformation is applied. Transformation could be anything, including as higher order polynomial, logarithm, exponential, etc.

Consider these examples:

$f(x) = 25 + 35x_1^2 + 12log(x_2)$

$f(x) = 3 + 4x_1 + 12x_2^4$