Regularization: Ridge Regression and LASSO

The vector notation

We discussed earlier a linear problem with a one-dimensional input $x$. Machine learning problems often consist of high-dimension problems in the sense that the input is a vector with many dimensions. For example, if we want to undertake image processing, we would represent a gray-scale image as a list of many gray-level values, one for each pixel. Within linear regression, this means that we should introduce one parameter for each input dimension. Such a linear model would look like the following: