Generalized Linear Regression for Multiple Targets

Multiple targets

Consider a regression dataset $D=\{(\bold x_1,\bold y_1),(\bold x_2,\bold y_2),\dots,(\bold x_n,\bold y_n)\}$, where $\bold x_i \in \R^d$ and $\bold y_i \in \R^m$. A function $f_W(\bold x) = W^T\phi(\bold x)$ is a generalized linear model for regression for any given mapping $\phi$ of the input features $\bold x$, and $W$ is a matrix with $m$ columns, one for each target. Note that $f_W(\bold x) \in \R^m$.

The optimal parameters $W^*$ can be determined by minimizing a regularized squared loss as follows:

$$W^*=\argmin_{W}\bigg\{\sum_{i=1}^n \|W^T\phi(\bold x_i)-\bold y_i\|_2^2 + \lambda \|W\|_F^2\bigg\}$$

Here, $\sum_{i=1}^n \|W^T\phi(\bold x_i)-\bold y_i\|_2^2 + \lambda \|W\|_F^2$ ...