Kernel Logistic Regression

We can kernelize logistic regression just like other linear models by observing that the parameter vector $\bold w$ is a linear combination of the feature vectors $\Phi(X)$, that is:

$$\bold w = \Phi(X) \bold a$$

Here, $\bold a$ is the dual parameter vector, and, in this case, the loss function now depends upon $\bold a$.

Minimzing BCE Loss

We need to find the model parameters ($\bold a$) that result in the smallest BCE loss function value to minimize the BCE loss. The BCE loss is defined as:

$$\begin{align*} L_{BCE}(\bold{a})&=\sum_{i=1}^n L_i\\ L_i &= -(y_ilog(\hat y_i)+(1-y_i)log(1-\hat y_i)) \\ \hat{y}_i&=\sigma(z_i)=\frac{1}{1+e^{-z_i}} \\ z_i&=\bold a^T \Phi(X)^T\phi(\bold{x}_i) \end{align*}$$ ...