Classification Through Regression

Multi-classification

We’ve already seen multiclass modeling using softmax. It should be noticed, however, that the model is still a regression model whether we use softmax or not. This is because it makes the predictions in the form of real numbers (probability distribution in the case of softmax). The prediction of the class label is one step ahead. We typically use thresholding on the output of the model for predicting the class label. Before this final thresholding, it’s always regression. The figure below illustrates this fact.

Notice that in the animation above, after softmax, we need the thresholding step for the class label. Before thresholding, the output of the model consists of real numbers just like an ordinary regression model.

Note: A classification model inherits from a regression model.

One-hot-encoding

One reason for using softmax is one-hot-encoding. It’s a way of encoding the class labels for training data. In particular, if we have $c$classes, then each class label from the set $\{1,2,...,c\}$ can be represented as a vector of $c$ components. The vector is entirely zero except for the index of the class label. At the class label index, the component of the vector is $1$.

Example

When the number of classes, $c$, is $5$, the one-hot-encoding for class $2$ is ${[\begin{array}{c}\end{array}}^{}$ ...