Logistic Regression

Moving towards class probability

Now, you will learn a classification algorithm, logistic regression. In this lesson, we will work with probabilities.

Instead of predicting the outcome directly, we can predict the probability of that outcome.

In the linear model, we used score $t$ to get the class or output $y$. If the score is greater than zero, we say that the class is positive. If the score is less than zero, we say that the class is negative. The score can range between negative infinity to positive infinity.

If the score approaches positive infinity (or a very high positive value), we can say with confidence that the class is positive. If the score approaches negative infinity (or a very high negative value), we can say with confidence that the class is negative. If the value is near zero, the answer is uncertain.

So, we have to map the score to a probability range $[0,1]$.

We can use a link function to do this transformation. This function takes the score and maps it to probability space [0,1].

Logistic function

We can use different types of functions to map score to a probability scale. The Sigmoid function is an example of a logistic function. It is defined as:

$$Sigmoid \; = \frac{1}{1 + e^{-Score}}$$ ...