Logistic Regression

Until now, we have been predicting numerical quantities. But what if we want a model to predict a categorical variable? Categorical data is divided into distinct classes. The task of predicting a categorical variable is known as classification. We can perform classification using logistic regression.

Logistic function

Logistic Regression is a classification method that is built on the same concept as linear regression. In linear regression, we take a linear combination of different variables plus an intercept term to predict the output. But in classification problems, the predicted variable is categorical. The simplest case of classification is when the predicted variable is binary, i.e., it has only two classes, e.g., yes/no, male/female, etc. Logistic regression also takes the linear combination of different variables plus the intercept term, but afterward, it takes the result and passes it through a logistic function. The logistic function also known as sigmoid is defined as:

$$sigmoid(t) = \frac{1}{1+ e^{-t} }$$

where t is the output of the linear regression equation, i.e., linear combination of variables plus the intercept term. Let’s look at the plot of the logistic function below.