Logistic Regression

Classification

Classification problems involve assigning labels or classes to instances. For example, an email can be spam or ham (not Spam). Logistic Regression is the basic algorithm used for the Binary Classification problems. It can be extended for Multi-class and Multi-label Classification problems. In this lesson, we will cover the mathematical working of Logistic Regression and how it can be used in Scikit Learn for real time problems.

Logistic Regression

Logistic Regression allows us to estimate the probability, $p$, of class membership of a given instance. The probability will range between 0 and 1. Logistic regression is also known in the literature as Logit regression, Maximum-Entropy Classification (MaxEnt), or the Log-Linear classifier.
We saw that Multivariate Linear Regression comes up with the following equation in higher dimensions.

$\hat{y} = w_0 * x_0 + w_1 * x_1 + w_2 * x_2 + w_3 * x_3 + w_4 * x_4$ … $w_n * x_n$

Here $x_0$ = 1

$\hat{y} = w^T * x$

In Logistic Regression, we pass the calculated predicted value $\hat{y}$ through a Logistic or Sigmoid function $g(z)$.

$\hat{y}_{Logistic}$ = $g(\hat{y})$

Where,

$g(z)$ = $\frac{1}{1+e^{-z}}$

$g(\hat{y})$ = $\frac{1}{1+e^{-\hat{y}}}$ = $\hat{y}_{Logistic}$