Invasion of the Sigmoids

Overview of sigmoids

Even though linear regression is not a natural fit for binary classification, that does not mean that we have to scrap our linear regression code and start from scratch. Instead, we can adapt our existing algorithm to this new problem using a technique that statisticians call logistic regression.

Let’s start by looking back at $\hat{y}$, the weighted sum of the inputs that we introduced in the lesson: Adding More Dimensions.

$$\hat{y} = x_1 * w_1 + x_2 * w_2 + x_3 * w_3 + ...$$

In linear regression, $\hat{y}$ could take any value. Binary classification, however, imposes a tight constraint that $\hat{y}$ must not drop below $0$ nor raise above $1$. Here’s an idea: maybe we can find a function that wraps around the weighted sum and constrains it to the range ...