Classification with PyCaret

Classification is one of the fundamental supervised learning tasks. Its goal is to predict a categorical variable known as the class label. This task is known as binary classification when there are only two classes ($0$ and $1$), or multiclass classification in case there are more. One of the most widely used binary classification models is logistic regression. It is defined in the following equation:

$$\log (\frac{p_{n}}{1-p_{n}})=\beta_{0}+\beta_{1} x_{n 1}+\cdots+\beta_{p} x_{n p}=\beta^{T} X_{n}$$

• $\log (\frac{p_{n}} {1-p_{n}})$ is the natural logarithm of the odds, known as the logit function.
• $x_1$ to $x_p$ are the feature variables.
• $\beta_{0}$ is the intercept term.
• $\beta_{1}$ to $\beta_{p}$ are the coefficients of the feature variables.
• $\beta^{T} X_{n}$ is the vectorized form of the equation. Our goal is to calculate $p_{n}$ which is the probability that an instance of the given dataset belongs to class $1$. The logistic function $\sigma(z)$ is the inverse of the logit (or log-odds) function, so we can apply it and get the desired result.

$$p_{n}=\sigma (\beta^{T} X_{n})=\frac{1}{1+\exp (-\beta^{T} X_{n})}$$