**Loss functions** are used to estimate and understand how well a particular algorithm performs. They are classified into two categories:

1. **Regression Models** are used to predict continuous values.

2. **Classification Models** are used to predict the particular output from a set of finite and categorical values.


# Regression losses
Let's discuss regression losses, such as Mean Squared Error (MSE), Mean Absolute Error (MAE), Mean Bias Error (MBE) and Huber Loss.
  
## Mean Squared Error

**MSE** is also referred to as *quadratic* or *L2 loss*. This method calculates the mean(average) of the square of the difference between predictions and actual observations.   


MSE = \frac{\sum_{i=1}^n(y_i - \^y_i)^2}{n}

## Mean Absolute Error

**MAE** is also referred to as *L1 loss*. This method calculates the mean(average) of the sum of the absolute differences between predictions and actual observations.





MAE = \frac{\sum_{i=1}^n|y_i - \^y_i|}{n}

## Mean Bias Error
**MBE** is similar to MSE but less accurate. However, it is used to conclude if the model's bias was negative or positive. 


MBE = \frac{\sum_{i=1}^n(y_i - \^y_i)}{n}

## Huber Loss
**Huber Loss** is also referred to as *Smooth Mean Absolute Error*. This is an absolute error and often becomes a quadratic error when the error is too tiny. However, it is not that sensitive to outliers. 

Huber \space Loss = \biggl\{ \begin{array}{ll}
        \frac{1}{2}(x - y)^2 & \text{if } (|x - y| \leq \delta )\\ \\
        \delta*|x - y| - \frac{1}{2}*\delta^2 & \text{otherwise}
    \end{array}

# Classification losses
Let's discuss classification losses, such as Cross Entropy Loss and Hinge Loss.
## Cross Entropy Loss

**Cross Entropy Loss** is also referred to as *Negative Log Likelihood* or *Log Loss*. It measures a classification model's performance where the output is a probability value between 0 and 1. Cross-entropy loss increases as the predicted probability diverges from the actual label.  





Cross \space Entropy \space Loss  = (y_i log (\^y_i) + (1 - y_i) log (1 - \^y_i))

## Hinge Loss
**Hinge Loss** is also referred to as a *multi-class SVM loss*. Hinge loss is applied for maximum-margin classification, most prominently for support vector machines (SVMs). It is used for training classifiers and is a convex function used in convex optimizers. 



SVM \space Loss  = \sum_{j \neq y_i} max (0, s_j - s_{y_i} + 1)

What is a loss function?

Loss functions are used to estimate and understand how well a particular algorithm performs. They are classified into two categories:   Regression Models are used to predict continuous values.   Classification Models are used to predict the particular output from a set of finite and categorical values.   