Learn the basics of data science and how to work with important libraries such as NumPy, pandas, Beautiful Soup, etc.

Become a Data Scientist

This module will get us familiar with the state of data science and its fundamental concepts. We'll be going through the applications of data science and a few libraries which are essential to solve any problem in this field, such as Beautiful Soup, Scrapy, NumPy, pandas, spaCy, etc.
We'll also learn about data manipulation with NumPy as well as probability and statistics.
By the end of this module, we'll have basic and essential knowledge of data science and its libraries.

Getting Started with Data Science


## Skewness

Skewness refers to the distortion or asymmetry in a bell curve or normal distribution. It tells us how much a distribution varies from the normal distribution. A normal distribution has a Skewness of zero. A distribution can be right (positively) or left (negatively) skewed.




### Consequence of skewness

* Skewness helps us locate the outliers (the data points exhibiting mysterious behaviour). For example, the transactions happening over a credit card abruptly jump to a higher amount than the normal transactions happening. This contributes to an outlier.

* The mean of a positively skewed distribution is greater than the median, while the mean of a negatively skewed distribution is less than the median.

* The mean of a positively skewed distribution is greater than the mode, while the mean of a negatively skewed distribution is less than the mode.


* Regression models in machine learning are affected by the presence of outliers,  which can be indicated by a skewed distribution. So, it becomes necessary at times to remove the skewness.
  

### How to check skewness

#### Naive approach

* The naive approach is to make the histogram or density curve of a column of a dataset. Check the curve to see if it is more Gaussian-like or skewed.


* There are some mathematical measures to check the skewness of a column.

#### Method 1

 Skewness = $\frac{\frac{1}{n}\sum_{i=1}^{n}(x_i-\bar{x})^3 }{[\frac{1}{n-1}\sum_{i=1}^{n}(x-\bar{x})^2]^\frac{3}{2}}$ 

* Here $n$ is the total number of values in a column of a dataset.

* $x_i$ is the single value in the dataset column.

* $\bar{x}$ is the mean of the dataset column. 

#### Inference

* If skewness is less than -1 or greater than 1, the distribution is highly skewed.

* If skewness is between -1 and -0.5 or between 0.5 and 1, the distribution is moderately skewed.

* If skewness is between -0.5 and 0.5, the distribution is approximately symmetric.




#### Method 2

Skewness = $\frac{\bar{x}-mode}{s}$

* $\bar{x}$ is the mean of the dataset column.

* Mode is the most occurring value in the dataset. 

* $s$ is the standard deviation of the dataset's column. It has been discussed in detail in the previous lesson.

* The above formula is called the Pearson first skewness coefficient or mode skewness.


Skewness = $\frac{3(\bar{x} - median)}{s}$

* The above formula is called the Pearson second skewness coefficient or median skewness.

* This is mostly used when the mode does not exist in the data.


#### Inference

* If Skewness > 0, the distribution is skewed to the right or positively skewed.

* If Skewness < 0, the distribution is skewed to the left or negatively skewed.

* If Skewness = 0, the distribution is symmetrical. It seldom happens in a real-world dataset.


### How to remove skewness

* Consider removing the extreme values causing skewness in the distribution. Extreme values can be introduced by rare events, measurement errors, and other sources. Domain knowledge would further assist in doing so.

* Taking the log transform of the respective column of a dataset can also help reduce the skewness.

* There is a famous Box-Cox Method, which takes in a parameter to transform the column of a dataset so it is more Gaussian-like. 



## Kurtosis

It is a statistical measure that shows us how the tails of the distribution differ from the tails of a normal distribution. 


### Types of kurtosis

![]()






# Skewness

Skewness refers to the distortion or asymmetry in a bell curve or normal distribution. It tells us how much a distribution varies from the normal distribution. A normal distribution has a Skewness of zero. A distribution can be right (positively) or left (negatively) skewed.




## Consequence of skewness

* Skewness helps us locate the outliers (the data points exhibiting mysterious behaviour). For example, the transactions happening over a credit card abruptly jump to a higher amount than the normal transactions happening. This contributes to an outlier.

* The mean of a positively skewed distribution is greater than the median, while the mean of a negatively skewed distribution is less than the median.

* The mean of a positively skewed distribution is greater than the mode, while the mean of a negatively skewed distribution is less than the mode.


* Regression models in machine learning are affected by the presence of outliers,  which can be indicated by a skewed distribution. So, it becomes necessary at times to remove the skewness.
  

## How to check skewness

### Naive approach

* The naive approach is to make the histogram or density curve of a column of a dataset. Check the curve to see if it is more Gaussian-like or skewed.


* There are some mathematical measures to check the skewness of a column.

### Method 1

 Skewness = $\frac{\frac{1}{n}\sum_{i=1}^{n}(x_i-\bar{x})^3 }{[\frac{1}{n-1}\sum_{i=1}^{n}(x-\bar{x})^2]^\frac{3}{2}}$ 

* Here $n$ is the total number of values in a column of a dataset.

* $x_i$ is the single value in the dataset column.

* $\bar{x}$ is the mean of the dataset column. 

### Inference

* If skewness is less than -1 or greater than 1, the distribution is highly skewed.

* If skewness is between -1 and -0.5 or between 0.5 and 1, the distribution is moderately skewed.

* If skewness is between -0.5 and 0.5, the distribution is approximately symmetric.




### Method 2

Skewness = $\frac{\bar{x}-mode}{s}$

* $\bar{x}$ is the mean of the dataset column.

* Mode is the most occurring value in the dataset. 

* $s$ is the standard deviation of the dataset's column. It has been discussed in detail in the previous lesson.

* The above formula is called the Pearson first skewness coefficient or mode skewness.


Skewness = $\frac{3(\bar{x} - median)}{s}$

* The above formula is called the Pearson second skewness coefficient or median skewness.

* This is mostly used when the mode does not exist in the data.


### Inference

* If Skewness > 0, the distribution is skewed to the right or positively skewed.

* If Skewness < 0, the distribution is skewed to the left or negatively skewed.

* If Skewness = 0, the distribution is symmetrical. It seldom happens in a real-world dataset.


## How to remove skewness

* Consider removing the extreme values causing skewness in the distribution. Extreme values can be introduced by rare events, measurement errors, and other sources. Domain knowledge would further assist in doing so.

* Taking the log transform of the respective column of a dataset can also help reduce the skewness.

* There is a famous Box-Cox Method, which takes in a parameter to transform the column of a dataset so it is more Gaussian-like. 



# Kurtosis

It is a statistical measure that shows us how the tails of the distribution differ from the tails of a normal distribution. 


## Types of kurtosis

![]()





Learn about the skewness and kurtosis, which are important properties of distributions.

Skewness and Kurtosis

Skewness

Consequence of skewness

How to check skewness

Naive approach

Method 1