Data Science in 5 Minutes: What is One Hot Encoding?

If you’re in the field of data science, you’ve probably heard the term “one hot encoding”. Even the Sklearn documentation tells you to “encode categorical integer features using a one-hot scheme”. But, what is one hot encoding, and why do we use it?

Most machine learning tutorials and tools require you to prepare data before it can be fit to a particular ML model. One hot encoding is a process of converting categorical data variables so they can be provided to machine learning algorithms to improve predictions. One hot encoding is a crucial part of feature engineering for machine learning.

In this guide, we will introduce you to one hot encoding and show you when to use it in your ML models. We’ll provide some real-world examples with Sklearn and Pandas.

This tutorial at a glance:

• What is one hot encoding?
• How to convert categorical data to numerical data
• One hot encoding with Pandas
• One hot encoding with Sklearn
• Next steps for your learning

Let’s apply this to an example. Say we have the values red and blue. With one-hot, we would assign red with a numeric value of 0 and blue with a numeric value of 1.

It’s crucial to be consistent when we use these values. This makes it possible to invert our encoding at a later point to get our original categorical back.

Once we assign numeric values, we create a binary vector that represents our numerical values. In this case, our vector will have 2 as its length since we have 2 values. Thus, the red value can be represented with the binary vector [1,0], and the blue value will be represented as [0,1].

Why use one hot encoding?#

One hot encoding is useful for data that has no relationship to each other. Machine learning algorithms treat the order of numbers as an attribute of significance. In other words, they will read a higher number as better or more important than a lower number.

While this is helpful for some ordinal situations, some input data does not have any ranking for category values, and this can lead to issues with predictions and poor performance. That’s when one hot encoding saves the day.

One hot encoding makes our training data more useful and expressive, and it can be rescaled easily. By using numeric values, we more easily determine a probability for our values. In particular, one hot encoding is used for our output values, since it provides more nuanced predictions than single labels.

• Line 7 shows that we’re using get_dummies to do one-hot encoding for a pandas DataFrame object. The parameter prefix indicates the prefix of the new column name.
• Line 9 shows us our output.

Let’s apply this to a practical example. Say we have the following dataset.

import pandas as pd
 
ids = [11, 22, 33, 44, 55, 66, 77]
countries = ['Seattle', 'London', 'Lahore', 'Berlin', 'Abuja']
 
df = pd.DataFrame(list(zip(ids, countries)),
                  columns=['Ids', 'Cities'])

Here we have a Pandas dataframe called df with two lists: ids and Cities. Let’s call the head() to get this result:

We see here that the Cities column contains our categorical values: the names of our cities. We must convert them in our new column Cities using the get_dummies() function we discussed above.

y = pd.get_dummies(df.Countries, prefix='City')
print(y.head())

Here, we are passing the value City for the prefix attribute of the method get_dummies(). If we run the code now, we will print our encoded values:

• We use LabelEncoder to convert the string to int on line 7 and line 8.
• Line 9 creates our OneHotEncoder object.
• Line 10 fits the original feature using fit().
• Line 11 converts the original feature to the new feature using one-hot encoding.
• You can see the new data from the output of line 15.

Note: In the newer version of sklearn, you don’t need to convert the string to int, as OneHotEncoder does this automatically.

Let’s see the OneHotEncoder class in action with another example. First, here’s how to import the class.

from sklearn.preprocessing import OneHotEncoder 

Like before, we first populate our list of unique values for the encoder.

When we print this, we get the following for our now encoded values:

[[1. 0. 0. 0. 0. 0. 0. 1.]
 [0. 1. 0. 0. 0. 1. 0. 0.]
 [0. 0. 1. 0. 0. 0. 0. 1.]
 [0. 0. 0. 1. 0. 0. 1. 0.]
 [0. 0. 0. 0. 1. 1. 0. 0.]]