Supervised vs unsupervised learning

What is machine learning?#

Machine learning is the subset of artificial intelligence (AI) that studies the algorithms and statistical models used by computer systems to perform tasks without being programmed to do so.

The major advantage of machine learning comes from its ability to enable computers to optimize their performance without needing explicit instructions. Instead, computer programmers can rely on machine learning to learn from the current context and generalize out to unseen tasks^[1] that adjust their programs without direct intervention.

As mentioned before, the huge volume of datasets being generated today has led to a proportionate demand in many industries for machine learning to extract relevant data^[2] that is capable of driving intelligent business decisions. At a corporate scale, machine learning is well-suited for making massive improvements to the efficiency of supply chains, energy consumption, and other areas with financial impact.

Supervised vs unsupervised learning#

Supervised learning is similar to how a student would learn from their teacher. The teacher acts as a supervisor, or, an authoritative source of information that the student can rely on to guide their learning. You can also think of the student’s mind as a computational engine.

Say these students are going on a field trip to the local zoo to learn about animals. The teacher shows the students each animal, and then provides the student with the animal’s name, or, label.

If the student makes a mistake when trying to identify certain animals, the teacher corrects their mistake by providing the correct name. As the teacher continues to train the student, the student begins to develop a pattern, or, model, in their minds.

Computational engines learn to recognize patterns and build models based on the training data provided by a supervisor. When that computational engine is presented with an unknown or unlabeled element, they can predict a label for it based on what they learned from the training data.

Essentially, the supervisor shows the computational engine an animal $\bold{x_i}$ and then tells the computational engine what label to use $y_i$ for that animal. Showing the computational engine more examples $(\bold{x_i},y_i)$ trains the computational engine to develop a model.

The supervisor shows the computational engine an unknown animal $\bold{x_t}$, and asks for its label $y_t$.

The computational engine predicts a label based on what it learned from the training data.

To summarize, the main difference is that input data will be accompanied by labels in supervised learning, but won’t have any labels in unsupervised learning.

Supervised learning#

In this section we will go over a brief comparison between regression vs. classification, and then move on to how those concepts relate to four popular supervised machine learning algorithms:

• Linear regression
• Support Vector Machine (SVM)
• Logistic regression
• Random forest

Regression vs classification#

Supervised learning models are especially well-suited for handling regression problems and classification problems.

Classification#

One machine learning method is classifying, and refers to the task of taking an input value and using it to predict discrete output values typically consisting of classes or categories.

Regression#

Regression refers to the task of predicting continuous output values such as temperature, height, or stock market trends.

Training data#

Training datasets can come in a variety of formats ranging from text to images, video, and audio. These datasets contain labeled data that helps train your machine learning algorithm to identify specific features and patterns in the data. Eventually, the training will enable your machine learning model to identify the features and patterns in unlabeled data.

Supervised learning focuses on the following sets of labeled data.

• Classification data: Training data where the labels $y_i$ represent different classes instead of a numeric value of some importance.
• Regression data: Training data where the labels $y_i$ have values of numeric importance, typically a real number.

Regression and classification are both types of supervised learning algorithms where the training data contains labels $y_i$.

Note: Other types of datasets include test data, which is used to benchmark the efficiency of a machine learning algorithm when predicting answers, and validation data, which is used to evaluate your training approach based on the algorithm and model parameters you have set.

Linear regression#

Linear regression was first developed in the field of statistics and is used in machine learning to create predictive models that assume a linear relationship between input variables (x) and an output variable (y).

• Simple linear regression: One input for x
• Multiple linear regression: Multiple input variables

Support Vector Machine (SVM)#

SVM is a popular binary-classification algorithm that provides a linear model for both classification and regression problems. For a while, SVM was the default choice because it provided simple models that avoided over-fitting. However, one drawback of SVM is that it couldn’t be extended to multi-class problems as easily as other algorithms.

Note: Non-linear SVMs also exist! Some datasets that can’t be optimally separated by a linear function can still be separated by a quadratic one.

Support vectors are the data points that lie closest to the decision surface (or hyperplane)^[1]. These data points are some of the most difficult ones to classify, and are critical to finding the optimal hyperplane. Removing any of these data points would ultimately change the position of the hyperplane.

The goal of an SVM is to maximize the margin around the hyperplane that separates these data points.

Note: In 2-dimensional space, data points can be separated by a line. SVM is especially effective when applied to spaces with higher dimensions because it allows the use of a hyperplane.

Logistic regression#

Despite its name, the logistic regression model is actually a linear model for classification. It is referred to as a logistic regression because it performs regression on logits^[2], which allows for the classification of data based on model probability predictions.

Like SVM, logistic regression estimates the classification boundary by maximizing the margin of all data points from the boundary. Unlike SVM, logistic regression can be extended to multiple classes with relative ease.

Random forest#

A random forest is referred to as such because it is essentially a group of decision trees!

With a random forest algorithm, the training model learns to predict the values of a target variable by learning the rules for making a decision. These decisions can be represented as a tree, with each branch leading to a decision node. Each node contains an attribute and asks for a decision to be made based on the available features.

Random forests are arguably one of the most popular algorithms used in supervised machine learning for regression and classification problems. The simplicity of this algorithm makes it approachable and easy to interpret for a wide range of problems.

Neural networks#

With over 80 billion neurons, the human brain is easily one of the most complex systems on Earth, and even after decades of study, the depth and breadth of its cognitive processes are nowhere close to being fully understood.

Biological neural networks like the human brain inspired the emergence of artificial neural networks (ANN). Deep learning (DL) is a subset of machine learning based on ANN technology, and attempts to expand the functionality of computers by enabling them to learn in a way that is similar to humans.

Neural networks are one of the most fundamental and ambitious concepts related to machine learning. Although traditional computers are great at performing many rapid calculations, they tend to struggle with solving problems that biological brains can handle with ease, like image recognition. Artificial neural networks aim to mimic cognitive processes in ways that can be used to perform interesting and more complex tasks.

One good example of how an artificial neural network is used in machine learning can be found in DeepMind’s AlphaGo, which used reinforcement learning to learn from millions of games of Go played against itself.

Applications of supervised learning#

Machine learning has been successfully applied in a wide variety of fields and industries ranging from pattern recognition, computer vision, spacecraft engineering, finance, entertainment, computational biology, and medicine^[3]. Below are some interesting examples of use cases for supervised learning algorithms.

Computer vision is a field of artificial intelligence concerning the ability of machines to be able to gain high-level understanding from images and videos. At the heart of computer vision is the task of image recognition. Image classification is used to train neural networks by taking raw images and processing them into usable data for machine learning.

Image recognition models are essential for many machine-based visual tasks like facial recognition, guiding autonomous robots, or helping self-driving cars avoid accidents.

Although image classification is essential for categorizing images with labels, object detection is equally important for telling us where objects exist within an image. This is done by using bounding boxes, which use (x, y) coordinates to tell us the location of each object in an image.

Anomaly detection in machine learning refers to the task of identifying outliers, abnormal data points, and other unexpected observations in the dataset. Supervised anomaly detection requires training a classifier using two labeled datasets with one labeled normal and the other labeled as abnormal.

Unsupervised learning#

Unsupervised learning models use datasets without labeled outcomes to predict outcomes of unseen data.

There are two main types of unsupervised learning algorithms:

Clustering algorithms: Data is processed into clusters of data points that bear similar features to other data points in the same cluster

Association algorithms: Interesting relationships between variables in large databases are found and used to identify underlying association rules for how and why certain data points are connected.

K-means clustering#

K-means clustering is an iterative process that first looks for a fixed number of clusters (K) in the dataset. Initially, these clusters are picked randomly but will be recomputed later until the inertia or within-cluster-sum-of-squares is completely minimized.

The inertia of a K-means cluster is reduced by calculating the center of the ‘K^th’ cluster is represented by ‘μ_k’, and is also referred to as a cluster centroid, average point, or sometimes the cluster-center. Cluster centroids are simply the mean of all points within that cluster.

Each instance of a data point is added to the nearest centroid by calculating measures of similarity or distance. Then the centroids are recomputed with the new average point of the cluster. Data points are again added to the closest cluster centroid, and the average is recomputed again until the average no longer changes.

Principal Component Analysis (PCA)#

Principal component analysis is a very popular method for performing exploratory data analysis, information compression, data compression, image processing, and more. However, it’s primarily used for dimensionality reduction. Dimensionality refers to the number of variables and attributes your data possesses.

Having a high number of input variables can severely limit the function and performance of the algorithm used. This problem is known as the curse of dimensionality^[4].

Another good reason for reducing input variables and dimensionality is to obtain a statistically sound and reliable result. When dimensionality increases, the amount of data needed to support your result grows exponentially.

Dimension reduction methods like PCA work for data points observed in high-dimensional spaces because it reduces the number of variables in a dataset while preserving the information needed to analyze and explore your data.

Given a dataset, PCA works by normalizing the size of the data. Each element of a dimension is subtracted from the mean of its corresponding dimension.