Explore machine learning's essentials for software engineers, delve into supervised learning, neural networks, and deep learning, and gain skills to tackle real-world data challenges effectively.

pml_final.tar.gz

just_run

chapter06

SPA_07

SPA_gdb_frontEnd

J12-1

J12-2

SPA-server_image

Handson-10

Machine learning is the future for the next generation of software professionals.

This course serves as a guide to machine learning for software engineers. You’ll be introduced to three of the most relevant components of the AI/ML discipline; supervised learning, neural networks, and deep learning. You’ll grasp the differences between traditional programming and machine learning by hands-on development in supervised learning before building out complex distributed applications with neural networks. You’ll go even further by layering networks to create deep learning systems. You’ll work with complex real-world datasets to explore machine behavior from scratch at each phase.

By the end of this course, you’ll have a working knowledge of modern machine learning techniques. Using software engineering, you’ll be prepared to build complex neural networks and wrangle real-world data challenges.

Fundamentals of Machine Learning for Software Engineers

# Softmax
Check out the activation functions in our neural network. So far, we took it for granted that both of those functions are sigmoids. However, most neural networks replace the last sigmoid, the one right before the output layer, with another function called the **softmax**.

Let’s see what the softmax looks like and why it is useful. Like the sigmoid, the softmax takes an array of numbers, that in this case are called the **logits**. And it returns an array with the same size as the input. Here is the formula of the softmax, in case we want to understand the math behind it:​

$$\text{softmax}(l_i)=\frac{e^{l_i}}{\sum{e^l}}$$


We can read this formula as: _take the exponential of each logit and divide it by the summed exponentials of all the logits._

We don't need to grok the formula of the softmax, as long as we understand what happens when we use the softmax instead of the sigmoid. Think about the output of the sigmoid in our MNIST classifier, back in [Decoding the Classifier’s Answers](https://www.educative.io/courses/fundamentals-of-machine-learning-for-software-engineers/decode-the-classifiers-answers). Let's recall that for each image in the input, the sigmoid gives us ten numbers between $0$ and $1.$ Those numbers tell us how confident the perceptron is about each classification. For example, if the fourth number is $0.9,$ that means _this image is probably a $3.$_ If it’s close to $0.1,$ that means _this image is unlikely to be a $3.$_ Among those ten results, we pick the one with the highest confidence.

Like the sigmoid, the softmax returns an array where each element is between $0$ and $1.$ However, the softmax has an additional property: **_the sum of its outputs is always $1.$_** In mathspeak, we would say that the softmax normalizes that sum to a value of $1.$

That’s a nice property because if the numbers add up to $1,$ then we can interpret them as probabilities. To give a concrete example, imagine that we run a three-class classifier, and the weighted sum after the hidden layer returns these logits:

| **logit** $1$  |  **logit** $2$ |  **logit** $3$ |
| - | - | - |
| 1.6  | 3.1  |  0.5 |

From these numbers, it seems likely that the item that we classify belongs to the second class, but it’s hard to gauge how likely. Now see what happens if we pass the logits through a softmax:

| **softmax** $1$  |  **softmax** $2$ |  **softmax** $3$ |
| - | - | - |
| 0.17198205  | 0.77077009  |  0.05724785 |

What’s the chance that the item belongs to the second class? Take a glance at these numbers, and we’ll know that the answer is 77%. The first class is way less likely, hovering around 17%, while the third is around 6%. When we add them together, we get the expected 100%. By softmax, we can convert vague numbers to human-friendly probabilities. That’s a good reason to use softmax as the last activation function in our neural network.

# Here’s the plan

Now that we know about the softmax, let’s replace the second sigmoid with a softmax and complete the design of our neural network:


Learn briefly about activation functions, especially softmax.

How Machine Learning Works

Our First Learning Program

Walking the Gradient

Hyperspace

A Discern Machine

Get Real

The Final Challenge

The Perceptron

Designing the Network

Building the Network

Training the Network

How Classifiers Work

Batchin’ Up

The Zen of Testing

Let’s Do Development

A Deeper Kind of Network

Diabetes Prediction Using Keras

Defeating Overfitting

Taming Deep Networks

Beyond Vanilla Networks

Into the Deep

Recognize Handwritten Digits Using a Deep Neural Network

Introduction to Softmax

Softmax