Explore machine learning's essentials for software engineers, delve into supervised learning, neural networks, and deep learning, and gain skills to tackle real-world data challenges effectively.

pml_final.tar.gz

just_run

chapter06

SPA_07

SPA_gdb_frontEnd

J12-1

J12-2

SPA-server_image

Handson-10

Machine learning is the future for the next generation of software professionals.

This course serves as a guide to machine learning for software engineers. You’ll be introduced to three of the most relevant components of the AI/ML discipline; supervised learning, neural networks, and deep learning. You’ll grasp the differences between traditional programming and machine learning by hands-on development in supervised learning before building out complex distributed applications with neural networks. You’ll go even further by layering networks to create deep learning systems. You’ll work with complex real-world datasets to explore machine behavior from scratch at each phase.

By the end of this course, you’ll have a working knowledge of modern machine learning techniques. Using software engineering, you’ll be prepared to build complex neural networks and wrangle real-world data challenges.

Fundamentals of Machine Learning for Software Engineers

# Test MNIST
It’s time to reach for our binary classification code (from [Classification in Action](https://www.educative.io/courses/fundamentals-of-machine-learning-for-software-engineers/classification-in-action)) and run it on MNIST. We have to adapt it a bit, but the changes are minimal. We can use the exact same functions that we used in the previous chapter, as long as we update the main code:



# An MNIST loader.

import numpy as np
import gzip
import struct


def load_images(filename):
    # Open and unzip the file of images:
    with gzip.open(filename, 'rb') as f:
        # Read the header information into a bunch of variables
        _ignored, n_images, columns, rows = struct.unpack('>IIII', f.read(16))
        # Read all the pixels into a NumPy array of bytes:
        all_pixels = np.frombuffer(f.read(), dtype=np.uint8)
        # Reshape the pixels into a matrix where each line is an image:
        return all_pixels.reshape(n_images, columns * rows)


def prepend_bias(X):
    # Insert a column of 1s in the position 0 of X.
    # (“axis=1” stands for: “insert a column, not a row”)
    return np.insert(X, 0, 1, axis=1)


# 60000 images, each 785 elements (1 bias + 28 * 28 pixels)
X_train = prepend_bias(load_images("../programming-machine-learning/data/mnist/train-images-idx3-ubyte.gz"))

# 10000 images, each 785 elements, with the same structure as X_train
X_test = prepend_bias(load_images("../programming-machine-learning/data/mnist/t10k-images-idx3-ubyte.gz"))


def load_labels(filename):
    # Open and unzip the file of images:
    with gzip.open(filename, 'rb') as f:
        # Skip the header bytes:
        f.read(8)
        # Read all the labels into a list:
        all_labels = f.read()
        # Reshape the list of labels into a one-column matrix:
        return np.frombuffer(all_labels, dtype=np.uint8).reshape(-1, 1)


def encode_fives(Y):
    # Convert all 5s to 1, and everything else to 0
    return (Y == 5).astype(int)


# 60K labels, each with value 1 if the digit is a five, and 0 otherwise
Y_train = encode_fives(load_labels("../programming-machine-learning/data/mnist/train-labels-idx1-ubyte.gz"))

# 10000 labels, with the same encoding as Y_train
Y_test = encode_fives(load_labels("../programming-machine-learning/data/mnist/t10k-labels-idx1-ubyte.gz"))


# A binary classifier that recognizes one of the digits in MNIST.

import numpy as np

# Applying Logistic Regression
def sigmoid(z):
    return 1 / (1 + np.exp(-z))

# Basically doing prediction but named forward as its 
# performing Forward-Propagation
def forward(X, w):
    weighted_sum = np.matmul(X, w)
    return sigmoid(weighted_sum)

# Calling the predict() function
def classify(X, w):
    return np.round(forward(X, w))


# Computing Loss over using logistic regression
def loss(X, Y, w):
    y_hat = forward(X, w)
    first_term = Y * np.log(y_hat)
    second_term = (1 - Y) * np.log(1 - y_hat)
    return -np.average(first_term + second_term)


# calculating gradient
def gradient(X, Y, w):
    return np.matmul(X.T, (forward(X, w) - Y)) / X.shape[0]

# calling the training function for desired no. of iterations
def train(X, Y, iterations, lr):
    w = np.zeros((X.shape[1], 1))
    for i in range(iterations):
        print('Iteration %4d => Loss: %.20f' % (i, loss(X, Y, w)))
        w -= gradient(X, Y, w) * lr
    return w

# Doing inference to test our model
def test(X, Y, w):
    total_examples = X.shape[0]
    correct_results = np.sum(classify(X, w) == Y)
    success_percent = correct_results * 100 / total_examples
    print("\nSuccess: %d/%d (%.2f%%)" %
          (correct_results, total_examples, success_percent))

# Test it
import mnist as data
w = train(data.X_train, data.Y_train, iterations=100, lr=1e-5)
test(data.X_test, data.Y_test, w)


import React from 'react';
import { shallow, configure } from 'enzyme';
import Adapter from 'enzyme-adapter-react-16';

configure({ adapter: new Adapter() });

import HelloWorld from './app';

var TestResult = function() {
 this.succeeded = false;
 this.reason = "";
 this.input = "";
 this.expected_output = "";
 this.actual_output = "";
}

export const executeTests = function() {

 var results = [];

 result = new TestResult();
 result.input = 'HelloWorld Component';
 result.expected_output = "span containing text 'Hello World'"

 let wrapper = shallow(<HelloWorld />);

 // Call your Challenge function here.

 let type = wrapper.type();
 let testSuccessful = true;
 let failureReason;

 if (type !== 'span') {
 testSuccessful = false;
 failureReason = "You need to render exactly one span HTML element";
 } else if (wrapper.props().children != "Hello World") {
 testSuccessful = false;
 failureReason = "You have rendered wrong message in your span element";
 }

 result.actual_output = wrapper.html();

 if (testSuccessful) {
 result.succeeded = true;
 result.reason = "Succeeded"
 } else {
 result.succeeded = false;
 result.reason = failureReason;
 }

 results.push(result);

 return results;
}

SPA_06

React

Dart

GoJS React

Typescript React

Vue.js

We use one line for training, one for testing. We don’t need to load and prepare the data, because our MNIST library already takes care of that.

And here’s the output of our first image classifier:

>Iteration 0 => Loss: 0.69314718055994528623 
Iteration 1 => Loss: 0.80042530259490185518 
Iteration 2 => Loss: 0.60370180008019158624 
... 
Iteration 99 => Loss: 0.11895658384798543650
>
>Success: 9637/10000 (96.37%) 

The result shows that over 96% of the program’s forecasts are proved accurate. _Our program recognizes images!_


While 96% might seem great, it does not necessarily mean that our program is very accurate. Think about it: only 10% of the digits in the MNIST test set are 5s. This means that a naive program that always forecasts $0$ (this is not a $5$) would hit the mark 90% of the time.

To be fair, our program’s 96% accuracy is better than that 90% baseline. However, it’s hard to gauge how much better it actually is. To get an intuitive sense of this program’s accuracy, we have to extend our code to recognize any digit, from 0 to 9. Let’s save our victory dance for the next chapter!

# Hyperparameter blues
We’ll find values for _iterations_ and $lr$ that work well for this new task. As usual, that search involves some trial and error. We have tweaked those hyperparameters until we achieve the result.

In this specific case, $lr$ must be very small. Let’s settle $lr$ on a value of 0.00001. With a larger $lr$, the program overflows when calculating the sigmoid and the loss. That overflow is a side effect of using exponentials and logarithms, which can easily churn out enormous numbers. Also, when we try a slightly larger $lr$, the program fails to converge, which is a fancy way of saying that gradient descent keeps skidding around the minimum loss instead of approaching it.

# Summary
* In this chapter we got up close and personal with MNIST. We wrote a library to import MNIST data and reshape it to $X$ and $Y$ matrices fit for our binary classification code. In the end, we used our program to recognize one of the digits in MNIST, with very encouraging results.

* Along the way, we learned a few interesting facts about image recognition. We also learned about testing ML systems and how the results of a test can be tricky to interpret because of _overfitting_.

Test the binary classification code on MNIST.

How Machine Learning Works

Our First Learning Program

Walking the Gradient

Hyperspace

A Discern Machine

Get Real

The Final Challenge

The Perceptron

Designing the Network

Building the Network

Training the Network

How Classifiers Work

Batchin’ Up

The Zen of Testing

Let’s Do Development

A Deeper Kind of Network

Diabetes Prediction Using Keras

Defeating Overfitting

Taming Deep Networks

Beyond Vanilla Networks

Into the Deep

Recognize Handwritten Digits Using a Deep Neural Network

The Real Thing

Test MNIST