Explore machine learning's essentials for software engineers, delve into supervised learning, neural networks, and deep learning, and gain skills to tackle real-world data challenges effectively.

pml_final.tar.gz

just_run

chapter06

SPA_07

SPA_gdb_frontEnd

J12-1

J12-2

SPA-server_image

Handson-10

Machine learning is the future for the next generation of software professionals.

This course serves as a guide to machine learning for software engineers. You’ll be introduced to three of the most relevant components of the AI/ML discipline; supervised learning, neural networks, and deep learning. You’ll grasp the differences between traditional programming and machine learning by hands-on development in supervised learning before building out complex distributed applications with neural networks. You’ll go even further by layering networks to create deep learning systems. You’ll work with complex real-world datasets to explore machine behavior from scratch at each phase.

By the end of this course, you’ll have a working knowledge of modern machine learning techniques. Using software engineering, you’ll be prepared to build complex neural networks and wrangle real-world data challenges.

Fundamentals of Machine Learning for Software Engineers

# Combat overfitting through regularization
Just like tuning hyperparameters, reducing overfitting is more art than science. Besides L1 and L2, we can use many other regularization methods. An overview of some of the techniques is given below:

**Small network size:** The most fundamental regularization technique is to **_make the overfitting network smaller_**. It is also the most efficient technique. After all, overfitting happens because the system is too smart for the data it’s learning. Smaller networks are not as smart as big networks. We should try to reduce the number of hidden nodes or remove a few layers. We’ll use this approach in the chapter’s closing exercise.

**Reduce input variables:** Instead of simplifying the model, we can also reduce overfitting by simplifying the data. We can remove a few input variables. Let’s say we predict a boiler’s consumption from a set of 20 input variables. An overfitting network strives to fit the details of that dataset, noise included. We can drop a few variables that are less likely to impact consumption (like the day of the week) in favor of the ones that seem more relevant (like the outside temperature). _The idea is that the fewer features we have, the less noise we inject into the system_.

**Curtail the training dataset:** Another way to reduce overfitting is to cut short the network’s training. This idea is not as weird as it sounds. If we look at the history of the network’s loss during training, we can see the system moving from underfitting to overfitting as it learns the noise in the training data. Once overfitting kicks in, the validation flattens and then diverges from the training loss. If we stop training at that point, we’ll get a network that has not yet learned enough to overfit the data. This technique is called **early stopping**.

**Increase learning rate:** Finally, and perhaps surprisingly, sometimes we can reduce overfitting by increasing a neural network’s learning rate. To understand why remember that the learning rate measures the size of each GD step. With the bigger learning rate, GD takes bolder, coarser steps. As a result, the trained model is likely to be less detailed, which might help reduce overfitting.

We went through a few regularization techniques, and we’ll learn about a couple more in the next chapter. Each of these approaches might or might not work for a specific network and dataset. Here, as in many other aspects of ML, our mileage may vary. Be ready to experiment with different approaches, either alone or in combination, and learn by experience which approaches work best in which circumstances.


Discover the benefits of regularization techniques.

How Machine Learning Works

Our First Learning Program

Walking the Gradient

Hyperspace

A Discern Machine

Get Real

The Final Challenge

The Perceptron

Designing the Network

Building the Network

Training the Network

How Classifiers Work

Batchin’ Up

The Zen of Testing

Let’s Do Development

A Deeper Kind of Network

Diabetes Prediction Using Keras

Defeating Overfitting

Taming Deep Networks

Beyond Vanilla Networks

Into the Deep

Recognize Handwritten Digits Using a Deep Neural Network

A Regularization Toolbox

Combat overfitting through regularization