Explore machine learning's essentials for software engineers, delve into supervised learning, neural networks, and deep learning, and gain skills to tackle real-world data challenges effectively.

pml_final.tar.gz

just_run

chapter06

SPA_07

SPA_gdb_frontEnd

J12-1

J12-2

SPA-server_image

Handson-10

Machine learning is the future for the next generation of software professionals.

This course serves as a guide to machine learning for software engineers. You’ll be introduced to three of the most relevant components of the AI/ML discipline; supervised learning, neural networks, and deep learning. You’ll grasp the differences between traditional programming and machine learning by hands-on development in supervised learning before building out complex distributed applications with neural networks. You’ll go even further by layering networks to create deep learning systems. You’ll work with complex real-world datasets to explore machine behavior from scratch at each phase.

By the end of this course, you’ll have a working knowledge of modern machine learning techniques. Using software engineering, you’ll be prepared to build complex neural networks and wrangle real-world data challenges.

Fundamentals of Machine Learning for Software Engineers

## L1 and L2 regularization
**L1** and **L2** are two of the most common methods to regularize a decision boundary. It is _the technical name for the operation that we informally called **smoothing out**_. L1 and L2 work similarly, and they have mostly similar effects. Once we get into advanced ML territory, we may want to look deeper into their relative merits — but for our purposes in this course, we follow a simple rule _either pick randomly between L1 and L2 or try both and see which one works better_.

Let’s see how L1 and L2 work.


### How L1 and L2 work
L1 and L2 rely on the same idea. **_They add a regularization term to the neural network’s loss_**. For example, here’s the loss augmented by L1 regularization:

$$
\large{
L_{\text{regularized}}=L_{\text{non-regularized}}+\lambda \sum{|w|}}
$$

In the case of our neural network, the non-regularized loss is the **cross-entropy loss**. To that original loss, L1 adds the sum of the absolute values of all the weights in the network, multiplied by a constant called **lambda** (or $\lambda$ in symbols).

Lambda is a new hyperparameter that we can use to tune the amount of regularization in the network. The higher the value of lambda, the higher the impact of the regularization term. If lambda is 0, the entire regularization term becomes 0, and we fall back to a non-regularized neural network.

To understand what the regularization term does to the network, remember that the entire point of training is to minimize the loss. Now that we have added that term, the absolute value of the weights has become part of the loss. That means that the gradient descent algorithm will automatically try to keep the weights small so that the loss can also stay small.


# L1 and L2 regularization
**L1** and **L2** are two of the most common methods to regularize a decision boundary. It is _the technical name for the operation that we informally called **smoothing out**_. L1 and L2 work similarly, and they have mostly similar effects. Once we get into advanced ML territory, we may want to look deeper into their relative merits — but for our purposes in this course, we follow a simple rule _either pick randomly between L1 and L2 or try both and see which one works better_.

Let’s see how L1 and L2 work.


## How L1 and L2 work
L1 and L2 rely on the same idea. **_They add a regularization term to the neural network’s loss_**. For example, here’s the loss augmented by L1 regularization:

$$
\large{
L_{\text{regularized}}=L_{\text{non-regularized}}+\lambda \sum{|w|}}
$$

In the case of our neural network, the non-regularized loss is the **cross-entropy loss**. To that original loss, L1 adds the sum of the absolute values of all the weights in the network, multiplied by a constant called **lambda** (or $\lambda$ in symbols).

Lambda is a new hyperparameter that we can use to tune the amount of regularization in the network. The higher the value of lambda, the higher the impact of the regularization term. If lambda is 0, the entire regularization term becomes 0, and we fall back to a non-regularized neural network.

To understand what the regularization term does to the network, remember that the entire point of training is to minimize the loss. Now that we have added that term, the absolute value of the weights has become part of the loss. That means that the gradient descent algorithm will automatically try to keep the weights small so that the loss can also stay small.


Learn about regularization techniques, L1 and L2.

How Machine Learning Works

Our First Learning Program

Walking the Gradient

Hyperspace

A Discern Machine

Get Real

The Final Challenge

The Perceptron

Designing the Network

Building the Network

Training the Network

How Classifiers Work

Batchin’ Up

The Zen of Testing

Let’s Do Development

A Deeper Kind of Network

Diabetes Prediction Using Keras

Defeating Overfitting

Taming Deep Networks

Beyond Vanilla Networks

Into the Deep

Recognize Handwritten Digits Using a Deep Neural Network

Regularize the Model

L1 and L2 regularization

How L1 and L2 work