    Gain insights into building and optimizing neural networks in Python. Delve into fundamental concepts, mathematical explanations, and practical implementations to enhance your machine learning skills.

Make your own neutral network in python final.png

Machine learning is one of the fastest growing fields, and we cannot emphasize enough about its importance. This course aims to teach one of the fundamental concepts of machine learning, i.e., Neural Network. You will learn the basic concepts of building a model as well as the mathematical explanation behind Neural Network and based on that; you will build one from scratch (in Python). You will also learn how to train and optimize your network to achieve a better result.

Make Your Own Neural Network in Python

# Error functions
The output of a neural network is a complex difficult function with many parameters and the link weights that influence its output. Can we use gradient descent to work out the right weights? Yes, as long as we pick the right error function.

The output function of a neural network isn't an error function itself. But we know we can turn it into one easily, because the error is the difference between the target training values and the actual output values. 

There's something to watch out for here. Look at the following table of training and actual values for three output nodes, together with candidates for an error function. 

Comparison of Different Error Functions

align-center

## Simple error
The first candidate for an error function is simply $(\text{target – actual})$. That seems reasonable enough, right? If we look at the sum over the nodes to get an overall figure for how well the network is trained, we'll see the sum is zero. 

What happened? Clearly the network isn’t perfectly trained because the first two node outputs are different from the target values. The sum of zero suggests there is no error. This happens because the positive and negative errors cancel each other out. Even if they didn't cancel out completely, we could see this is a bad measure of error.

## Absolute error
Let's correct this by taking the absolute value of the difference. That means ignoring the sign and writing it as $|\text{target – actual}|$. That could work because nothing can ever cancel out. The reason this isn’t popular is because the slope isn’t continuous near the minimum. That makes gradient descent not work so well, because we can bounce around the V-shaped valley that this error function has. The slope doesn't get smaller closer to the minimum, so our steps don't get smaller, which means we risk overshooting. 

## Squared error
The third option is to take the square of the difference $(\text{target – actual})^2$. There are several reasons why we prefer this third one over the second one, including:
- The algebra needed to work out the slope for gradient descent is easy  with this squared error.
- The error function is smooth and continuous, making gradient descent work well—there are no gaps or abrupt jumps.
- The gradient gets smaller closer to the minimum, meaning the risk of overshooting the objective gets smaller if we use it to moderate the step sizes.

Is there a fourth option? Yes, we can construct all kind of complicated and interesting cost functions. Some don’t work well at all, some work well for particular kinds of problems, and some do work but aren’t worth the extra complexity.

Learn how to transform the output into an error function and how to design that function from our neural network.


Prologue

A Little Background

Let's Get Started!

Backward Propagation of Error

Adjusting the Link Weights

A Gentle Start with Python

Neural Network with Python

Testing Neural Network against MNIST Dataset

Some Suggested Improvements

Even More Fun!

Epilogue

Appendix: A Small Guide to Calculus

Transform the Output into an Error Function

Error functions