    Gain insights into building and optimizing neural networks in Python. Delve into fundamental concepts, mathematical explanations, and practical implementations to enhance your machine learning skills.

Make your own neutral network in python final.png

Machine learning is one of the fastest growing fields, and we cannot emphasize enough about its importance. This course aims to teach one of the fundamental concepts of machine learning, i.e., Neural Network. You will learn the basic concepts of building a model as well as the mathematical explanation behind Neural Network and based on that; you will build one from scratch (in Python). You will also learn how to train and optimize your network to achieve a better result.

Make Your Own Neural Network in Python

## Adjust the learning rate 

A 95\% performance score on the MNIST dataset with our first neural network, using only simple ideas and simple Python, is not bad at all. If we wanted to stop here, it would be entirely justified.

But let's see if we can make some easy improvements.

The first improvement we can try is to adjust the learning rate. Previously, we set it at $0.3$, without really experimenting with different values. 

Let's try doubling it to $0.6$, to see if a boost will be helpful or harmful to the overall network learning. If we run the code, we get a performance score of $0.9047$. That's worse than before. So, it looks like the larger learning rate leads to some bouncing around and overshooting during the gradient descent. 

Let's try again with a learning rate of $0.1$. This time, the performance is an improvement at $0.9523$. It's similar in performance to one listed on that website, which has 1000 hidden nodes. We're doing pretty well with fewer!

What happens if we keep going and set the learning rate to an even smaller $0.01$? The performance isn't so good at $0.9241$. So, it seems having too small of a learning rate is damaging. This makes sense because we're limiting the speed at which gradient descent happens, and we're making the steps too small.

## Performance graph against learning rate
The following image plots a graph of these results. It's not a very scientific approach, because we should really do these experiments many times to reduce the effect of randomness and bad journeys down the gradient descent, but it is still useful to see the general idea that there is a sweet spot for learning rate.

# Adjust the learning rate 

A 95\% performance score on the MNIST dataset with our first neural network, using only simple ideas and simple Python, is not bad at all. If we wanted to stop here, it would be entirely justified.

But let's see if we can make some easy improvements.

The first improvement we can try is to adjust the learning rate. Previously, we set it at $0.3$, without really experimenting with different values. 

Let's try doubling it to $0.6$, to see if a boost will be helpful or harmful to the overall network learning. If we run the code, we get a performance score of $0.9047$. That's worse than before. So, it looks like the larger learning rate leads to some bouncing around and overshooting during the gradient descent. 

Let's try again with a learning rate of $0.1$. This time, the performance is an improvement at $0.9523$. It's similar in performance to one listed on that website, which has 1000 hidden nodes. We're doing pretty well with fewer!

What happens if we keep going and set the learning rate to an even smaller $0.01$? The performance isn't so good at $0.9241$. So, it seems having too small of a learning rate is damaging. This makes sense because we're limiting the speed at which gradient descent happens, and we're making the steps too small.

# Performance graph against learning rate
The following image plots a graph of these results. It's not a very scientific approach, because we should really do these experiments many times to reduce the effect of randomness and bad journeys down the gradient descent, but it is still useful to see the general idea that there is a sweet spot for learning rate.

Explore how the learning rate affects the accuracy of results.

Prologue

A Little Background

Let's Get Started!

Backward Propagation of Error

Adjusting the Link Weights

A Gentle Start with Python

Neural Network with Python

Testing Neural Network against MNIST Dataset

Some Suggested Improvements

Even More Fun!

Epilogue

Appendix: A Small Guide to Calculus

Tweak the Learning Rate

Adjust the learning rate