    Gain insights into building and optimizing neural networks in Python. Delve into fundamental concepts, mathematical explanations, and practical implementations to enhance your machine learning skills.

Make your own neutral network in python final.png

Machine learning is one of the fastest growing fields, and we cannot emphasize enough about its importance. This course aims to teach one of the fundamental concepts of machine learning, i.e., Neural Network. You will learn the basic concepts of building a model as well as the mathematical explanation behind Neural Network and based on that; you will build one from scratch (in Python). You will also learn how to train and optimize your network to achieve a better result.

Make Your Own Neural Network in Python

# Refine the parameters 
How should we use the classification error $E$ to guide us to a more refined parameter $A$? That's the important question. Let's take a step back from this task and think again. We want to use the error in $y$, which we call $E$, to inform the required change in parameter $A$. To do this, we need to know how the two are related. If we know their relationship, we can understand how changing one affects the other. Let's start with the linear function for the classifier:

$$
y = Ax
$$

We know from our initial guesses of the value of $A$, that this gives the wrong answer for $y$, which should be the value given by the training data. Let’s call the correct desired value $t$ for the target value. To get that value $t$, we need to adjust $A$ by a small amount. Mathematicians use delta $Δ$, meaning "a small change in."

$$
t = (A + ΔA)x
$$

Let's graph this to make it easier to understand. You can see the new slope $(A + ΔA)$.

Remember, the error $E$ was the difference between the desired correct value and the one we calculated based on our current guess for $A$. That is, $E$ was $t - y$. Let's write that out to make it more clear:

$$t - y = (A + ΔA)x - Ax$$

Now, let's expand the terms and simplify:

$$E = t - y = Ax + (ΔA)x - Ax$$

$$E = (ΔA)x$$

The error $E$ is related to $ΔA$ in a very simple way. It’s so simple that we might think it is wrong, but it is indeed correct. This simple relationship makes our job much easier. It's easy to get lost or distracted by the algebra. Let's remind ourselves of what we want to get out of all this, in plain English.

We want to know how much to adjust $A$ to improve the slope of the line so it is a better classifier, being informed by the error $E$. To do this, we rearrange that last equation to put $ΔA$ on its own:

$$ 
ΔA = E / x 
$$ 


That's it! That's the magic expression we've been looking for. We can use the error $E$ to refine the slope $A$ of the classifying line by an amount $ΔA$. Let's update that initial slope. The error was $0.35$ and $x$ was $3.0$. That gives $ΔA = E / x$ as $0.35 / 3.0 = 0.1167$. That means we need to change the current $A = 0.25$ by $0.1167$. So the new, improved value for $A$ is $(A + ΔA)$, which is $0.25 + 0.1167 = 0.3667$. As it happens, the calculated value of $y$ with this new $A$ is $1.1$, since it's the desired target value.

Now that we're done with one training example, let's learn from the next one. Here, we have a known true pairing of $x = 1.0$ and $y = 3.0$. Let's see what happens when we put $x = 1.0$ into the linear function, which is now using the updated $A = 0.3667$. We get $y = 0.3667 \times 1.0 = 0.3667$. That's not very close to the training example with $y = 3.0$ at all.

Using the same reasoning as before, that we don't want the line to cross the training data but instead be just above or below it, we can set the desired target value at $2.9$. This way, the training example of a caterpillar is just above the line, not on it. The error $E$ is $(2.9 - 0.3667) = 2.5333$. That's a bigger error than before, but the only data we've had for the linear function to learn from so far is a single training example, which clearly biases the line towards that single example.

Let's update $A$ again, just like we did before. The value of $ΔA$ is $E / x$, which is $2.5333 / 1.0 = 2.5333$. That means the even newer $A$ is $0.3667 + 2.5333 = 2.9$. That means for $x = 1.0$, the function gives $2.9$ as the answer, which is the desired value. Let’s pause and visualize what we’ve done. The following plot shows the initial line, the updated line after learning from the first training example, and the final line after learning from the second training example.

Learn how to refine our slope parameter based on the error value.

Prologue

A Little Background

Let's Get Started!

Backward Propagation of Error

Adjusting the Link Weights

A Gentle Start with Python

Neural Network with Python

Testing Neural Network against MNIST Dataset

Some Suggested Improvements

Even More Fun!

Epilogue

Appendix: A Small Guide to Calculus

Refine the Parameters of the Training Classifier

Refine the parameters