Delve into PyTorch basics: autograd, model classes, datasets, and data loaders. Gain insights into model development while avoiding common pitfalls. Start creating and training your own PyTorch models.

Deep learning with Pytorch, A beginner's Guide By Daniel Godoy.png

jupyter_final.tar.gz

codesnip1

tensorboard

jupyterp-autograd

jupyterp-computation

jupyterp-rethinking

jupyterp-minibatch

jupyterp-dataloader

jupyterp-evaluation

jupyterp-saveload

jupyterp-predictions

jupyterp-training-predictions

jupyterp-checkpointing

jupyterp-classy

jupyterp-classification

jupyterp-retest

jupyterp-retest-copy

jupyterp-retest-copy-copy

jupyterp-retest-copy-copy-copy

jupyterp-retest-copy-copy-copy-copy

jupyterp-retest-question

jupyterp-retest-question-copy

jupyterp-retest-question-copy-copy

jupyterp-retest-question-copy-copy-copy

jupyterp-retest-question-copy-copy-copy-copy

jupyterp-retest-challenge

jupyterp-retest-challenge-copy

jupyterp-retest-challenge-copy-copy

jupyterp-retest-challenge-copy-copy-copy

jupyterp-retest-challenge-copy-copy-copy-copy

jupyterp-putting1

jupyterp-putting2

jupyterp-putting3

jupyterp-putting4

jupyterp-tensorboard

jupyterp-tensorboard-graphs

jupyterp-tensorboard-scalers

jupyterp-tensorboard-training

GUI-Tensorboard

GUI-Tensorboard-graphs

GUI-Tensorboard-scalers

GUI-Tensorboard-training

GUI-Chapter04

GUI-No-Jupyter-Tensorboard

This course is designed to provide you with an easy-to-follow, structured, incremental, and from-first-principles approach to learning PyTorch. In this course, you’ll be introduced to the fundamentals of PyTorch: autograd, model classes, datasets, data loaders, and more. You will develop, step-by-step, not only the models themselves but also your understanding of them. You'll be shown both the reasoning behind the code and how to avoid some common pitfalls and errors along the way. By the time you finish this course, you’ll have a thorough understanding of the concepts and tools necessary to start developing and training your own models using PyTorch.

Deep Learning with PyTorch Step-by-Step: Part I - Fundamentals

# Difference between error and loss

There is a subtle but fundamental difference between error and loss. 

The error is the difference between the **actual value (label)** and the **predicted value** computed for a single data point. So, for a given _i_<sup><i>th</i></sup> point (from our dataset of `N` points), its error is:

<div style="font-size: 1.8em;">
<center>

$error_i = \hat{y_i} - y_i$

</center>
</div>

The error of the first point in our dataset (_i_ = 0) can be represented like this:

On the other hand, the loss is some sort of aggregation of errors for a set of data points. 

It seems rather obvious to compute the loss for all (`N`) data points, right? Well, yes and no. Although it will surely yield a more stable path from the initial random parameters to the parameters that minimize the loss, it will also surely be slow.

This means one needs to sacrifice (a bit) the stability for the sake of speed. This can be easily accomplished by randomly choosing (without replacement) a subset of `n` out of `N` data points each time we compute the loss.

# Gradient descent and its relationship with loss

The following presents a brief overview of the three main gradient descent types that are mostly used in Deep Learning applications and how they relate to loss:

*  If we use all points in the training set (`n = N`) to compute the loss, we are performing a **batch** gradient descent.

*  If we were to use a single point (`n = 1`) each time, it would be a **stochastic** gradient descent.
*  Anything else (`n`) in-between 1 and `N` characterizes a **mini-batch** gradient descent.

# Computing the loss

For a regression problem, the loss is given by the **Mean Squared Error (MSE)**. MSE is, the average of all squared errors, meaning the average of all squared differences between labels (`y`) and predictions (`b + wx`). It has the following formula:

<div style="font-size: 1.33em;">
<center>

$MSE = \dfrac{1}{n} \sum_{i=1}^n{error_i ^ 2}$

$\quad\quad\quad\space\space= \dfrac{1}{n} \sum_{i=1}^n{(\hat{y_i} - y_i) ^ 2}$

$\quad\quad\quad\quad\quad\quad= \dfrac{1}{n} \sum_{i=1}^n{(b + w x_i - y_i) ^ 2}$

</center>
</div>

Let us now compute the loss using Python. 

# Difference between error and loss

There is a subtle but fundamental difference between error and loss. 

The error is the difference between the **actual value (label)** and the **predicted value** computed for a single data point. So, for a given _i_<sup><i>th</i></sup> point (from our dataset of `N` points), its error is:

<div>
<center>

$error_i = \hat{y_i} - y_i$

</center>
</div>

The error of the first point in our dataset (_i_ = 0) can be represented like this:

On the other hand, the loss is some sort of aggregation of errors for a set of data points. 

It seems rather obvious to compute the loss for all (`N`) data points, right? Well, yes and no. Although it will surely yield a more stable path from the initial random parameters to the parameters that minimize the loss, it will also surely be slow.

This means one needs to sacrifice (a bit) the stability for the sake of speed. This can be easily accomplished by randomly choosing (without replacement) a subset of `n` out of `N` data points each time we compute the loss.

# Gradient descent and its relationship with loss

The following presents a brief overview of the three main gradient descent types that are mostly used in Deep Learning applications and how they relate to loss:

*  If we use all points in the training set (`n = N`) to compute the loss, we are performing a **batch** gradient descent.

*  If we were to use a single point (`n = 1`) each time, it would be a **stochastic** gradient descent.
*  Anything else (`n`) in-between 1 and `N` characterizes a **mini-batch** gradient descent.

# Computing the loss

For a regression problem, the loss is given by the **Mean Squared Error (MSE)**. MSE is, the average of all squared errors, meaning the average of all squared differences between labels (`y`) and predictions (`b + wx`). It has the following formula:

<div>
<center>

$MSE = \dfrac{1}{n} \sum_{i=1}^n{error_i ^ 2}$

$\quad\quad\quad\space\space= \dfrac{1}{n} \sum_{i=1}^n{(\hat{y_i} - y_i) ^ 2}$

$\quad\quad\quad\quad\quad\quad= \dfrac{1}{n} \sum_{i=1}^n{(b + w x_i - y_i) ^ 2}$

</center>
</div>

Let us now compute the loss using Python. 

Learn about the difference between error and loss, the relationship between gradient descent and loss, and how the loss can be computed. 

Step 2a - Compute the Loss 

Learn about the difference between error and loss, the relationship between gradient descent and loss, and how the loss can be computed.

Step 2a - Compute the Loss

Introduction

Visualizing Gradient Descent

A Simple Regression Problem

Rethinking the Training Loop

Going Classy

A Simple Classification Problem

Conclusion

Appendix

Step 2a - Compute the Loss

Difference between error and loss