Delve into PyTorch basics: autograd, model classes, datasets, and data loaders. Gain insights into model development while avoiding common pitfalls. Start creating and training your own PyTorch models.

Deep learning with Pytorch, A beginner's Guide By Daniel Godoy.png

jupyter_final.tar.gz

codesnip1

tensorboard

jupyterp-autograd

jupyterp-computation

jupyterp-rethinking

jupyterp-minibatch

jupyterp-dataloader

jupyterp-evaluation

jupyterp-saveload

jupyterp-predictions

jupyterp-training-predictions

jupyterp-checkpointing

jupyterp-classy

jupyterp-classification

jupyterp-retest

jupyterp-retest-copy

jupyterp-retest-copy-copy

jupyterp-retest-copy-copy-copy

jupyterp-retest-copy-copy-copy-copy

jupyterp-retest-question

jupyterp-retest-question-copy

jupyterp-retest-question-copy-copy

jupyterp-retest-question-copy-copy-copy

jupyterp-retest-question-copy-copy-copy-copy

jupyterp-retest-challenge

jupyterp-retest-challenge-copy

jupyterp-retest-challenge-copy-copy

jupyterp-retest-challenge-copy-copy-copy

jupyterp-retest-challenge-copy-copy-copy-copy

jupyterp-putting1

jupyterp-putting2

jupyterp-putting3

jupyterp-putting4

jupyterp-tensorboard

jupyterp-tensorboard-graphs

jupyterp-tensorboard-scalers

jupyterp-tensorboard-training

GUI-Tensorboard

GUI-Tensorboard-graphs

GUI-Tensorboard-scalers

GUI-Tensorboard-training

GUI-Chapter04

GUI-No-Jupyter-Tensorboard

This course is designed to provide you with an easy-to-follow, structured, incremental, and from-first-principles approach to learning PyTorch. In this course, you’ll be introduced to the fundamentals of PyTorch: autograd, model classes, datasets, data loaders, and more. You will develop, step-by-step, not only the models themselves but also your understanding of them. You'll be shown both the reasoning behind the code and how to avoid some common pitfalls and errors along the way. By the time you finish this course, you’ll have a thorough understanding of the concepts and tools necessary to start developing and training your own models using PyTorch.

Deep Learning with PyTorch Step-by-Step: Part I - Fundamentals

# Defining the appropriate loss

We already have a model, and now we need to define an appropriate loss for it. A binary classification problem calls for the binary cross-entropy (BCE) loss, which is sometimes known as **log loss**.

The **BCE loss** requires the predicted probabilities, as returned by the sigmoid function, and the **true labels** (`y`) for its computation. For each data point `i` in the training set, it starts by computing the error corresponding to the point's true class.

If the data point belongs to the positive class (`y=1`), we would like our model to predict a probability close to one, right? A perfect one would result in the logarithm of one, which is zero. It makes sense; a perfect prediction means zero loss. It goes like this:

<div style="font-size: 1.4em;">
<center>

$y_i = 1 => error_i = log(P(y_i = 1))$

</center>
</div>

What if the data point belongs to the negative class (`y=0`)? Then, we cannot simply use the predicted probability. Why not? Because the model outputs the probability of a point belonging to the positive, not the negative class. Luckily, the latter can be easily computed:

<div style="font-size: 1.4em;">
<center>

$P(y_i = 0) = 1 - P(y_i = 1)$

</center>
</div>

And thus, the error associated with a data point belonging to the negative class goes like this:

<div style="font-size: 1.4em;">
<center>

$y_i = 0 => error_i = log(1 - P(y_i = 1))$

</center>
</div>

Once all errors are computed, they are aggregated into a loss value. 

# Binary cross-entropy loss

For the binary cross-entropy loss, we simply take the average of the errors and invert its sign. This will have the following equation:

<div style="font-size: 1.1em;">
<center>

$BCE(y) = -\dfrac{1}{(N_{pos} \space + \space N_{neg})} \begin{bmatrix}{\sum_{i=1}^{N_{pos}}} log(P(y_i = 1))+ \sum_{i=1}^{N_{neg}} log(1 - P(y_i = 1))\end{bmatrix}$

</center>
</div>

Let us assume we have two dummy data points, one for each class. Then, let us pretend our model made predictions for them: 0.9 and 0.2. The predictions are not bad since it predicts a 90% probability of being positive for an actual positive and only 20% of being positive for an actual negative. What does this look like in code? Here it is:

# Defining the appropriate loss

We already have a model, and now we need to define an appropriate loss for it. A binary classification problem calls for the binary cross-entropy (BCE) loss, which is sometimes known as **log loss**.

The **BCE loss** requires the predicted probabilities, as returned by the sigmoid function, and the **true labels** (`y`) for its computation. For each data point `i` in the training set, it starts by computing the error corresponding to the point's true class.

If the data point belongs to the positive class (`y=1`), we would like our model to predict a probability close to one, right? A perfect one would result in the logarithm of one, which is zero. It makes sense; a perfect prediction means zero loss. It goes like this:

<div>
<center>

$y_i = 1 => error_i = log(P(y_i = 1))$

</center>
</div>

What if the data point belongs to the negative class (`y=0`)? Then, we cannot simply use the predicted probability. Why not? Because the model outputs the probability of a point belonging to the positive, not the negative class. Luckily, the latter can be easily computed:

<div>
<center>

$P(y_i = 0) = 1 - P(y_i = 1)$

</center>
</div>

And thus, the error associated with a data point belonging to the negative class goes like this:

<div>
<center>

$y_i = 0 => error_i = log(1 - P(y_i = 1))$

</center>
</div>

Once all errors are computed, they are aggregated into a loss value. 

# Binary cross-entropy loss

For the binary cross-entropy loss, we simply take the average of the errors and invert its sign. This will have the following equation:

<div>
<center>

$BCE(y) = -\dfrac{1}{(N_{pos} \space + \space N_{neg})} \begin{bmatrix}{\sum_{i=1}^{N_{pos}}} log(P(y_i = 1))+ \sum_{i=1}^{N_{neg}} log(1 - P(y_i = 1))\end{bmatrix}$

</center>
</div>

Let us assume we have two dummy data points, one for each class. Then, let us pretend our model made predictions for them: 0.9 and 0.2. The predictions are not bad since it predicts a 90% probability of being positive for an actual positive and only 20% of being positive for an actual negative. What does this look like in code? Here it is:

Understand the steps in computing the appropriate loss, and explore the binary-cross entropy loss. 

Loss

Understand the steps in computing the appropriate loss, and explore the binary-cross entropy loss.

Introduction

Visualizing Gradient Descent

A Simple Regression Problem

Rethinking the Training Loop

Going Classy

A Simple Classification Problem

Conclusion

Appendix

Loss

Defining the appropriate loss