Delve into PyTorch basics: autograd, model classes, datasets, and data loaders. Gain insights into model development while avoiding common pitfalls. Start creating and training your own PyTorch models.

Deep learning with Pytorch, A beginner's Guide By Daniel Godoy.png

jupyter_final.tar.gz

codesnip1

tensorboard

jupyterp-autograd

jupyterp-computation

jupyterp-rethinking

jupyterp-minibatch

jupyterp-dataloader

jupyterp-evaluation

jupyterp-saveload

jupyterp-predictions

jupyterp-training-predictions

jupyterp-checkpointing

jupyterp-classy

jupyterp-classification

jupyterp-retest

jupyterp-retest-copy

jupyterp-retest-copy-copy

jupyterp-retest-copy-copy-copy

jupyterp-retest-copy-copy-copy-copy

jupyterp-retest-question

jupyterp-retest-question-copy

jupyterp-retest-question-copy-copy

jupyterp-retest-question-copy-copy-copy

jupyterp-retest-question-copy-copy-copy-copy

jupyterp-retest-challenge

jupyterp-retest-challenge-copy

jupyterp-retest-challenge-copy-copy

jupyterp-retest-challenge-copy-copy-copy

jupyterp-retest-challenge-copy-copy-copy-copy

jupyterp-putting1

jupyterp-putting2

jupyterp-putting3

jupyterp-putting4

jupyterp-tensorboard

jupyterp-tensorboard-graphs

jupyterp-tensorboard-scalers

jupyterp-tensorboard-training

GUI-Tensorboard

GUI-Tensorboard-graphs

GUI-Tensorboard-scalers

GUI-Tensorboard-training

GUI-Chapter04

GUI-No-Jupyter-Tensorboard

This course is designed to provide you with an easy-to-follow, structured, incremental, and from-first-principles approach to learning PyTorch. In this course, you’ll be introduced to the fundamentals of PyTorch: autograd, model classes, datasets, data loaders, and more. You will develop, step-by-step, not only the models themselves but also your understanding of them. You'll be shown both the reasoning behind the code and how to avoid some common pitfalls and errors along the way. By the time you finish this course, you’ll have a thorough understanding of the concepts and tools necessary to start developing and training your own models using PyTorch.

Deep Learning with PyTorch Step-by-Step: Part I - Fundamentals

# General overview

This finishes our journey through the inner workings of gradient descent. By now, we hope you are able to develop better intuition about the many different aspects involved in the process. 

In time and with practice, you will observe the behaviors described here in your own models. Make sure to try plenty of different combinations: mini-batch sizes, learning rates, etc. This way, not only your models will learn but so will you!

This is a (not so) short recap of everything we covered in this chapter:

* Defining a **simple linear regression model**.

* Generating **synthetic data** for it.
* Performing a **train-validation split** on our dataset.
* **Randomly initializing the parameters** of our model.
* Performing a **forward pass**; that is, making predictions using our model.
* Computing the **errors** associated with our predictions.
* Aggregating the errors into a **loss** (MSE).
* Learning that the **number of points** used to compute the loss defines the kind of gradient descent we're using: **batch** (all), **mini-batch**, or **stochastic** (1).
* Visualizing an example of a **loss surface** and using its **cross-sections** to get the **loss curves** for individual parameters.
* Learning that a **gradient is a partial derivative**, and it represents **how much the loss changes if one parameter changes a little bit**.

* Computing the gradients for our model's parameters using **equations**, **code**, and **geometry**.

* Learning that **larger gradients** correspond to **steeper loss curves**.

* Learning that **backpropagation** is nothing more than "chained" gradient descent.

* Using the **gradients** and a **learning rate** to **update the parameters**.

* Comparing the **effects on the loss** of using **small**, **big**, and **very big** learning rates.

* Learning that **loss curves** for all parameters should be, ideally, **similarly steep**.

* Visualizing the effects of using a **feature with a larger range**, making the loss curve for the corresponding parameter **much steeper**.

* Using Scikit Learn's `StandardScaler` to bring a feature to a reasonable range, and thus making the **loss surface more bowl-shaped** and its cross-sections **similarly steep**.

* Learning that **preprocessing steps** like scaling should be applied **after the train-validation split** to prevent **leakage**.

* Figuring that performing **all steps** (forward pass, loss, gradients, and parameter update) makes **one epoch**.

* Visualizing the **path of gradient descent** over many epochs and realizing it is heavily **dependent on the kind of gradient descent** used: batch, mini-batch, or stochastic.

* Learning that there is a **trade-off** between the stable and smooth path of batch gradient descent and the fast and somewhat chaotic path of stochastic gradient descent, making the use of **mini-batch gradient descent a good compromise** between the other two.

You are now ready to put it all together and actually train a model using PyTorch!

# Jupyter notebook

To understand the concepts tested in this chapter, you have been provided with the following Jupyter notebook, which contains the entire code that has been discussed in this chapter. You can run the complete code to see the results for yourself, and play around and experiment with it. 

Let's have a quick overview of what you have learned and test it on the given notebook. 

Recap

Introduction

Visualizing Gradient Descent

A Simple Regression Problem

Rethinking the Training Loop

Going Classy

A Simple Classification Problem

Conclusion

Appendix

Recap

General overview