Gain insights into basic and intermediate deep learning concepts, including CNNs, RNNs, GANs, and transformers. Delve into fundamental architectures to enhance your machine learning model training skills.

docker.tar.gz

Pytorch

GANS

This course is an accumulation of well-grounded knowledge and experience in deep learning. It provides you with the basic concepts you need in order to start working with and training various machine learning models. 

You will cover both basic and intermediate concepts including but not limited to: convolutional neural networks, recurrent neural networks, generative adversarial networks as well as transformers.

After completing this course, you will have a comprehensive understanding of the fundamental architectural components of deep learning. Whether you’re a data and computer scientist, computer and big data engineer, solution architect, or software engineer, you will benefit from this course.

Introduction to Deep Learning & Neural Networks

> Optimization is the selection of the best element (with regard to some criterion) from a set of available alternatives.

In the simplest case, an optimization problem consists of maximizing or minimizing a real function by systematically choosing input values from within an allowed set and computing the value of the function. 

In the case of machine learning, **optimization refers to minimizing the loss function by systematically updating the network weights**. Mathematically, this is expressed as:

$$ w^{\prime} = arg min_w C(w)$$ 

given a loss function $C$ and weights $w$.

Intuitively, it can be thought of as descending a high-dimensional landscape. If we could project it in 2D plot, the height of the landscape would be the value of the loss function, and the horizontal axis would be the values of our weights w. Ultimately, the goal is to reach the bottom of the landscape by iteratively exploring the space around us.

# Gradient descent

Gradient descent is based on the basic idea of following the local slope of our landscape. We essentially introduce physics and the law of gravity in the mix. Calculus provides us with an elegant way to calculate the slope of the landscape, which is the derivative of the function at this point (also known as the gradient) with respect to the weights.

$$ w = w - \lambda \cdot \nabla_w C(w) $$

> Learning rate ($\lambda$) is a constant value that determines the step size at each iteration, while moving toward a minimum of a loss function.


Algorithmically, this can be expressed in Python as below:

```python
for t in range(steps):
  dw = gradient(loss, data, w)
  w = w - learning_rate *dw
```

Visually, we can imagine the following diagram which corresponds to a 2d space:

Explore the different variations of the gradient descent algorithm.

Learn Deep Learning

Neural Networks

Training Neural Networks

Convolutional Neural Networks

Recurrent Neural Networks

Autoencoders

Generative Adversarial Networks

Attention and Transformers

Graph Neural Networks

Conclusion

Final Quiz

Optimization

Gradient descent