Discover fundamental deep learning concepts, master coding models using NumPy and Keras, and test your knowledge with quizzes and coding challenges. Gain insights into effective deep learning techniques.

Dockerfile.tar.gz

keras python

keras python - using models

This beginner level and highly comprehensive course is intended for learners who are familiar with Python programming. You will become familiar with the fundamental concepts and terminologies used in deep learning. In addition, this course will help you understand the importance of deep learning techniques. You will examine simple models like perceptron before learning more complex yet powerful deep learning models. The course will provide hands-on practical knowledge of how to code simple and complex deep learning models in NumPy, a powerful Python library and Keras, a cutting-edge library for deep learning in Python. You can test your knowledge with the quizzes that are provided at the end of every lesson and coding challenges that will help you gain a higher understanding. By the end of the course, you should have a general understanding of the basics in deep learning and you will be equipped with the right tools to learn more advanced concepts.

A Beginner's Guide to Deep Learning

# Exploratory data analysis
We have two features X1 and X2 and a label. Each feature has ten data points and the label is 0 or 1. Make a decision boundary that separates the two classes.
Let's look at the #concept=Dataset#dataset.#concept#


> 📝 Note: Remember, we can only apply the perceptron algorithm if the data is linearly separable. For this, we need to draw the points on the graph to visualize the data.


Draw the data points on the 
#concept=EDA#graph #concept#.





 

import numpy as np
import matplotlib.pyplot as plt

def sigmoid(z):
    """The sigmoid activation function on the input x"""
    return 1 / (1 + np.exp(-z))

def forward_propagation(X, W, b):
    """
     Computes the forward propagation operation of a perceptron and 
     returns the output after applying the sigmoid activation function
    """
    weighted_sum = np.dot(X, W) + b # calculate the weighted sum of X and W
    prediction = sigmoid(weighted_sum) # apply the sigmoid activation function
    return prediction


def calculate_error(y, y_predicted):
   """Computes the binary cross entropy error"""
   loss = np.sum(- y * np.log(y_predicted) - (1 - y) * np.log(1 - y_predicted)) # calculate error
   return loss

def gradient(X, Y, Y_predicted):
    """"Gradient of weights and bias"""
    Error = Y_predicted - Y # Calculate error
    dW = np.dot(X.T, Error) # Compute derivative of error w.r.t weight, i.e., (target - output) * x
    db = np.sum(Error) # Compute derivative of error w.r.t bias
    return dW, db # return derivative of weight and bias

def update_parameters(W, b, dW, db, learning_rate):
    """Updating the weights and bias value"""
    W = W - learning_rate * dW # update weight
    b = b - learning_rate * db # update bias
    return W, b # return weight and bias


def train(X, Y, learning_rate, W, b, epochs, losses):
    """Training the perceptron using batch update"""
    for i in range(epochs): # loop over the total epochs
        Y_predicted = forward_propagation(X, W, b) # compute forward pass
        losses[i, 0] = calculate_error(Y, Y_predicted) # calculate error
        dW, db = gradient(X, Y, Y_predicted) # calculate gradient
        W, b = update_parameters(W, b, dW, db, learning_rate) # update parameters

    return W, b, losses

# Initialize parameters 
# features
X = np.array(
   [[2.78, 2.55],
	[1.46, 2.36],
	[3.39, 4.40],
	[1.38, 1.85],
	[3.06, 3.00],
	[7.62, 2.75],
	[5.33, 2.08],
	[6.92, 1.77],
	[8.67, -0.24],
	[7.67, 3.50]]) 

Y = np.array([0, 0, 0, 0, 0, 1, 1, 1, 1, 1]) # target label
weights = np.array([0.0, 0.0]) # weights of perceptron
bias = 0.0 # bias value
epochs = 10000 # total epochs
learning_rate = 0.01 # learning rate
losses = np.zeros((epochs, 1)) # compute loss
print("Before training")
print("weights:", weights, "bias:", bias)
print("Target labels:", Y)
W, b, losses = train(X, Y, learning_rate, weights, bias, epochs, losses)

# Evaluating the performance 
plt.figure() 
plt.plot(losses) 
plt.xlabel("EPOCHS") 
plt.ylabel("Loss value") 
plt.show() 
plt.savefig('output/legend.png')

print("\nAfter training")
print("weights:", W, "bias:", b)
# Predict value
A2 = forward_propagation(X, W, b)
pred = (A2 > 0.5) * 1

print("Predicted labels:", pred)

# Explanation

## Initialization parameters
The table summarizes the initialized parameters:

|**Variables**| **Definition**|   
|:---:|:---:|
|`X`| An input feature array of size 10 * 2|
|`Y`| The output labels of size 10 * 1|
|`weights`|  An array of size 1 * 2 initialized with 0|
|`bias`| The bias of size 1 * 1 initialized with 0.0|
|`learning_rate`| The learning rate initialized with value 0.1|
|`epochs`| The epochs initialized with 10000|


After initializing the parameters, the perceptron is trained by calling the train function.


## Training the perceptron

> 📝 **Note:** For the Batch Gradient Descent, it is important to make the variables compatible for multiplication and dot product by taking transpose (when required). The dimensions of the input parameters are specified in the parenthesis under the function explanation for a better understanding of concepts.


**`train` function:**

- It takes the features `X`, labels `Y`, `weights`, `bias`, and the `learning_rate`.
- A `for` loop iterates `epochs` times while updating the weights and bias in a batch manner.
Within the epoch:
    -  Calls the `forward propagation` to compute the predicted value and saves the return value in `Y_predicted`.
     - Calls the `compute_error` function to calculate error in each epoch and saves the return value in the `losses` array.
     - Calls the `gradient` function to compute the gradient of the error with respect to the weights and bias.
     -  Calls the `update_parameters` function that returns the updated value of weights and bias.


**`forward_propagation` function:**

The function takes the input `X`, weight `W`, and `bias`.
- The dimensions of `X` are 10 * 2 and that of `W` are 2 * 1.
- `np.dot` computes the `weighted_sum` of size `10 * 1`:
  ```
   weighted_sum(10 * 1) = X(10 * 2) . W(2 * 1) + bias (1 * 1) 
  ```  
- The `sigmoid` activation function is applied to the `weighted_sum`.
- We get an output vector of size `10 * 1`.

**`gradient_descent` function:**

The function takes in the input `X`, actual labels `Y_predicted`, and target labels `Y`.
- It computes the error by subtracting `Y` from `Y_predicted`. Note that the dimensions of both are 10 * 1 and we get an output of 10 * 1.
  ```
  Error(10 * 1) = Y_predicted(10 * 1) - Y(10 * 1) 
  ```
- Then, it computes the derivative of error w.r.t weight. The input vector `X` is of size 10 * 2. We must take the transpose for making it compatible to the dot product with `Error`.
   > 📝 Note: While calculating the derivative of error w.r.t bias, we sum the values across the y-axis to get a vector of size 1 * 1. 
 
  ```Python
   dW(2 * 1) = X.T(2 * 4) . Error(4 * 1) 
  ```
- The derivative of error w.r.t bias is the error:
  ```
  db(1 * 1) = sum(Error(10 * 1))
  ``` 

**`update_parameters` function:**

The function takes in the parameters, weights `W`, and bias `b`, along with their derivatives, `dW` and `db` respectively and applies the update rule:

  ```Python
  W(2 * 1) = W(2 * 1) -  learning_rate (1 * 1) * dW(2 * 1)

b(1 * 1) = b(1 * 1) -  learning_rate  * db(1 * 1)
  ```


This method of updating weights is known as **Batch Gradient Descent**. 


The following illustration will help visualize Batch Gradient Descent:

> 📝 For simplicity, we have considered only 2 data points.



Update the parameters using the Batch Gradient Descent.

✨Before We Begin

✨Introduction to Deep Learning

✨Simple Perceptron Models in NumPY

✨Towards Deep Neural Networks in NumPY

🖥️ Project: Build a Letter Classification Model

Deep Learning in NumPy Exam

✨Building Deep Learning Models with Keras

✨Fine-tune Keras Model

🖥️ Project: Build a Digit Recognition Model

✨Conclusion

Deep Learning in Keras Exam

Gradient Descent: The Batch Update

Exploratory data analysis