kaggle_username

kaggle_key

Gain insights into Generative Adversarial Networks with PyTorch. Delve into neural networks, adversarial training, and building GANs for image generation, including conditional GANs for class-specific data.

GAN.tar.gz

torch

J03T

J03O

J03P

J04AF

J04OF

J04N

J04CR

J05UCP

J07TMG

J07TTMG

J16TG

J16CGR

JAppebdixB

JAppebdixD

SPA04

SPA16

SPA13

SPA09

This course is an introduction to Generative Adversarial Networks (GANs) and a practical step-by-step tutorial on making your own with PyTorch. Through this course, you will learn how to build GANs with industry-standard tools.

In the first section, you will dive into PyTorch and refresh your understanding of neural networks by building a simple image classifier. In the second section of this course, you will explore the idea of adversarial training and build progressively more sophisticated GANs; first by learning a simple 1010 pattern, then monochrome images of handwritten digits, and finally full-color images of faces.

The third, and last section, extends the core GAN concept, applying it to convolutional neural networks, and developing a conditional GAN for generating data of the desired class.

By the end of this course, you will have a solid understanding of how to build GANs for your machine learning projects.

Make Your First GAN Using PyTorch

The code is similar to the code we used for training the simple 1010 pattern [here](https://www.educative.io/courses/make-your-first-gan-pytorch/training-the-1010-pattern-gan).

# Training the 01110 pattern GAN

We’re ready to train the GAN using the **3** step training loop. Have a look at the following code:

import torch
import torch.nn as nn

# generator class

class Generator(nn.Module):
    
    def __init__(self):
        # initialise parent pytorch class
        super().__init__()
        
        # define neural network layers
        self.model = nn.Sequential(
            nn.Linear(1, 3),
            nn.Sigmoid(),
            nn.Linear(3, 5),
            nn.Sigmoid()
        )

        # create optimiser, simple stochastic gradient descent
        self.optimiser = torch.optim.SGD(self.parameters(), lr=0.01)

        # counter and accumulator for progress
        self.counter = 0;
        self.progress = []
        
        pass
    
    
    def forward(self, inputs):        
        # simply run model
        return self.model(inputs)
    
    
    def train(self, D, inputs, targets):
        # calculate the output of the network
        g_output = self.forward(inputs)
        
        # pass onto Discriminator
        d_output = D.forward(g_output)
        
        # calculate error
        loss = D.loss_function(d_output, targets)

        # increase counter and accumulate error every 10
        self.counter += 1;
        if (self.counter % 10 == 0):
            self.progress.append(loss.item())
            pass

        # zero gradients, perform a backward pass, update weights
        self.optimiser.zero_grad()
        loss.backward()
        self.optimiser.step()

        pass
    
    
    def plot_progress(self):
        df = pandas.DataFrame(self.progress, columns=['loss'])
        df.plot(ylim=(0, 1.0), figsize=(16,8), alpha=0.1, marker='.', grid=True, yticks=(0, 0.25, 0.5))
        pass
    
    pass

import torch
import torch.nn as nn

# discriminator class

class Discriminator(nn.Module):
    
    def __init__(self):
        # initialise parent pytorch class
        super().__init__()
        
        # define neural network layers
        self.model = nn.Sequential(
            nn.Linear(5, 3),
            nn.Sigmoid(),
            nn.Linear(3, 1),
            nn.Sigmoid()
        )
        
        # create loss function
        self.loss_function = nn.MSELoss()

        # create optimiser, simple stochastic gradient descent
        self.optimiser = torch.optim.SGD(self.parameters(), lr=0.01)

        # counter and accumulator for progress
        self.counter = 0;
        self.progress = []

        pass
    
    
    def forward(self, inputs):
        # simply run model
        return self.model(inputs)
    
    
    def train(self, inputs, targets):
        # calculate the output of the network
        outputs = self.forward(inputs)
        
        # calculate loss
        loss = self.loss_function(outputs, targets)

        # increase counter and accumulate error every 10
        self.counter += 1;
        if (self.counter % 10 == 0):
            self.progress.append(loss.item())
            pass
        if (self.counter % 10000 == 0):
            print("counter = ", self.counter)
            pass

        # zero gradients, perform a backward pass, update weights
        self.optimiser.zero_grad()
        loss.backward()
        self.optimiser.step()

        pass
    
    
    def plot_progress(self):
        df = pandas.DataFrame(self.progress, columns=['loss'])
        df.plot(ylim=(0, 1.0), figsize=(16,8), alpha=0.1, marker='.', grid=True, yticks=(0, 0.25, 0.5))
        pass
    
    pass

# import libraries
import torch
import torch.nn as nn

import pandas
import matplotlib.pyplot as plt
import random
import numpy

from Discriminator import Discriminator
from Generator import Generator

def generate_real():
    real_data = torch.FloatTensor(
        [random.uniform(0.0, 0.2),
         random.uniform(0.8, 1.0),
         random.uniform(0.8, 1.0),
         random.uniform(0.8, 1.0),
         random.uniform(0.0, 0.2)])
    return real_data

def generate_random(size):
    random_data = torch.rand(size)
    return random_data

# create Discriminator and Generator
D = Discriminator()
G = Generator()

# define an image list to store how output evolves during training
image_list = []

# train Discriminator and Generator
for i in range(10000):
    
    # train discriminator on true
    D.train(generate_real(), torch.FloatTensor([1.0]))
    
    # train discriminator on false
    # use detach() so gradients in G are not calculated
    D.train(G.forward(torch.FloatTensor([0.5])).detach(), torch.FloatTensor([0.0]))
    
    # train generator
    G.train(D, torch.FloatTensor([0.5]), torch.FloatTensor([1.0]))
    
    # add image to list every 1000
    if (i % 1000 == 0):
      image_list.append( G.forward(torch.FloatTensor([0.5])).detach().numpy() )

    pass

# manually run generator to see it's outputs
print("Generator's output after training:", G.forward(torch.FloatTensor([0.5])))


# plot images collected during training
plt.figure(figsize = (16,8))
plt.imshow(numpy.array(image_list).T, interpolation='none', cmap='Blues')
plt.savefig('output/legend.png')


import warnings
warnings.filterwarnings("ignore", category=UserWarning)



## Discriminator and generator objects 
We first create fresh discriminator and generator objects, before running the training loop **10,000** times. 

## Training loop

Let's examine the `main.py` file. 
Inside the loop, we can see the **3** steps of the GAN training loop we talked about earlier. 

- For **step 1**, we can see the discriminator being trained on the real data (*line 37*).

- For **step 2**, we train the discriminator with a pattern from the generator (*line 41*). Note that `detach()` applied to the output of the generator detaches it from the computation graph.

- For **step 3**, we train the generator, passing it the discriminator object, and a generator input value of **0.5**  (*line 44*).



Finally, we manually run the generator to see the output (_line 53_). The generator is well trained to generate the pattern $01110$.🎉

**Lines 57-79** help to visualize how the $01110$ pattern evolves during training. This chart shows quite clearly how the generator improves over time.🎉

Learn how we can now train the 01110 pattern GAN.

✨Introduction

✨PyTorch Basics

🖥️ Project: Computation Graph and Gradient Calculation

✨First PyTorch Neural Network

✨Refinements

🖥️ Project: Fashion MNIST Classification Model

✨CUDA Basics

Make Your First GAN: Part 1 - Exam

✨The GAN Idea

✨Simple 1010 Pattern

🖥️ Project: A Simple 01110 Pattern GAN

✨Handwritten Digits

🖥️ Project: Fashion MNIST GAN

✨Human Faces

Make Your First GAN: Part 2 - Exam

✨Convolutional GANs

🖥️ Project: A Fashion MNIST CNN

✨Conditional GANs

🖥️ Project: Fashion MNIST Conditional GAN

Make Your First GAN: Part 3 - Exam

✨Conclusion

✨Appendix

Training the 01110 Pattern GAN

Training the 01110 pattern GAN