Fundamentals of Machine Learning: A Pythonic Introduction/

...

/

Implementation of a Neural Network

Implementation of a Neural Network

Implement a neural network for binary classification in PyTorch.

We'll cover the following...

Let’s implement a simple neural network example using a popular framework, torch.

Introduction to PyTorch

PyTorch is an open-source deep learning framework, quite popular among researchers, developers, and data scientists due to its simplicity, flexibility, and efficiency in building and training deep neural networks. It’s developed and maintained by Facebook’s AI Research Lab (FAIR). PyTorch provides an intuitive and Pythonic interface that enables users to easily construct complex computational graphs for deep learning models.

Press + to interact

Binary classification for numerical dataset

An example of implementing a simple neural network using PyTorch is given below. In this example, we’ll create a neural network with one hidden layer for a binary classification task. We’ll use the famous Iris dataset and aim to classify whether a given flower belongs to the Iris setosa species or not.

Note: The slight variation in the plots, each time the code is run, can be attributed to the random initialization of the neural network weights and biases, as well as the randomness introduced during the train-test split. This can lead to different initial conditions and data splits, that result in slightly different training trajectories and test performance.

Press + to interact
import torch
import torch.nn as nn
import torch.optim as optim
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
import matplotlib.pyplot as plt
# Load Iris dataset
data = load_iris()
X, y = data.data, data.target
X = X[y != 2]  # Consider only two classes: Iris Setosa (0) and Iris Versicolor (1)
y = y[y != 2]
# Split data into training and testing sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
# Normalize features
scaler = StandardScaler()
X_train = scaler.fit_transform(X_train)
X_test = scaler.transform(X_test)
# Convert NumPy arrays to PyTorch tensors
X_train = torch.tensor(X_train, dtype=torch.float32)
y_train = torch.tensor(y_train, dtype=torch.float32)
X_test = torch.tensor(X_test, dtype=torch.float32)
y_test = torch.tensor(y_test, dtype=torch.float32)
# Define the simple neural network model
class SimpleNeuralNetwork(nn.Module):
    def __init__(self, input_size, hidden_size, output_size):
        super(SimpleNeuralNetwork, self).__init__()
        self.fc1 = nn.Linear(input_size, hidden_size)
        self.relu = nn.ReLU()
        self.fc2 = nn.Linear(hidden_size, output_size)
        self.sigmoid = nn.Sigmoid()
    def forward(self, x):
        out = self.fc1(x)
        out = self.relu(out)
        out = self.fc2(out)
        out = self.sigmoid(out)
        return out
# Set the model parameters
input_size = X_train.shape[1]
hidden_size = 5
output_size = 1
# Initialize the model
model = SimpleNeuralNetwork(input_size, hidden_size, output_size)
# Define the loss function and optimizer
criterion = nn.BCELoss()  # Binary Cross Entropy Loss
optimizer = optim.SGD(model.parameters(), lr=0.01)
# Lists to store training and test losses
train_losses = []
test_losses = []
train_accuracies = []
test_accuracies = []
# Training the model
num_epochs = 1000
for epoch in range(num_epochs):
    # Forward pass
    outputs = model(X_train)
    loss = criterion(outputs, y_train.view(-1, 1))
    # Backward and optimize
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()
    if (epoch + 1) % 100 == 0:
        print(f'Epoch [{epoch+1}/{num_epochs}], Loss: {loss.item():.4f}')
    
    train_losses.append(loss.item())
    
    # Compute training accuracy
    predicted_train = (outputs >= 0.5).squeeze().float()
    train_accuracy = (predicted_train == y_train).sum().item() / y_train.size(0)
    train_accuracies.append(train_accuracy)
    
    # Evaluate the model on the test set
    with torch.no_grad():
        model.eval()
        test_outputs = model(X_test)
        test_loss = criterion(test_outputs, y_test.view(-1, 1))
        test_losses.append(test_loss.item())
        
        # Compute test accuracy
        predicted_test = (test_outputs >= 0.5).squeeze().float()
        test_accuracy = (predicted_test == y_test).sum().item() / y_test.size(0)
        test_accuracies.append(test_accuracy)
# Plot the training and test loss curves
plt.figure(figsize=(12, 5))
plt.subplot(1, 2, 1)
plt.plot(range(1, num_epochs+1), train_losses, label='Training Loss', color='blue')
plt.plot(range(1, num_epochs+1), test_losses, label='Test Loss', color='orange')
plt.xlabel('Epochs')
plt.ylabel('Loss')
plt.legend()
plt.grid(True)
plt.title('Training and Test Loss Curves')
# Plot the training and test accuracy curves
plt.subplot(1, 2, 2)
plt.plot(range(1, num_epochs+1), train_accuracies, label='Training Accuracy', color='blue')
plt.plot(range(1, num_epochs+1), test_accuracies, label='Test Accuracy', color='orange')
plt.xlabel('Epochs')
plt.ylabel('Accuracy')
plt.legend()
plt.grid(True)
plt.title('Training and Test Accuracy Curves')
# Evaluate the model on the test set
with torch.no_grad():
    model.eval()
    test_outputs = model(X_test)
    predicted = (test_outputs >= 0.5).squeeze().float()
    accuracy = (predicted == y_test).sum().item() / y_test.size(0)
    print(f'Test Accuracy: {accuracy:.2f}')

Here is the explanation for the code above:

  • Lines 1–6: We import necessary libraries such as torch, nn, optim, load_iris, train_test_split, and StandardScaler for data preprocessing. We also use matplotlib.pyplot to visualize the insights.

  • Lines 10–27: Data preprocessing involves loading the Iris dataset, filtering it for classes 0 and 1, splitting it into training and test sets, and converting it to PyTorch tensors.

  • Lines 30–47: We define the SimpleNeuralNetwork neural network with two layers: one with a ReLU activation function and the other with a sigmoid activation function.

  • Lines 50–61: We configure by setting model parameters like input_size, hidden_size, and output_size, and create the neural network instance model.

  • Lines 64–69: We define the loss function as BCE loss (nn.BCELoss()) and choose the optimization method as Stochastic Gradient ...