PyTorch library in Python
PyTorch is a machine-learning library developed by scientists at Facebook. This library offers efficient computation means that we help train large models. Also, there are built-in models like VGG16, ResNet18, WideResNet, etc. We can perform transfer learning with the help of PyTorch and build models as per your requirements.
Let us discuss the functionalities provided by this library.
Arithmetic operations
Basic arithmetic operations, such as adding, subtracting, and multiplying between two numbers, are supported by PyTorch. The syntax is simple; for example, to add two numbers, we have a function torch.add(). Have a code below to understand the functionalities. You can practice by passing different parameters as well.
import torchimport numpy as np# Creating tensorsa = torch.tensor([1, 2, 3])b = torch.tensor([4, 5, 6])# Additionadd = torch.add(a, b)# Subtractionsub = torch.sub(a, b)# Multiplicationmul = torch.mul(a, b)# Divisiondiv = torch.div(a, b)# Element-wise powerpower = torch.pow(a, 2)print("Addition:", add)print("Subtraction:", sub)print("Multiplication:", mul)print("Division:", div)print("Power:", power)
Reduction operations
The PyTorch library summarizes the meaningful information from the large data set. Reduction operations include various statistics methods such as mean, median, and mode, and extracting minimum and maximum from the data. It helps in making decisions related to the dataset. The following example torch.min() returns the minimum and index of the array where that minimum is present. Have a look at the torch.max() as well.
import torch# Creating a tensora = torch.tensor([17, 26, 3, 42, 51])# Sum of all elementsresult_sum = torch.sum(a)# Maximum element and its indexresult_max, index_max = torch.max(a, dim=0)# Minimum element and its indexresult_min, index_min = torch.min(a, dim=0)print("Sum:", result_sum)print("Max and Index:", result_max, index_max)print("Min and Index:", result_min, index_min)
Slicing operations
Slicing extracts a contiguous subset of elements from the original array. There are various functions of slicing in an array, some of which are torch.index_select(), torch.gather(), and torch.masked() . The examples of each of these functions are given below.
import torch# Creating a tensora = torch.tensor([[1, 2, 3], [4, 5, 6], [7, 8, 9]])# Index selectselected_rows = torch.index_select(a, dim=0, index=torch.tensor([0, 2]))# Gathergathered_elements = torch.gather(a, dim=1, index=torch.tensor([[0, 2], [1, 0], [2, 1]]))# Masked selectmask = a > 5masked_elements = torch.masked_select(a, mask)print("Index Select:", selected_rows)print("Gather:", gathered_elements)print("Masked Select:", masked_elements)
Concatenation
In the process of concatenation, we join two strings. This powerful function plays a significant role while dealing with the data. To concatenate an array along rows, we have the function torch.cat(). On the other hand, to combine an array along the columns, we use torch.stack() function. We can also an array split with the help of torch.split().
import torch# Creating tensorsa = torch.tensor([[3, 4]])b = torch.tensor([[5, 6]])# Concatenate along rows (dimension 0)result_cat = torch.cat((a, b), dim=0)# Concatenate along columns (dimension 1)result_stack = torch.stack((a, b), dim=1)# Split tensor along rows (dimension 0)result_split = torch.split(result_cat, split_size_or_sections=1, dim=0)print("Concatenate:", result_cat)print("Stack:", result_stack)print("Split:", result_split)
Reshaping operations
This function plays a significant role in image processing because we must reshape an image concerning the model's input. Some reshaping operations in PyTorch are given below.
import torch# Creating a tensora = torch.tensor([[1, 2], [3, 4], [5, 6]])# Reshape tensorreshaped_a = torch.reshape(a, (2, 3))# Squeeze tensor to remove dimensions with size 1squeezed_a = torch.squeeze(a)# Unsqueeze tensor to add dimensions with size 1unsqueezed_a = torch.unsqueeze(a, dim=1)# Transpose tensortransposed_a = torch.transpose(a, 0, 1)print("Reshaped:", reshaped_a)print("Squeezed:", squeezed_a)print("Unsqueezed:", unsqueezed_a)print("Transposed:", transposed_a)
Layers in neural network
PyTorch has allowed us to make layers in the neural network depending on the model's architecture. For example, nn.Linear() is mainly used if we are required to perform a linear regression, nn.conv2d() is used to extract the features from an image. Find the explanation and syntax of other layers in the code below.
import torchimport torch.nn as nn# Linear layerlinear_layer = nn.Linear(in_features=10, out_features=5)# Convolutional layerconv_layer = nn.Conv2d(in_channels=3, out_channels=16, kernel_size=3, stride=1, padding=1)# LSTM layerlstm_layer = nn.LSTM(input_size=10, hidden_size=20, num_layers=2, batch_first=True)# Batch Normalization layer (for 2D convolutional layers)batch_norm_layer = nn.BatchNorm2d(num_features=16)# Activation function: ReLUactivation_relu = nn.ReLU()# Activation function: Sigmoidactivation_sigmoid = nn.Sigmoid()# Activation function: Tanhactivation_tanh = nn.Tanh()# Activation function: LeakyReLUactivation_leakyrelu = nn.LeakyReLU(negative_slope=0.01)
Loss functions
The use of loss function is essential for training a neural network model. Loss functions provide details about how well the neural network is performing on the unseen data set. We aim to minimize the loss function throughout our training process. The syntax of loss functions with explanation is in the code.
import torch.nn as nn# CrossEntropyLoss for multi-class classificationcross_entropy_loss = nn.CrossEntropyLoss()# Mean Squared Error Loss for regressionmse_loss = nn.MSELoss()# Binary Cross Entropy Loss for binary classificationbce_loss = nn.BCELoss()# Negative Log Likelihood Loss for multi-class classification with log-probabilitiesnll_loss = nn.NLLLoss()
Optimization algorithms
Optimization algorithms help in achieving the training of the model efficiently. These functions aim at finding the global minimum for the loss functions. lr rate in the optimizers tells us about how small steps we should take. This parameter is chosen based on hit and trial. Sometimes while training, we are stuck at the local minima of the loss function. To cater to this problem, we use momentum; the momentum keeps track of the previous gradients with increased speed, and if it encounters local minima, momentum will escape it.
import torch.optim as optim# Stochastic Gradient Descent optimizersgd_optimizer = optim.SGD(params=model.parameters(), lr=0.01, momentum=0.9)# Adaptive Moment Estimation optimizer (Adam)adam_optimizer = optim.Adam(params=model.parameters(), lr=0.001, betas=(0.9, 0.999))# Root Mean Square Propagation optimizer (RMSprop)rmsprop_optimizer = optim.RMSprop(params=model.parameters(), lr=0.01, alpha=0.9)
Autograd
The beauty of PyTorch lies within the autograd function. The function y.backward() computes the gradients and performs backpropagation. In neural networks, this function saves several lines of code. In the following code, we are computing the gradients of this equation.
import torch# Wrapper class for tensor operations with automatic differentiationx = torch.tensor([1.0, 2.0, 3.0], requires_grad=True)# Computes the gradients of specified variables with respect to some other variablesy = 8 * x + 10y.backward(torch.tensor([1.0, 3.0, 5.0])) # Compute gradients with respect to xprint(x.grad) # Access the gradients using x.grad attribute
Conclusion
In this Answer, we explored the various functionalities provided by the PyTorch library, a robust machine-learning framework developed by Facebook. PyTorch offers efficient computation capabilities, allowing users to train large models effectively. It has built-in models like VGG16, ResNet18, WideResNet, and more, making it easier to work on different tasks. Additionally, PyTorch supports transfer learning, using pre-trained models for specific applications.
What is the function of the optimizer in PyTorch?
It preprocesses data before feeding it to the model.
It computes gradients during backpropagation.
It minimizes the loss function and updates model parameters.
It visualizes the neural network architecture.
Free Resources
