How does CNN work in computer vision tasks?

We, as humans, understand and make meaningful inferences from what we see. But how do we make computers learn and infer meaningful data from a set of images and videos?

Computer vision (CV) is the field of study that focuses on enabling computers to understand and interpret visual information. It aims to replicate human visual perception and processing capabilities using computer algorithms and models. Computer vision involves a range of tasks, including image classification, object detection, and image segmentation.

Before discussing the role of CNNs in CV tasks, let’s explore convolutional neural networks in detail.

Convolutional neural network

Convolutional neural networks (CNNs) are a type of deep learning model that are designed to process grid-like structures such as images. CNNs are highly suited to computer vision tasks such as facial recognition systems and self-driving cars.

Let's visualize how a CNN is different from a typical neural network.

A typical neural network model inputs a 1D vector for its training, but in the case of visual data, this serves as a challenge. Each image is a 2-dimensional matrix, where each element of the matrix contains a certain color. This image can be converted into a 1D vector by flattening, but this will make it lose a lot of spatial information in the image. To resolve this, researchers came up with the CNN architecture, which is more suited to handle 2D input.

How do CNNs work in CV?

A basic convolutional neural network typically has 3 types of layers: a convolutional layer, a pooling layer, and a fully connected layer along with the input and output layer.

Let's discuss in detail what each layer does.

• Input layer: The CNN takes an input image represented as a grid of pixels. For color images, each pixel has three color channels (red, green, and blue), while for grayscale images, there is only one channel.

Note: ImageNet is a large-scale dataset that contains millions of labeled images spanning thousands of different categories for data collection in computer vision tasks.

• Convolutional layer: The convolutional layer in CNN performs the convolution operation using learnable filters or kernels. These kernels slide over the input image, multiplying their weights with the corresponding pixels in the receptive field and summing the results to produce a feature map. Multiple kernels capture different features, which allows the network to learn a wide range of patterns.

• Activation function: After the convolution operation, an activation function (e.g., ReLU) is applied element-wise to introduce nonlinearity into the network. This allows CNNs to learn complex relationships between the extracted features.

• Pooling layers: Pooling layers are used to downsample the feature maps generated by the convolutional layers. Pooling reduces the spatial dimensionality of the feature maps while preserving the most important information.

• Repeat convolution, activation, and pooling: Each subsequent layer learns more abstract features by building upon the representations learned in the previous layers.

• Flattening: After several convolutional and pooling layers, the resulting feature maps are flattened into a one-dimensional vector. This collapses the spatial structure of the features into a linear representation.

• Fully connected layers: The flattened vector is fed into fully connected layers, which perform traditional neural network operations. These layers learn to classify the input based on the extracted features. The final fully connected layer produces the output, representing the predicted class probabilities or specific values for the task at hand.

• Training and optimization: During the training phase, the CNN's parameters (filter weights, biases, etc.) are optimized to minimize a defined loss function. This is done through backpropagation, where gradients are computed and used to update the parameters via optimization algorithms like gradient descent. The training process adjusts the network's weights to improve its ability to make accurate predictions.

Note: ResNet50 is a popular pre-trained model for CV tasks trained on ImageNet.

• Inference: Once the CNN is trained, it can be used for inference on new, unseen images. The forward pass through the network generates predictions or class probabilities for the given input image.

Now that we have examined the basics of how CNNs work in CV, let's discuss how CNN works in specific tasks.

Image classification by CNN

In image classification, CNNs analyze input images using a series of convolutional layers to extract meaningful features. These features are then fed into fully connected layers that learn to classify the image into different categories or predict class probabilities. By automatically learning and capturing relevant patterns and features, CNNs have achieved state-of-the-art accuracy in classifying images across various domains.

Object detection by CNN

Object detection goes beyond image classification by not only recognizing objects but also localizing them within the image. CNN-based object detection models typically incorporate region proposal networks or anchor-based methods. These models utilize convolutional layers to generate object proposals and classify and precisely localize objects within the proposed region.

Image segmentation by CNN

Image segmentation involves pixel-level labeling of an image to partition it into meaningful regions. CNNs have made significant advancements in image segmentation by utilizing encoder-decoder architectures. The encoder network processes the input image through convolutional and pooling layers to extract high-level features. The decoder network then upsamples the features and refines them through convolutional layers, generating dense predictions for each pixel.

Conclusion

Convolutional neural networks (CNNs) have had a significant impact on computer vision tasks, providing powerful tools for image analysis and understanding. By leveraging hierarchical feature extraction, CNNs can automatically learn and extract meaningful features from images applicable in diverse fields, including medicine and autonomous driving. Despite challenges like data requirements and computational resources, the future of CNNs holds the potential for enhanced interpretability, efficiency, and domain-specific advancements.