What are Graph Neural Networks?

Neural networks on graphs

Graph neural networks (GNNs) are deep learning methods that work on graph data. They belong to the family of geometric deep learning methods. In recent years, we've seen a rise in the popularity of graph neural networks, for good reason. The application of GNNs has been studied in several domains, and efficient algorithms have been developed. Currently, much research and development are being put into GNNs to extend their applications to solve complex real-world problems.

Until now, we've seen how to generate graph embeddings and formulate different graph analytics tasks. Using GNNs, we can formulate the same tasks, but the algorithmic approach for the problem will be different. The illustration above shows that a GNN manipulates the node features of a graph. We'll learn more about this in the upcoming sections.

GNNs trace their origin back to the nineties when recurrent neural networks were first used on directed acyclic graphs. Later, the algorithms were modified to work on different graph architectures like cyclic graphs. With the advancements in deep neural networks, especially convolutional neural networks, the research on GNNs has sprung up again. The idea of image convolutions has been translated to the graph setting very well, and efficient graph convolutional networks have been developed. This was a breakthrough in the application of deep learning to graphs.

Other factors, like the production of capable GPUs that can handle huge computational tasks, have contributed to the research and development of GNNs.

Out of the different types of GNNs, graph convolutional networks are the most popular ones. We'll start by first understanding convolutions.

Convolutions

The image processing and classification tasks are dominated by convolutional neural networks (CNNs). CNN models use a filter or a kernel to perform convolutions. An image (represented as a grid of different numbers) is made to pass through a series of convolution filters, pooling fully connected layers, and finally, it passes through a softmax function to perform the classification task.

During convolution, we extract features from the input image.

The illustration above shows how image convolution is done. The example image of size 5 x 5 is considered. Convolution is applied with a 3 x 3 filter.

To compute the first value of the output matrix, place the filter (shown in red) on the input matrix (shown in green), perform element-wise multiplication, and sum the values.

...