Types of Neural Networks: Part I

In this section, we’ll see the most popular kinds of neural networks used in DL today. Some of these will sound familiar based on the previous section, but it will help to familiarize ourselves with some of these concepts, especially if we plan on working as a product manager for a DL product. Even if we aren’t currently working in this capacity, we’ll want to take a look through these in case our career does veer toward DL products in the future. The following is a list of some of the most used ANNs in DL:

• Multilayer perceptrons (MLPs)

• Radial basis function networks (RBFNs)

• Self-organizing maps (SOMs)

• Convolutional neural networks (CNNs)

• Recurrent neural networks (RNNs)

• Long short-term memory networks (LSTMs)

• Generative adversarial networks (GANs)

• Deep belief networks (DBNs)

In the following section, we will touch on these various neural networks to have an idea of what they are best suited for. We’ll describe some of the most popular use cases of each type of ANN so that we can understand, at least in a general sense, what some of the core competencies of each ANN are so that we can start to keep those ideas in mind should we pursue the creation of our own DL products in the future. If our aim is to specialize exclusively in supporting or building DL products of our own, this will be a great summary overview of each ANN.

Multilayer perceptrons (MLPs)

After David Rumelhart, Geoffrey Hinton, and Ronald Williams’s paper Learning representations by back-propagating errors came out in 1986, MLPs were popularized because, in that paper, they used backpropagation to train an MLP. Unlike RNNs, MLPs are another form of feedforward neural network that uses backpropagation to optimize the weights.

For this reason, we can think of MLPs as some of the most basic forms of ANNs because they were among the first to appear, and today, they’re still used often to deal with the high computing power that’s needed by some of the newer ANNs out there. Their accessibility and reliability are still useful today, so we wanted to start this list with MLPs to give us a good foundation for conceptualizing the rest of the DL algorithms.