Artificial neural networks (ANNs) are the foundation of many modern machine learning techniques, mimicking the learning abilities of the human brain. One of the most basic types of ANNs is the feedforward artificial neural network (FANN).
In this Answer, we will delve into the world of FANNs, understand their structure, working, and see them in action with some Python code.
In a feedforward artificial neural network, node connections are organized without forming cycles, distinguishing it from recurrent neural networks that incorporate cycles among the nodes. The term 'feedforward' comes from how information travels through the network - it moves forward, from the input layer, through the hidden layers, to the output layer, without looping back.
A feedforward artificial neural network comprises multiple layers of nodes or 'neurons.' These layers include:
Input layer: The network's starting point. Each node in this layer corresponds to one feature in the dataset. The number of nodes equals the number of features.
Note: To learn more about input layers in Keras , refer to this Answer.
Hidden layer(s): Hidden layers, situated between the input and output layers, constitute an essential part of a feedforward artificial neural network (FANN). The network can include one or multiple hidden layers, and each neuron within these layers employs a nonlinear activation function.
Output layer: The network's endpoint. The output layer transforms the values from the last hidden layer into a format suitable for the task at hand.
Let's visualize this with a simple diagram:
The mathematical operations in a feedforward ANN involve two steps: the weighted sum of the inputs and the activation function.
The weighted sum of inputs: Each input is multiplied by a weight, and the results are added together with a bias.
Activation function: After calculating the weighted sum, the output is fed into an activation function, which serves the crucial role of introducing non-linearity to the neuron's output. Among various activation functions, the sigmoid function is one of the most frequently utilized choices.
Here's a basic illustration of how a feedforward neural network works:
Here is a simple Python code to implement a feedforward artificial neural network (FANN) using the keras
and scikit-learn (sklearn
) library:
Note: To learn about scikit-learn library in more detail, refer to this Answer.
# Import necessary librariesfrom keras.models import Sequentialfrom keras.layers import Densefrom sklearn.datasets import make_classificationfrom sklearn.model_selection import train_test_split# Generate a datasetdata_features, data_labels = make_classification(n_samples=200, n_features=25, n_informative=20, n_redundant=5, random_state=42)# Split the dataset into training set and test settrain_features, test_features, train_labels, test_labels = train_test_split(data_features, data_labels, stratify=data_labels, random_state=42)# Define the modelneural_network = Sequential()neural_network.add(Dense(50, input_dim=25, activation='relu'))neural_network.add(Dense(50, activation='relu'))neural_network.add(Dense(50, activation='relu'))neural_network.add(Dense(1, activation='sigmoid'))# Compile the modelneural_network.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])# Train the modelneural_network.fit(train_features, train_labels, epochs=200, batch_size=20)# Evaluate the model_, model_accuracy = neural_network.evaluate(test_features, test_labels)print('Model Accuracy: %.2f' % (model_accuracy*100))
Lines 1–5: Import necessary libraries. Sequential
and Dense
are imported from keras
for creating the neural network model. make_classification
is imported from sklearn.datasets
for generating a random n-class classification problem. train_test_split
is imported from sklearn.model_selection
for splitting arrays or matrices into random train and test subsets.
Line 7: Generate a dataset with 200
samples, 25
features, 20
informative features, and 5
redundant features. The generated dataset is stored in data_features
and data_labels
.
Line 11: Split the dataset into a training set and a test set. 75% of the data is used for training and 25% of the data is used for testing.
Line 14: Initialize a Sequential
model. This is a linear stack of layers that you can add to sequentially.
Lines 15–18: Add layers to the model. The Dense
function is used to create fully connected layers.
Note: To learn more about Keras dense layers, refer to this Answer.
The first layer has 50
neurons and expects 25
input features. The activation function used is ‘relu’
. The second and third layers also have 50
neurons each. The final layer has 1
neuron (for binary classification) and uses the ‘sigmoid’
activation function.
Line 21: Compile the model. The loss function used is ‘binary_crossentropy’
, which is suitable for binary classification problems. The optimizer used is ‘adam’
, a popular choice for many machine learning models. The model will also measure ‘accuracy’
during training and testing.
Line 24: Train the model using the training data. The model will run through the data 200
times (epochs
), and update the model weights after every 20
samples (batch_size
).
Line 27: Evaluate the model using the test data. The model’s accuracy is printed out. The _
is a common convention for a variable that we won’t use (the evaluate function returns the loss value and the metrics value, but we only care about the latter).
Feedforward ANNs have a wide range of applications in real-world problems:
Image recognition: They are used in computer vision tasks for object detection, face recognition, and more.
Natural language processing: They are used for language translation, sentiment analysis, and other language-related tasks.
Medical diagnosis: They predict diseases based on symptoms and medical history.
Financial forecasting: They predict stock prices and other financial indicators.
Feedforward ANNs are fundamental in the field of artificial intelligence. They are the stepping stone to understanding more complex networks. The understanding of their structure, the mathematical formulas involved, and their visual representation are crucial in the journey of learning artificial intelligence and machine learning.
A type of neural network where the information travels in only one direction.
Keras
The activation function used in the final layer of the neural network model in the provided code.
Feedforward artificial neural network
A popular Python library used for creating neural network models.
Sigmoid
Here are some more guides on neural networks: