Fundamentals of Machine Learning for Software Engineers/

...

/

Run on Convolutions

Run on Convolutions

Learn and test CNN used for CIFAR-10.

We'll cover the following...

CNN for CIFAR-10

A convolutional neural network for CIFAR-10 is:

In the code below, our Cifar-10 classifier undergoes training for 3 epochs, but to obtain better accuracy we must increase the number of epochs. The results of training the classifier for 20 epochs are available below this code widget.

A convolutional neural network for CIFAR-10 is given below:

# A convolutional neural network that trains on CIFAR-10 images.

import numpy as np
from keras.models import Sequential
from keras.layers import Conv2D, Dropout, Dense
from keras.layers import BatchNormalization, Flatten
from tensorflow.keras.optimizers import Adam
from tensorflow.keras.utils import to_categorical
from keras.datasets import cifar10

# Preparing dataset
(X_train_raw, Y_train_raw), (X_test_raw, Y_test_raw) = cifar10.load_data()

# Using a portion of the larger dataset 
X_train_raw = X_train_raw[0:10000]
Y_train_raw = Y_train_raw[0:10000]
X_test_raw = X_test_raw[0:1000]
Y_test_raw = Y_test_raw[0:1000]

X_train = X_train_raw / 255
X_test_all = X_test_raw / 255
X_validation, X_test = np.split(X_test_all, 2)
Y_train = to_categorical(Y_train_raw)
Y_validation, Y_test = np.split(to_categorical(Y_test_raw), 2)

# Building the CNN
model = Sequential()

model.add(Conv2D(16, (3, 3), activation='relu'))
model.add(BatchNormalization())
model.add(Dropout(0.5))

model.add(Conv2D(32, (3, 3), activation='relu'))
model.add(BatchNormalization())
model.add(Dropout(0.5))

model.add(Flatten())

model.add(Dense(1000, activation='relu'))
model.add(BatchNormalization())
model.add(Dropout(0.5))

model.add(Dense(512, activation='relu'))
model.add(BatchNormalization())
model.add(Dropout(0.5))

model.add(Dense(10, activation='softmax'))


# Compiling the model
model.compile(loss='categorical_crossentropy',
              optimizer=Adam(),
              metrics=['accuracy'])

# Fitting the model to test its performance(training)
history = model.fit(X_train, Y_train,
                    validation_data=(X_validation, Y_validation),
                    epochs=3, batch_size=32)
CNN for CIFAR-10

The code above involves many arbitrary decisions. For example, we decide to have two convolutional layers to use ReLUs in every layer. We also use techniques from the previous chapter for batch normalization to improve accuracy, and dropout to reduce overfitting. We didn’t check whether those decisions result in a better neural network than the alternatives. That exploration will be done in the final exercise of this chapter.

However, some of the choices are part of the default design of a CNN and are proven useful by many deep learning practitioners. One is the idea of building a CNN as a sequence of convolutional layers, followed by a handful of fully connected layers. Nothing prevents us from using convolutional layers throughout, but it’s customary to have those fully connected layers at the end.

Let’s traverse the network from start to end, and look at each layer in detail.

First, the two convolutional layers:

model.add(Conv2D(16, (3, 3), activation='relu'))
model.add(BatchNo
...