Creating a DBN with the Keras Model API
Learn how to implement deep belief networks (DBN) using TensorFlow 2.0.
We'll cover the following...
The creation of a single-layer RBM to generate images is the building block required to create a full-fledged DBN. Usually, for a model in TensorFlow 2, we only need to extend tf.keras.Model
and define an initialization (where the layers are defined) and a call function (for the forward pass). For our DBN model, we also need a few more custom functions to define its behavior.
Implementing the RBM
Let’s continue by combining multiple RBMs in layers to create a more powerful model—the deep belief network (DBN).
RBM initialization
First, in the initialization, we need to pass a list of dictionaries that contain the parameters for our RBM layers (number_hidden_units
, number_visible_units
, and learning_rate, cd_steps
):
class DBN(tf.keras.Model):def __init__(self, rbm_params=None, name='deep_belief_network', num_epochs=100,tolerance=1e-3, batch_size=32, shuffle_buffer=1024, **kwargs):super().__init__(name=name, **kwargs)self._rbm_params = rbm_paramsself._rbm_layers = list()self._dense_layers = list()for num, rbm_param in enumerate(rbm_params):self._rbm_layers.append(RBM(**rbm_param))self._rbm_layers[-1].build([rbm_param["number_visible_units"]])if num < len(rbm_params)-1:self._dense_layers.append(tf.keras.layers.Dense(rbm_param["number_hidden_units"],activation=tf.nn.sigmoid))else:self._dense_layers.append(tf.keras.layers.Dense(rbm_param["number_hidden_units"],activation=tf.nn.softmax))self._dense_layers[-1].build([rbm_param["number_visible_units"]])self._num_epochs = num_epochsself._tolerance = toleranceself._batch_size = batch_sizeself._shuffle_buffer = shuffle_buffer
At the same time, we also initialize a set of sigmoidal dense layers with a softmax at the end, which we can use for fine-tuning through backpropagation once we’ve trained the model using the generative procedures outlined earlier.
To train the DBN, we begin a new code block to start the generative learning process for the stack of RBMs:
# pretraining:inputs_layers = []for num in range(len(self._rbm_layers)):if num == 0:inputs_layers.append(inputs)self._rbm_layers[num] = self.train_rbm(self._rbm_layers[num], inputs)else:# Pass all data through the previous layerinputs_layers.append(inputs_layers[num-1].map(self._rbm_layers[num-1].forward))self._rbm_layers[num] = self.train_rbm(self._rbm_layers[num], inputs_layers[num])
It initializes an empty list inputs_layers
...