LSTM Layer Operations and Network Architectures

The cell input-output and the operations here were explained. In this section, the view is zoomed out of the cell, and the network operations at the layer level are explained.

The illustrations below bring an LSTM layer’s internal and external connections into perspective. It provides visibility of the layer’s input-output mechanism. The illustrations provide an understanding of stacking layers around an LSTM layer and the way they interact.

LSTM network input and hidden layers. The input is a batch of time-window of observations. This makes each sample in a batch a 2D array and the input batch a 3D array. The time-window is arbitrarily taken as three for illustration. The cells in blue boxes within the hidden LSTM layer are unwrapped to their time-step iterations shown with green boxes. The connected arcs show the transmission of time-indexed information between the layers. The first LSTM layer is emitting sequences (LSTM(...,return_sequences=True)). These sequences have the same notional time order as the input and are processed in the same order by the second LSTM layer. If the model is stateful, the cell state from the prior batch processing is preserved and accessed by the next batch.

The expanded view is split into two parts for clarity. The network layers are expanded in the illustrations above and below.

The illustration above shows the left part of the network—from input to the ...