...
/Bidirectional LSTM for Enhanced Temporal Learning
Bidirectional LSTM for Enhanced Temporal Learning
Explore how bidirectional LSTM networks leverage past and future data insights for superior temporal pattern recognition.
We'll cover the following...
Bidirectional
A regular LSTM, or any RNN, learns the temporal patterns in a forward direction going from the past to the future. Meaning that at any time-step, the cell state learns only from the past. It doesn’t have visibility of the future. This concept is clearer in the illustration below. As shown in the illustration, the truck of memories (cell state) moves from left to right.
Schuster and Paliwal 1997 made a significant contribution by proposing a bidirectional RNN. A bidirectional RNN adds a mirror RNN layer to the original RNN layer. The input sequence is passed as is to the original layer and in reverse to the mirror.
This enables the cell states to learn from all the input information, both the past and the future. This is shown in the illustration below. Similar to a regular LSTM layer, the information from the past flows into the cell state from left to right in the top lane. Additionally, the information from the future flows back to the cell states right to left in the bottom lane.
This ability to have collective information from the past and the future makes a bidirectional LSTM layer more powerful. Graves and Schmidhuber also quote an important requirement for using a bidirectional LSTM as follows:
“…for temporal problems like speech recognition, relying on knowledge of the future seems at first sight to violate causality—at least if the task is online.”
“…However, human listeners do exactly ...