Long Short Term Memory Networks (LSTM)

What are LSTMs?

Long Short Term Memory Networks (LSTMs) in short, are modifications in the standard Recurrent Neural Network. Look at the image of a standard RNN architecture below.

In the above diagram:

• $X_{t-1}$ represents the input at time step t-1.
• $h_{t-1}$ represents the output of the RNN cell at time step t-1.

This is the architecture that has the vanishing gradient problem, which we discussed in our previous lessons. We’ve also maintained that, as a rule of thumb, if the recurring weight (denoted by $W_{rec}$ ...