Summary: Recurrent Neural Networks
Understand the fundamentals of recurrent neural networks, including their design from feed-forward networks and training using backpropagation through time. Learn about challenges like vanishing and exploding gradients, different RNN architectures, practical NLP applications such as named entity recognition, and techniques for generating token embeddings using character embeddings and convolution.
In this chapter, we looked at RNNs, which are different from conventional feed-forward neural networks and more powerful in terms of solving temporal tasks. Specifically, we discussed how to arrive at an RNN from a feed-forward neural network-type structure. We assumed a sequence of inputs and outputs and designed a computational graph that can represent the sequence of inputs and outputs.
This computational graph resulted in a series of copies of functions that we applied to each individual input-output tuple in the sequence. Then, by generalizing this model to any given single time step