Improving LSTMs: Generating Text with Words Instead of N-grams

Here, we’ll discuss ways to improve LSTMs. We have so far used bigrams as our basic unit of text. But we would get better results by incorporating words instead of bigrams. This is because using words reduces the overhead of the model by alleviating the need to learn to form words from bigrams. We’ll discuss how we can employ word vectors in the code to generate better-quality text compared to using bigrams.

The curse of dimensionality

One major limitation stopping us from using words instead of n-grams as the input to our LSTM is that this will drastically increase the number of parameters in our model. Let’s try to understand this through an ...