Search⌘ K
Home
Courses
Natural Language Processing with Machine Learning

Combined State

Explore the process of combining final forward and backward states from each BiLSTM layer into LSTMStateTuple objects for Seq2Seq models. Understand how this combined state is created and passed to the decoder, enabling effective representation of encoder states for NLP tasks like machine translation and semantic analysis.

We'll cover the following...

Chapter Goals:

  • Combine the final states for each BiLSTM layer

A. LSTMStateTuple initialization

We initialize an LSTMStateTuple object with a hidden state (c) and state output (h).

Below we show an example of initializing an LSTMStateTuple object using the final forward and backward states from a single layer BiLSTM encoder.

Python 3.5
import tensorflow as tf
# Forward state of single-layer BiLSTM final states
fw_c = forward_final_state[0]
fw_h = forward_final_state[1]
# Backward state of single-layer BiLSTM final states
bw_c = backward_final_state[0]
bw_h = backward_final_state[1]
# Concatenate along final axis
final_c = tf.concat([fw_c, bw_c], -1 )
final_h = tf.concat([fw_h, bw_h], -1)
combined_state = tf.compat.v1.nn.rnn_cell.LSTMStateTuple(
    final_c, final_h)
print(combined_state)

In the above example, we combined the BiLSTM forward and backward states into a single LSTMStateTuple object, which can be passed into ...