Decoder: Feedforward and Add and Norm Component
Explore the decoder's crucial sublayers in transformer architecture, including the feedforward network and add and norm components. Understand how the decoder processes outputs using linear and softmax layers to predict the next word in a sequence, enhancing your grasp of transformer mechanisms.
We'll cover the following...
We'll cover the following...
Feedforward network
The next sublayer in the decoder is the feedforward network, as shown in the following figure:
A decoder block
The feedforward layer in the decoder works exactly the same as what we learned in the encoder. Next, we will look into the add and norm component.
Add and norm component
Just like we learned with the encoder, the add and norm component connects the input and output of a sublayer, as shown in the following figure:
A decoder block with an add and norm component
Next, we will look ...