Decoder: Feedforward and Add and Norm Component
Learn about the feedforward component and add and norm component of the decoder, as well as the linear and softmax layers of the transformer.
We'll cover the following...
Feedforward network
The next sublayer in the decoder is the feedforward network, as shown in the following figure:
Press + to interact
A decoder block
The feedforward layer in the decoder works exactly the same as what we learned in the encoder. Next, we will look into the add and norm component.
Add and norm component
Just like we learned with the encoder, the add and norm component connects the input and output of a sublayer, as shown in the following figure:
Press + to interact
A decoder block with an add and norm component
Next, we will look ...