Gain insights into processing text data, creating word embeddings, and using LSTMs for semantic analysis and machine translation. Explore industry-relevant NLP techniques with Python and TensorFlow.

nlp_docker.tar.gz

mnist-fashion

In this course you'll learn techniques for processing text data, creating word embeddings, and using long short-term memory networks (LSTM) for tasks such as semantic analysis and machine translation. After completing this course, you will be able to solve the important day-to-day NLP problems faced in industry, which is incredibly useful given the prevalence of text data.

The code for this course is built around the TensorFlow framework, one of the premier frameworks for industry machine learning, and the Python pandas library for data analysis. Knowledge of Python and TensorFlow are prerequisites. 

This course was created by AdaptiLab, a company specializing in evaluating, sourcing, and upskilling enterprise machine learning talent. It is built in collaboration with industry machine learning experts from Google, Microsoft, Amazon, and Apple.

Natural Language Processing with Machine Learning

<h2 class="section-title">Chapter Goals:</h2>
<ul>
	<li>Learn about attention and understand why it's useful</li>
	<li>Incorporate attention into the decoder LSTM</li>
</ul>


<h2>A. Using the encoder</h2>

<div id="section1_l5" class="collapse show">
<p>Based on the encoder-decoder model architecture, the only thing that the decoder gets from the encoder is the final state in each layer. The final states basically encapsulate the encoder's extracted information from the input sequence, which is passed into the decoder.</p>
<p>However, trying to encapsulate all the useful information from an input sequence into a final state is a difficult task, especially if the input sequence is large and contains long-term dependencies. This is a problem that has been shown to exist in practice, where decoders perform poorly on input sequences with long-term dependencies.</p>

<p>The obvious solution to this issue is to give the decoder access to each of the encoder's intermediate time step outputs. In the previous chapter's diagram, the encoder's outputs were not used. However, if we were to use the encoder's outputs as additional input for the decoder, it would give the decoder a lot more useful information about the input sequence. The way we do this is by using <i>attention</i>.</p>
</div>


<h2>B. How attention works</h2>

<div id="section2_l5" class="collapse show">
<p>Although we want to use the input sequence for the decoder, we don't necessarily want to use each input token equally. With attention, we let the decoder decide which encoder outputs are most useful for the decoder at the current decoding time step.</p>
<p>Using the decoder's hidden state at the current time step, as well as the encoder outputs, attention will calculate something called a <i>context vector</i>. The context vector encapsulates the most meaningful information from the input sequence for the current decoder time step, and it's used as additional input for the decoder when calculating the time step's output.</p>
</div>


<h2>Chapter Goals:</h2>
<ul>
	<li>Learn about attention and understand why it's useful</li>
	<li>Incorporate attention into the decoder LSTM</li>
</ul>


<h2>A. Using the encoder</h2>

<div id="section1_l5">
<p>Based on the encoder-decoder model architecture, the only thing that the decoder gets from the encoder is the final state in each layer. The final states basically encapsulate the encoder's extracted information from the input sequence, which is passed into the decoder.</p>
<p>However, trying to encapsulate all the useful information from an input sequence into a final state is a difficult task, especially if the input sequence is large and contains long-term dependencies. This is a problem that has been shown to exist in practice, where decoders perform poorly on input sequences with long-term dependencies.</p>

<h2>B. How attention works</h2>

<div id="section2_l5">
<p>Although we want to use the input sequence for the decoder, we don't necessarily want to use each input token equally. With attention, we let the decoder decide which encoder outputs are most useful for the decoder at the current decoding time step.</p>
<p>Using the decoder's hidden state at the current time step, as well as the encoder outputs, attention will calculate something called a <i>context vector</i>. The context vector encapsulates the most meaningful information from the input sequence for the current decoder time step, and it's used as additional input for the decoder when calculating the time step's output.</p>
</div>


Learn about the attention mechanism and why it's important.

What you'll learn from this course

Word Embeddings

Language Model

Text Classification

Seq2Seq Model

Attention

Chapter Goals:

A. Using the encoder