Gain insights into processing text data, creating word embeddings, and using LSTMs for semantic analysis and machine translation. Explore industry-relevant NLP techniques with Python and TensorFlow.

nlp_docker.tar.gz

mnist-fashion

In this course you'll learn techniques for processing text data, creating word embeddings, and using long short-term memory networks (LSTM) for tasks such as semantic analysis and machine translation. After completing this course, you will be able to solve the important day-to-day NLP problems faced in industry, which is incredibly useful given the prevalence of text data.

The code for this course is built around the TensorFlow framework, one of the premier frameworks for industry machine learning, and the Python pandas library for data analysis. Knowledge of Python and TensorFlow are prerequisites. 

This course was created by AdaptiLab, a company specializing in evaluating, sourcing, and upskilling enterprise machine learning talent. It is built in collaboration with industry machine learning experts from Google, Microsoft, Amazon, and Apple.

Natural Language Processing with Machine Learning

<h2 class="section-title">Chapter Goals:</h2>
<ul>
	<li>Learn about candidate sampling and why it is useful for embedding training</li>
</ul>


<h2>A. Large vocabularies</h2>

<div id="section1_l5" class="collapse show">
<p>To obtain good word embeddings, it is usually necessary to train an embedding model on a large amount of text data. This means that the vocabulary size will likely be very large, often reaching tens of thousands of words. However, having a large vocabulary size can significantly slow down training.</p>
<p>Training an embedding model is equivalent to multiclass classification, where the possible classes include every single vocabulary word. This means that we would need to calculate a softmax loss across every single vocabulary word during training, which can be incredibly time consuming for large vocabularies.</p>
<p>In order to mitigate the costly full softmax operation, we apply something called <i>candidate sampling</i>. With candidate sampling, we choose a much smaller fraction of the possible classes (i.e. vocabulary words) for computing the loss. This speeds up training significantly while also maintaining performance if we use the proper candidate samplers and loss function (more on this in the next chapter).</p>
</div>


<h2>Chapter Goals:</h2>
<ul>
	<li>Learn about candidate sampling and why it is useful for embedding training</li>
</ul>


<h2>A. Large vocabularies</h2>

<div id="section1_l5">
<p>To obtain good word embeddings, it is usually necessary to train an embedding model on a large amount of text data. This means that the vocabulary size will likely be very large, often reaching tens of thousands of words. However, having a large vocabulary size can significantly slow down training.</p>
<p>Training an embedding model is equivalent to multiclass classification, where the possible classes include every single vocabulary word. This means that we would need to calculate a softmax loss across every single vocabulary word during training, which can be incredibly time consuming for large vocabularies.</p>
<p>In order to mitigate the costly full softmax operation, we apply something called <i>candidate sampling</i>. With candidate sampling, we choose a much smaller fraction of the possible classes (i.e. vocabulary words) for computing the loss. This speeds up training significantly while also maintaining performance if we use the proper candidate samplers and loss function (more on this in the next chapter).</p>
</div>


Understand why candidate sampling is used for embedding training.

What you'll learn from this course

Word Embeddings

Language Model

Text Classification

Seq2Seq Model

Candidate Sampling