Gain insights into processing text data, creating word embeddings, and using LSTMs for semantic analysis and machine translation. Explore industry-relevant NLP techniques with Python and TensorFlow.

nlp_docker.tar.gz

mnist-fashion

In this course you'll learn techniques for processing text data, creating word embeddings, and using long short-term memory networks (LSTM) for tasks such as semantic analysis and machine translation. After completing this course, you will be able to solve the important day-to-day NLP problems faced in industry, which is incredibly useful given the prevalence of text data.

The code for this course is built around the TensorFlow framework, one of the premier frameworks for industry machine learning, and the Python pandas library for data analysis. Knowledge of Python and TensorFlow are prerequisites. 

This course was created by AdaptiLab, a company specializing in evaluating, sourcing, and upskilling enterprise machine learning talent. It is built in collaboration with industry machine learning experts from Google, Microsoft, Amazon, and Apple.

Natural Language Processing with Machine Learning

# Chapter Goals:

<ul>
	<li>Learn how to perform decoding for inference</li>
	<li>Learn about variable scopes for declaring and retrieving variables</li>
</ul>


<h2>A. Decoding without ground truth</h2>


<p>During training, we have access to ground truth tokens, which are used as input for the decoder. However, for inference (i.e. generating output in real-time), we only have access to an input sequence. In this case, the input for the decoder at each time is just the decoder's output token from the previous time step.</p>



<p>In TensorFlow, we can switch from training to inference mode just by changing the <code>TrainingSampler</code> object to an inference helper. The most commonly used inference helper object is <code><a href="https://www.tensorflow.org/api_docs/python/tf/contrib/seq2seq/GreedyEmbeddingHelper" target="_blank">GreedyEmbeddingSampler</a></code>.</p>

<h2>B. Greedy decoding</h2>


<p>The name of <code>GreedyEmbeddingSampler</code> comes from the fact that it utilizes a greedy decoding method and incorporates an embedding matrix.</p>

<p>The greedy decoding method means that the output at each decoder time step (also the input into the next time step) is just the highest probability vocabulary word at the time step. The output is converted into a word embedding prior to being used as input for the next time step.</p>

<p>Below we create a <code>GreedyEmbeddingSampler</code> with a vocabulary size of 598 as first argument and second argument hidden size which is half of the vocabulary size.</p>

Understand the difference between training and inference decoding.

What you'll learn from this course

Word Embeddings

Language Model

Text Classification

Seq2Seq Model

Inference Decoding

Chapter Goals:

A. Decoding without ground truth

B. Greedy decoding