Gain insights into processing text data, creating word embeddings, and using LSTMs for semantic analysis and machine translation. Explore industry-relevant NLP techniques with Python and TensorFlow.

nlp_docker.tar.gz

mnist-fashion

In this course you'll learn techniques for processing text data, creating word embeddings, and using long short-term memory networks (LSTM) for tasks such as semantic analysis and machine translation. After completing this course, you will be able to solve the important day-to-day NLP problems faced in industry, which is incredibly useful given the prevalence of text data.

The code for this course is built around the TensorFlow framework, one of the premier frameworks for industry machine learning, and the Python pandas library for data analysis. Knowledge of Python and TensorFlow are prerequisites. 

This course was created by AdaptiLab, a company specializing in evaluating, sourcing, and upskilling enterprise machine learning talent. It is built in collaboration with industry machine learning experts from Google, Microsoft, Amazon, and Apple.

Natural Language Processing with Machine Learning

<h2 class="section-title">Chapter Goals:</h2>
<ul>
	<li>Convert your LSTM model's outputs into logits</li>
	<li>Use a padding mask to calculate the overall loss</li>
</ul>


<h2>A. Logits & loss</h2>

<div id="section1_l7" class="collapse show">
<p>As mentioned in earlier chapters, the task for a language model is no different from regular multiclass classification. Therefore, the loss function will still be the regular softmax cross entropy loss. We use a final fully-connected layer to convert model outputs into logits for each of the possible classes (i.e. vocabulary words).</p>


import tensorflow as tf
# Output from an LSTM
# Shape: (batch_size, time_steps, cell_size)
lstm_outputs = tf.compat.v1.placeholder(tf.float32, shape=(None, 10, 7))
vocab_size = 100
#print(lstm_outputs)
logits = tf.keras.layers.Dense(units=vocab_size)(lstm_outputs)
# Target tokenized sequences
# Shape: (batch_size, time_steps)
target_sequences = tf.compat.v1.placeholder(tf.int64, shape=(None, 10))
loss = tf.nn.sparse_softmax_cross_entropy_with_logits(
    labels=target_sequences,
    logits=logits)


<p>The function used to calculate the softmax cross entropy loss for feed-forward neural networks is <code>tf.nn.softmax_cross_entropy_with_logits</code>. However, we can only use this function if the <code>labels</code> and <code>logits</code> arguments both have the same shape.</p>
<p>In our example, <code>logits</code> has 3 dimensions while <code>labels</code> (<code>target_sequences</code>) only has 2. In this case, the <code>labels</code> are referred to as <i>sparse</i> (i.e. they represent class indexes rather than one-hot vectors), so we use the sparse version of the loss function.</p>
</div>


<h2>Chapter Goals:</h2>
<ul>
	<li>Convert your LSTM model's outputs into logits</li>
	<li>Use a padding mask to calculate the overall loss</li>
</ul>


<h2>A. Logits & loss</h2>

<div id="section1_l7">
<p>As mentioned in earlier chapters, the task for a language model is no different from regular multiclass classification. Therefore, the loss function will still be the regular softmax cross entropy loss. We use a final fully-connected layer to convert model outputs into logits for each of the possible classes (i.e. vocabulary words).</p>


What you'll learn from this course

Word Embeddings

Language Model

Text Classification

Seq2Seq Model

Calculating Loss

Chapter Goals:

A. Logits & loss