Gain insights into processing text data, creating word embeddings, and using LSTMs for semantic analysis and machine translation. Explore industry-relevant NLP techniques with Python and TensorFlow.

nlp_docker.tar.gz

mnist-fashion

In this course you'll learn techniques for processing text data, creating word embeddings, and using long short-term memory networks (LSTM) for tasks such as semantic analysis and machine translation. After completing this course, you will be able to solve the important day-to-day NLP problems faced in industry, which is incredibly useful given the prevalence of text data.

The code for this course is built around the TensorFlow framework, one of the premier frameworks for industry machine learning, and the Python pandas library for data analysis. Knowledge of Python and TensorFlow are prerequisites. 

This course was created by AdaptiLab, a company specializing in evaluating, sourcing, and upskilling enterprise machine learning talent. It is built in collaboration with industry machine learning experts from Google, Microsoft, Amazon, and Apple.

Natural Language Processing with Machine Learning

# Chapter Goals

* Learn about the types of models used to create embeddings
* Understand the difference between the skip-gram and CBOW models


# A. Embedding models


In the previous chapter, we discussed how embedding vectors are based on target words and context windows. Specifically, we use these target words and context windows to create training pairs. Depending on the type of embedding model we use, the pairs will either be target-context pairs or context-target pairs.


## Skip-gram
The skip-gram model uses target-context training pairs. Each pair has a target word as the first element and a context word as the second element. This means that every context window will produce multiple training pairs. For example, if the context window is "paul likes singing in french", (where singing is the target word), the training pairs would be

$$(singing, paul), (singing, likes)$$

$$(singing, in), (singing, french)$$


When training an embedding model, we convert the first element in each pair (for the skip-gram model, this is the target word) into its corresponding embedding vector. This will be discussed in greater depth in later chapters.

## CBOW

The continuous-bag-of-words (CBOW) model uses context-target training pairs. Each pair will have all the context words as the first element and the target word as the second element. For example, if the context window is "tom eats **spicy** crab salad" (where **spicy** is the target word), the training pair would be


$$(tom, eats, crab, salad), spicy)$$


Since the context element for the CBOW model contains multiple words, we use the average context embedding vector when training our embedding model. So in the example above, the context embedding vector would be the average between the four embedding vectors for "tom", "eats", "crab", and "salad".

Understand the difference between skip-gram and CBOW embedding models.

What you'll learn from this course

Word Embeddings

Language Model

Text Classification

Seq2Seq Model

Skip-gram

Chapter Goals

A. Embedding models

Skip-gram