ALBERT : Embeddings Extraction

With Hugging Face's transformers, we can use the ALBERT model just like how we used BERT. Let's explore this with a small example.

Suppose we need to get the contextual word embedding of every word in the sentence: 'Paris is a beautiful city'. Let's see how to do that with ALBERT.

Import the necessary modules

Let's first import the necessary modules:

Loading the model and tokenizer

Now, we download and load the pre-trained ALBERT model and tokenizer. We'll use the ALBERT-base model: