The WordPiece Tokenizer

Learn about the WordPiece tokenizer and how it works.

BERT uses a special type of tokenizer called a WordPiece tokenizer. The WordPiece tokenizer follows the subword tokenization scheme. Let's understand how the WordPiece tokenizer works with the help of an example. Consider the following sentence:

Get hands-on with 1400+ tech skills courses.