This course covers Google BERT including its architecture, pre-training and variants like M-BERT, VideoBERT, ALBERT, RoBERTa, ELECTRA, and SpanBERT.
Intermediate
120 Lessons
25h
Certificate of Completion
This course covers Google BERT including its architecture, pre-training and variants like M-BERT, VideoBERT, ALBERT, RoBERTa, ELECTRA, and SpanBERT.
AI-POWERED
AI-POWERED
This course includes
This course includes
Course Overview
The transformer, a cutting-edge deep learning architecture, plays a pivotal role in Natural Language Processing (NLP) tasks. Leading NLP models like BERT, GPT, and T5 are based on the transformer. BERT, in particular, has revolutionized NLP by delivering state-of-the-art results. This course is your entry into Google's BERT architecture. It begins with the basics of transformers and covers encoder and decoder mechanisms. You will learn to pre-train and use BERT. Exploring various BERT variants and related ...Show More
TAKEAWAY SKILLS
Transformer Models
Machine Learning
What You'll Learn
An in-depth knowledge of transformer models, including the workings of encoders and decoders
A firm grasp of BERT’s core concepts, pre-training methods, and subword tokenization algorithms
The ability to effectively use pre-trained BERT models, from extracting contextual embeddings to fine-tuning for various tasks
A thorough understanding of various BERT variants, such as ALBERT, RoBERTa, ELECTRA, and SpanBERT
An understanding of BERT’s applications in multilingual contexts and domain-specific variants like ClinicalBERT and BioBERT
What You'll Learn
An in-depth knowledge of transformer models, including the workings of encoders and decoders
Show more
Course Content
Before We Start
Starting Off with BERT
A Primer on Transformers
Understanding the BERT Model
Getting Hands-On with BERT
Exploring BERT Variants
1 Lesson
Different BERT Variants
12 Lessons
BERT Variants—Based on Knowledge Distillation
14 Lessons
Applications of BERT
1 Lesson
Exploring BERTSUM for Text Summarization
8 Lessons
Applying BERT to Other Languages
18 Lessons
Exploring Sentence and Domain-Specific BERT
10 Lessons
Working with VideoBERT, BART, and More
10 Lessons
Conclusion
1 Lesson
Course Author
Trusted by 1.4 million developers working at companies
Anthony Walker
@_webarchitect_
Evan Dunbar
ML Engineer
Carlos Matias La Borde
Software Developer
Souvik Kundu
Front-end Developer
Vinay Krishnaiah
Software Developer
Eric Downs
Musician/Entrepeneur
Kenan Eyvazov
DevOps Engineer
Souvik Kundu
Front-end Developer
Eric Downs
Musician/Entrepeneur
Anthony Walker
@_webarchitect_
Evan Dunbar
ML Engineer
See how Educative uses AI to make your learning more immersive than ever before.