M-BERT Generalization

Learn how the M-BERT model generalizes across different scripts and typological features.

We'll cover the following

Generalization across scripts
Generalization across typological features
- Accuracy of the POS tagging task
Effect of language similarity

Generalization across scripts

Let's investigate whether the M-BERT model can generalize across languages that are written in different scripts. Let's conduct a small experiment to understand this. Say we are performing a POS (part of speech) tagging task. First, we fine-tune M-BERT for the POS Part-of-speechtagging task in the Urdu language. Then, we evaluate the fine-tuned M-BERT model in a different language, say Hindi. Note that Urdu follows Arabic script, while Hindi follows Devanagari script. A simple example of Urdu and Hindi text is given here:

Get hands-on with 1400+ tech skills courses.

Before We Start

Starting Off with BERT

A Primer on Transformers

Understanding the BERT Model

Getting Hands-On with BERT

Exploring BERT Variants

Different BERT Variants

BERT Variants—Based on Knowledge Distillation

Applications of BERT

Exploring BERTSUM for Text Summarization

Applying BERT to Other Languages

Exploring Sentence and Domain-Specific BERT

Working with VideoBERT, BART, and More

Conclusion

M-BERT Generalization

Generalization across scripts