Introduction: Interpreting Black Box Transformer Models

Get an overview of what we will cover in this chapter.

We'll cover the following

Chapter Overview
- What is LIT?
- What is LIME?
Topics to be covered

Million-to billion-parameter transformer models seem like huge opaque boxes that nobody can interpret. As a result, many developers and users have sometimes been discouraged when dealing with these mind-blowing models. However, recent research has begun to solve the problem with innovative, cutting-edge tools.

Chapter Overview

It is beyond the scope of this course to describe all of the explainable AI methods and algorithms. So instead, this chapter will focus on ready-to-use visual interfaces that provide insights for transformer model developers and users.

The chapter begins by importing and running BertViz by Jesse Vig. Jesse did quite an excellent job of building a visual interface that shows the activity in the attention heads of a BERT transformer model. BertViz interacts with the BERT models and provides a well-designed interactive interface.

Get hands-on with 1200+ tech skills courses.

Course Introduction

Basics of Transformers

Architecture of the Transformer Model

Fine-Tuning BERT Models

Pretraining a RoBERTa Model from Scratch

Downstream NLP Tasks with Transformers

Machine Translation with the Transformer

The Rise of Transformers with GPT-3 Engines

Applying Transformers for AI Text Summarization

Matching Tokenizers and Datasets

Transformers for NLP—Exam 1

Semantic Role Labeling with BERT-Based Transformers

Your Data Speaks: Story, Questions, and Answers

Detecting Customer Emotions to Make Predictions

Analyzing Fake News with Transformers

Interpreting Black Box Transformer Models

From NLP to Task-Agnostic Transfomer Models

The Emergence of Transformer-Driven Copilots

Transformers for NLP—Exam 2

Conclusion

Appendix

Introduction: Interpreting Black Box Transformer Models

Chapter Overview