Challenge: Preprocess and Clean a Czech-English Dataset

Show off what you've learned so far by preprocessing and cleaning data.

We'll cover the following

Task 1
- Steps
- Code playground
Task 2
- Steps
- Code playground

Task 1

Your first task in this challenge is to preprocess a sample Czech-English dataset. The files to preprocess are:

europar1-v7.cs-en.en
europar1-v7.cs-en.cs

Steps

While performing preprocessing, make sure to implement these steps:

Find the maximum and minimum sentence lengths.
Create a translation table for removing punctuation.
Remove punctuation marks.
Tokenize on white spaces.
Convert to lowercase text.
Remove words with numbers inside of them.

Code playground

Implement your solution in the Jupyter notebook below. Feel free to edit the helper function names already present in the notebook. The code for loading the dataset is already provided to you. Happy building!

Get hands-on with 1400+ tech skills courses.

Course Introduction

Basics of Transformers

Architecture of the Transformer Model

Fine-Tuning BERT Models

Pretraining a RoBERTa Model from Scratch

Downstream NLP Tasks with Transformers

Machine Translation with the Transformer

The Rise of Transformers with GPT-3 Engines

Applying Transformers for AI Text Summarization

Matching Tokenizers and Datasets

Transformers for NLP—Exam 1

Semantic Role Labeling with BERT-Based Transformers

Your Data Speaks: Story, Questions, and Answers

Detecting Customer Emotions to Make Predictions

Analyzing Fake News with Transformers

Interpreting Black Box Transformer Models

From NLP to Task-Agnostic Transfomer Models

The Emergence of Transformer-Driven Copilots

Transformers for NLP—Exam 2

Conclusion

Appendix

Challenge: Preprocess and Clean a Czech-English Dataset

Task 1

Steps

Code playground