Gain insights into self-supervised learning. Delve into pseudo label generation, similarity maximization, redundancy reduction, and masked image modeling to apply and modify these algorithms on unlabelled datasets.

dockerfile.tar.gz

This course covers self-supervised algorithms, which are useful for large pools of unlabelled data or when obtaining a high-quality labeled dataset is difficult. These algorithms leverage the supervisory signals from the structure of the unlabeled data to predict any unobserved or hidden property of the input.

You’ll start with the fundamentals of self-supervised learning and then implement your first class of algorithms. You’ll learn to generate pseudo labels and use these labels for training models using supervised learning. Next, you’ll learn about similarity maximization-based self-supervised algorithms. You’ll also look into redundancy reduction, which reduces the redundancy in the feature representations while maximizing the similarity between similar images. Lastly, you’ll learn to implement masked image modeling. 

After learning all this, you'll be able to apply the self-supervised models to unlabelled datasets. Furthermore, you’ll be able to implement and modify existing self-supervised algorithms.

Mastering Self-Supervised Algorithms for Learning without Labels

Learn how to implement BYOL,  a widely used distillation algorithm.


Introduction to Self-Supervised Learning

Pretext Tasks

Similarity Maximization and Redundancy Reduction

Masked Image Modeling

Appendix

BYOL Training

Student and teacher architectures