Contrastive Learning: The SimCLR Algorithm

What is contrastive learning?

The objective of contrastive learning is to learn neural network embeddings such that embeddings from related images should be closer than embeddings from unrelated or dissimilar images. So, given an image $X_a$, we can call it an “anchor image” and define two terms:

• Positives: Images that are closely related to the anchor image, $X_a$. Let’s represent them by $X_p$

• Negatives: Images that are unrelated or dissimilar to $X_a$. Let’s call them $X_n$.

The contrastive learning objective thus learns a neural network $f(.)$ such that: