Masked Siamese Networks: Masking Strategy and Encoder

Masked Siamese Networks (MSNs) is a self-supervised framework that leverages similarity maximization and masked image modeling concepts. As shown in the figure below, MSNs generate two augmented views of an image, where one is masked and the other is unchanged. The objective of MSNs is to learn similar network representations (a vision transformer) for both views.

MSNs don’t explicitly reconstruct the image pixels from the masked input. Instead, they incorporate this mask-denoising step in their feature representations itself by making representations of masked and unmasked image views similar.