Training with an EMA (Exponential Moving Average)

The PyTorch Image Model framework supports an exponential moving average (EMA), which maintains moving averages of the trained variables by employing an exponential decay.

The implementation of an EMA is as follows:

1. Add shadow copies of