Appendix A: Ideal Loss Values

When training a GAN, the ideal state we want to reach is a balance between the generator and the discriminator. When this happens, the discriminator is no longer able to separate real data from generated data. This is because the generator has learned to create data that looks like it could have come from the real dataset.

Let’s work out what the discriminator loss should be when this balance is reached. We’ll do this for both the mean squared error loss and the binary cross-entropy loss.