Wasserstein GAN
Martin Arjovsky, Soumith Chintala, Leon Bottou2017ICML 2017
Read it on arxiv.org(opens in a new tab)Why this one
Read this after Generative Adversarial Networks. The useful lesson is not that WGAN is the GAN you should use in production, since its first weight clipping version is famously crude. The lesson is that a loss function is also a geometry. If two generated distributions barely overlap the real one, the usual GAN objective can tell the generator almost nothing about how to move. Wasserstein distance still gives a direction, because it measures the cost of transporting mass instead of asking whether a discriminator can separate supports. People often remember this paper as a stability trick or a cure for mode collapse. Treat it instead as a debugging lens: when your metric is discontinuous with respect to the parameters you are training, better optimizers will not save you. It is worth your evening because it makes probability distances feel like engineering choices, not math decoration.
What to take away
- WGAN replaces discriminator classification with a critic that estimates a transport distance.
- The loss curve becomes more useful because it tracks sample quality better than vanilla GAN losses.
- The Lipschitz constraint is the price of the theory, and weight clipping is the paper's weakest practical part.
Reads with
- Generative Adversarial Networks
wgan is a repair of the original gan objective when supports barely overlap
- Auto-Encoding Variational Bayes
the contrasting generative route optimizes a likelihood bound instead of a learned distance
- Denoising Diffusion Probabilistic Models
diffusion later sidesteps adversarial geometry with a stable denoising training signal