See you there!
Rim and Sai
TITLE A Variational Inequality Perspective on Generative Adversarial Networks
KEYWORDS
GANs, variational inequality, mini-max optimization
ABSTRACT
Generative adversarial networks (GANs) form a generative modeling approach known for producing appealing samples, but they are notably difficult to train. One common way to tackle this issue has been to propose new formulations of the GAN objective. Yet, surprisingly few studies have looked at optimization methods designed for this adversarial training. In this work, we cast GAN optimization problems in the general variational inequality framework. Tapping into the mathematical programming literature, we counter some common misconceptions about the difficulties of saddle point optimization and propose to extend methods designed for variational inequalities to the training of GANs. We apply averaging, extrapolation and a novel computationally cheaper variant that we call extrapolation from the past to the stochastic gradient method (SGD) and Adam.
BIO
Gauthier Gidel received the Diplôme de l’École Normale Supérieure in 2017 (ULM MPI2013) and the Master of Science MVA from École Normale supérieur Paris-Saclay in 2016. Gauthier is currently pursuing his PhD at Mila and DIRO from Université de Montréal under the supervision of Simon Lacoste-Julien.Gauthier’s PhD thesis topic revolves around saddle point optimization (a.k.a mini-max problems) for machine learning and more generally variational inequalities on which Gauthier has published several papers [Gidel et al. 2017, Gidel et al. 2018].