The UdeM-McGill-MITACS machine learning seminar series is continuing its fall schedule.* Friday*'s seminar:
*Adaptive stochastic search: tuning Gaussians' covariances*
by Rémi Bardenet and Djalel Benbouzid University Paris-Sud XI
*Location*: Pavillon André-Aisenstadt (UdeM), room AA-3195 *Time*: *Friday, December 3, 10:30 (!)*
*Abstract*: Stochastic search algorithms are ubiquitous in optimization and statistics: they are at the core of some of the most efficient blackbox optimization techniques (e.g., Evolutionary Strategies), and of great practical use in Bayesian inference (e.g., in Metropolis-Hastings algorithms). In this talk I will concentrate on stochastic search techniques with Gaussian proposals that learn from their exploration to tune their covariance. After a quick overview of the field and methods, I will present 1) an evolutionary optimization algorithm with mixture proposals and its application to a Bayesian optimization problem, and 2) a novel attempt-in-progress at defining nonlinear adaptive Gaussian proposals. The first item deals with Gaussian Process based surrogate optimization: when the function to optimize is costly to evaluate (e.g., hyperparameter optimization in Machine Learning), one often relies on a surrogate model that is learnt on the fly, but optimizing this auxiliary surrogate can be a difficult task itself. The second problem is about mixing good properties of adaptive MCMC with reproducing kernel Hilbert spaces to exploit nonlinearity in the data.