December 2010 - Lisa_seminaires - mailman.iro.umontreal.ca

Another UdeM-McGill-MITACS machine learning seminar Thu Dec. 16@16h00, MC-103
by Dumitru Erhan 15 Dec '10

15 Dec '10

There's a MITACS seminar at McGill this Thursday, too: * Approximate Inference for the Loss-Calibrated Bayesian *by Simon Lacoste-Julien Department of Engineering University of Cambridge *Location*: McConnell Engineering Bldg (McGill), room* 103* *Time*: *Thursday, December 16, 16:00 * * * Bayesian decision theory provides a well-defined theoretical framework for rational decision making under uncertainty. However, even if we assume that our subjective beliefs about the world have been well-specified, we usually need to resort to approximations in order to use them in practice. Despite the central role of the loss in the decision theory formulation, most prevalent Bayesian approximation methods focus on approximating the posterior over parameters with no consideration of the loss. In this talk, our main point is to bring back in focus the need to *calibrate* the approximation methods to the *loss* under consideration. This philosophy has already been widely applied in the frequentist statistics / discriminative machine learning literature, as for example with the use of surrogate loss functions, but not in Bayesian statistics surprisingly. We provide examples showing the limitation of disregarding the loss in standard approximate inference schemes and outline several interesting research directions arising from this new perspective. As a first loss-calibrated attempt, we propose an EM-like algorithm on the Bayesian posterior risk and show how it can improve a standard approach to Gaussian process classification when the losses are asymmetric.

1 0

UdeM-McGill-MITACS machine learning seminar Fri Dec. 17@10h30, Z-209
by Dumitru Erhan 15 Dec '10

15 Dec '10

The UdeM-McGill-MITACS machine learning seminar series has its last Fall seminar this* Friday. *Please note the changed location and time: * Label Embedding Trees for Large Multi-Class Tasks *by Samy Bengio Google Research *Location*: Pavillon Claire-McNicoll (UdeM), room* Z-209* *Time*: *Friday, December 17, 10:30 (!)* *Abstract*: Multi-class classification becomes challenging at test time when the number of classes is very large and testing against every possible class can become computationally infeasible. This problem can be alleviated by imposing (or learning) a structure over the set of classes. We propose an algorithm for learning a tree-structure of classifiers which, by optimizing the overall tree loss, provides superior accuracy to existing tree labeling methods. We also propose a method that learns to embed labels in a low dimensional space that is faster than non-embedding approaches and has superior accuracy to existing embedding approaches. Finally we combine the two ideas resulting in the {\em label embedding tree} that outperforms alternative methods including One-vs-Rest while being orders of magnitude faster.

1 0

UdeM-McGill-MITACS machine learning seminar Fri Dec. 3@10h30, AA-3195
by Dumitru Erhan 02 Dec '10

02 Dec '10

The UdeM-McGill-MITACS machine learning seminar series is continuing its fall schedule.* Friday*'s seminar: *Adaptive stochastic search: tuning Gaussians' covariances* by Rémi Bardenet and Djalel Benbouzid University Paris-Sud XI *Location*: Pavillon André-Aisenstadt (UdeM), room AA-3195 *Time*: *Friday, December 3, 10:30 (!)* *Abstract*: Stochastic search algorithms are ubiquitous in optimization and statistics: they are at the core of some of the most efficient blackbox optimization techniques (e.g., Evolutionary Strategies), and of great practical use in Bayesian inference (e.g., in Metropolis-Hastings algorithms). In this talk I will concentrate on stochastic search techniques with Gaussian proposals that learn from their exploration to tune their covariance. After a quick overview of the field and methods, I will present 1) an evolutionary optimization algorithm with mixture proposals and its application to a Bayesian optimization problem, and 2) a novel attempt-in-progress at defining nonlinear adaptive Gaussian proposals. The first item deals with Gaussian Process based surrogate optimization: when the function to optimize is costly to evaluate (e.g., hyperparameter optimization in Machine Learning), one often relies on a surrogate model that is learnt on the fly, but optimizing this auxiliary surrogate can be a difficult task itself. The second problem is about mixing good properties of adaptive MCMC with reproducing kernel Hilbert spaces to exploit nonlinearity in the data.

1 1