Lisa_seminaires Mars 2008

lisa_seminaires@iro.umontreal.ca

2 participants
6 discussions

REMINDER: UdeM-McGill-MITACS machine learning seminar Tues March 25th, 10:00am, PAA3195
by Hugo Larochelle 24 Mar '08

24 Mar '08

This week, we are having a double feature seminar (see http:// www.iro.umontreal.ca/article.php3?id_article=107&lang=en): Image Classification with Higher-Order Neural Models and Deep Learning with Denoising Autoencoders by James Bergstra and Pascal Vincent, Département d’Informatique et de Recherche Opérationnelle Université de Montréal Location: Pavillon André Aisenstadt (UdeM), room 3195 Time: March 25th 2008, 10h00 First talk, by James Bergstra: Neural network research in machine learning grew out of theories from computational neuroscience from the 1960s. While the class of affine- sigmoidal feature extractors has been studied extensively since the mid 1980s, the computational neuroscience community has moved on to new models that are qualitatively different and more descriptive, without being substantially more computationally expensive. This paper brings a particular model proposed in (Rust, 2005) for low level neurons in the macaque visual system into a machine-learning context: we evaluate their model as an activation function (feature- extractor) for single-layer neural networks that perform image classification. The function we evaluate is somewhat similar to the higher-order processing units discussed in (Minsky, 1969) and the Sigma-Pi units, but avoids the computational difficulties associated with these models by representing the second-order interaction weights with a low-rank positive semi-definite matrix, and avoids the learning difficulties associated with these models by using a gentler non-linearity than the logistic sigmoid. Remarkably good comparative results are obtained on three image classification tasks including 1.4 \% error on MNIST using a single-layer network. These results suggest that a single hidden layer neural network equipped with this neuron model can capture important patterns that escape standard models such as sigmoid neural networks and support vector machines based on gaussian and polynomial kernels. Second talk, by Pascal Vincent: Previous work has shown that the difficulties in learning deep generative or discriminative models can be overcome by an initial unsupervised learning step that maps inputs to useful intermediate representations. We introduce and motivate a new training principle for unsupervised learning of a representation based on the idea of making the learned representations robust to partial corruption of the input pattern. This approach can be used to train autoencoders, and these denoising autoencoders can be stacked to initialize deep architectures. The algorithm can be motivated from a manifold learning and information theoretic perspective or from a generative model perspective. Comparative experiments clearly show the surprising advantage of corrupting the input of autoencoders on a pattern classification benchmark suite.

1 0

UdeM-McGill-MITACS machine learning seminar Tues March 25th, 10:00am, PAA3195
by Hugo Larochelle 21 Mar '08

21 Mar '08

Next week, we are having a double feature seminar (see http:// www.iro.umontreal.ca/article.php3?id_article=107&lang=en): Image Classification with Higher-Order Neural Models and Deep Learning with Denoising Autoencoders by James Bergstra and Pascal Vincent, Département d’Informatique et de Recherche Opérationnelle Université de Montréal Location: Pavillon André Aisenstadt (UdeM), room 3195 Time: March 25th 2008, 10h00 First talk, by James Bergstra: Neural network research in machine learning grew out of theories from computational neuroscience from the 1960s. While the class of affine- sigmoidal feature extractors has been studied extensively since the mid 1980s, the computational neuroscience community has moved on to new models that are qualitatively different and more descriptive, without being substantially more computationally expensive. This paper brings a particular model proposed in (Rust, 2005) for low level neurons in the macaque visual system into a machine-learning context: we evaluate their model as an activation function (feature- extractor) for single-layer neural networks that perform image classification. The function we evaluate is somewhat similar to the higher-order processing units discussed in (Minsky, 1969) and the Sigma-Pi units, but avoids the computational difficulties associated with these models by representing the second-order interaction weights with a low-rank positive semi-definite matrix, and avoids the learning difficulties associated with these models by using a gentler non-linearity than the logistic sigmoid. Remarkably good comparative results are obtained on three image classification tasks including 1.4 \% error on MNIST using a single-layer network. These results suggest that a single hidden layer neural network equipped with this neuron model can capture important patterns that escape standard models such as sigmoid neural networks and support vector machines based on gaussian and polynomial kernels. Second talk, by Pascal Vincent: Previous work has shown that the difficulties in learning deep generative or discriminative models can be overcome by an initial unsupervised learning step that maps inputs to useful intermediate representations. We introduce and motivate a new training principle for unsupervised learning of a representation based on the idea of making the learned representations robust to partial corruption of the input pattern. This approach can be used to train autoencoders, and these denoising autoencoders can be stacked to initialize deep architectures. The algorithm can be motivated from a manifold learning and information theoretic perspective or from a generative model perspective. Comparative experiments clearly show the surprising advantage of corrupting the input of autoencoders on a pattern classification benchmark suite.

1 0

REMINDER: UdeM-McGill-MITACS machine learning seminar Fri March 14th, 12:00pm, MC437 NEW TIME!
by Hugo Larochelle 13 Mar '08

13 Mar '08

Next week's seminar (see http://www.iro.umontreal.ca/article.php3? id_article=107&lang=en): Discriminative Methods with Structure by Simon Lacoste-Julien, Computer Science University of California Location: McConnell Engineering Building (McGill), room 437 Time: March 14th 2008, 12h00 Real world problems such as machine translation involve complex dependencies. Generative models have provided an elegant and flexible framework to model those dependencies, but they appear to lack robustness to model misspecification compared to discriminative models for classification. In this talk, we present methods for leveraging the advantages of generative models in the discriminative framework. In the first part of the talk, we tackle the word alignment problem from natural language processing. We formulate it as a weighted bipartite matching problem and show how to learn the weights by using a large-margin approach for structured prediction. By providing a flexible discriminative modeling framework, we were able to cut the Alignment Error Rate in half compared to the previous best performing generative models for word alignment. In the second part of the talk, we study probabilistic topic models which have been popular for modeling latent structures in text documents (as bag of words) or images (as bag of visual words). They are usually trained as generative models with maximum likelihood estimation, though this could be suboptimal if one is interested in doing classification. In contrast, we present a discriminative version of the Latent Dirichlet Allocation (LDA) model which attempts to uncover the latent structure in the documents while optimizing its predictive power for the task of classification. We present results in the domains of document classification and scene categorization. (joint work with Fei Sha, Ben Taskar, Dan Klein and Michael I. Jordan)

1 0

NEW TIME! UdeM-McGill-MITACS machine learning seminar Fri March 14th, 12:00pm, MC437
by Hugo Larochelle 07 Mar '08

07 Mar '08

1 0

Hummmm... beer....
by Hugo Larochelle 06 Mar '08

06 Mar '08

Hi everyone, For those who know Simon Osindero, I've been planning to go get drinks for a while now with him and we've converged to two possible times. I know some of you guys have mentioned interest in such a gathering, so I'm trying to get an idea of who wants to come and when could they come... The first option would be tonight (Thursday) at around 22h15, and the other would be tomorrow (Friday) for either a "6 à 8" or later on. We would go to McCarrolds (corner côtes-des-neiges and lacombe). This will also be an opportunity for me to taste my last beer in a bar for a LONG while ;-) So, those who are interested, please reply to this email as soon as possible (say before 14h30) so that I can tell Simon which option it's going to be... Cheers! Hugo

1 0

REMINDER: UdeM-McGill-MITACS machine learning seminar Tues March 4th, 10:00am, MC437
by Hugo Larochelle 03 Mar '08

03 Mar '08

This week's seminar (see http://www.iro.umontreal.ca/article.php3? id_article=107&lang=en): A Stochastic Algorithm for Partially Observable Markov Decision Processes (POMDPs) by Francois Laviolette, Département d’Informatique Université Laval Location: McConnell Engineering Building (McGill), room 437 Time: March 4th 2008, 10h00 We introduce a new backup operator for point-based POMDP’s algorithms which performs a look-ahead search at depth greater than one. We apply this operator into a new algorithm, called Stochastic Search Value Iteration (SSVI). This new algorithm relies on stochastic explo- ration of the environment in order to update the value function. The un- derlying ideas are very similar to temporal difference learning algorithms for MDPs. In particular, SSVI takes advantage of a soft-max actions se- lection function and of the random character of the environment itself. This is in opposition with existing POMDP’s point-based algorithms. Empirical results show that our algorithm is very competitive on usual benchmark problems. This suggests that stochastic algorithms are an alternative for solving large POMDPs.

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

Lisa_seminaires Mars 2008