- Lisa_seminaires - mailman.iro.umontreal.ca

Oct 10th, 14:00 - UdeM-McGill-MITACS seminar by Ronan Collobert
by Pascal Lamblin 07 Oct '08

07 Oct '08

Ladies and gentlemen, Here comes the first MITACS seminar, by Ronan Collobert, this friday, 2 pm at UdeM. Ronan Collobert Friday Oct 10th, 2008, 14:00 Room 3195 (pav. André-Aisenstadt) Large Scale Learning for Natural Language Processing We describe a neural network architecture that given a sentence, outputs a host of language processing predictions: part-of-speech tags, chunks, named entity tags, semantic roles, semantically similar words and the likelihood that the sentence makes sense (grammatically and semantically) using a language model. The entire network is trained jointly on all these tasks using weight-sharing, an instance of multitask learning. All the tasks use labeled data except the language model which is learnt from unlabeled text and represents a novel way of performing semi-supervised learning for the shared tasks. We show how both multitask learning and semi-supervised learning improve the generalization of the shared tasks, resulting in a learnt model with state-of-the-art performance. -- Pascal

1 0

UdeM-McGill-MITACS machine learning seminar: Fri August 22nd, 14:00, AA-3195
by Dumitru Erhan 21 Aoû '08

21 Aoû '08

TOMORROW'S UdeM-McGill-MITACS seminar (see http://www.iro.umontreal.ca/article.php3?id_article=107&lang=en) : Differentiable Sparse Coding by J. Andrew Bagnell Carnegie Mellon Robotics Institute and Machine Learning Department Location : Pavillon André-Aisenstadt (UdeM) room 3195 Date and Time : Friday, August 22 2008, 14h00 Sparse approximation is a key technique developed recently in engineering and the sciences which attempts to approximate an input signal, denoted here by X, in terms of a “sparse” combination of ﬁxed bases B. This approach relies on an optimization algorithm to infer the most probable weights \hat{W} to reconstruct input signals, given the model X ≈ f (BW). Priors which produce sparse solutions for W , especially L1 regularization, have gained attention because of their usefulness in ill-posed engineering problems ranging from geology to magnetic resonance imaging, their ability to elucidate certain neuro-biological phenomena, and their ability to condense a high-dimensional input signal into useful features for classiﬁcation. Sparse coding – closely connected to Independent Component Analysis as well as certain approaches to matrix factorization – extends sparse approximation by not only performing optimization to compute the best set of weights for a given input signal, but also learning bases B which lead to a compact representation of input signals. Unfortunately, existing sparse coding algorithms that efficiently infer the latent weight vector are difficult to integrate into larger learning architectures. It has been convincingly demonstrated that back-propagation is a crucial tool for tuning an existing generative model’s performance discriminatively to lead to good supervised performance. Similarly, greedy layer-wise strategies to building deep generative models rely upon a back-propagation step to achieve excellent model performance. Existing sparse coding architectures produce a latent representation \hat{W} that is an unstable, discontinuous function of the inputs and bases; an arbitrarily small change in input can lead to the selection of a completely different set of latent weights. We present a new approach to coding with an efﬁcient, convex inference step based on minimizing KL-divergence. We show this increased stability leads to better semi-supervised classiﬁcation performance. Additionally, although inferring the latent weights requires an optimization procedure (i.e. is not closed form) we demonstrate that for a large class of Bregman-divergence based priors and loss functions, we may use implicit differentiation to efficiently backpropagate error signals. The sparse coding bases can then be optimized discriminatively leading to outstanding empirical performance and enabling sparse coding to form a part of a larger learning architecture. Joint work with David M. Bradley

1 0

Rappel: prédoc oral d'Olivier Delalleau à 14h00 en 3195
by Pascal Vincent 19 Aoû '08

19 Aoû '08

Vous êtes tous cordialement invités au prédoc oral d'Olivier Delalleau, étudiant au DIRO sous la direction de Yoshua Bengio. Date: mardi 19 aout Heure: 14h00 Salle: 3195, Pav. Aisenstadt Titre : Problématiques de l'extraction de caractéristiques hiérarchique Résumé: De nombreux algorithmes d'apprentissage machine voient leurs performances se dégrader lorsqu'appliqués à des données complexes en haute dimension, alors qu'ils peuvent être très efficaces sur des données plus simples. L'extraction de caractéristiques peut être vue comme une étape de "simplification" des données, cruciale pour les performances. Lors de cette présentation, je passerai en revue les principales techniques d'extraction de caractéristiques et leurs limitations. Je motiverai ainsi l'utilisation de méthodes hiérarchiques, i.e. où les caractéristiques extraites à chaque niveau se basent sur des caractéristiques extraites aux niveaux précédents. J'aborderai plus en détails le problème de l'évaluation de l'extraction de caractéristiques, en particulier dans le cas des modèles génératifs à énergie comme les machines de Boltzmann restreintes. Finalement, je présenterai les axes de recherche privilégiés dans cette thèse, dont le but est d'obtenir des algorithmes d'extraction de caractéristiques hiérarchique plus performants. -- Pascal Vincent, Professeur adjoint / Assistant Professor, Département d'Informatique et Recherche Opérationnelle, Université de Montréal.

1 0

[Fwd: Prédoc oral de Olivier Delalleau, mardi 19 août]
by Dumitru Erhan 18 Aoû '08

18 Aoû '08

---------------------------- Message original ---------------------------- Objet: Prédoc oral de Olivier Delalleau, mardi 19 août De: "Pascal Vincent" <vincentp(a)iro.umontreal.ca> Date: Ven 15 août 2008 17:13 À: "seminaires" <seminaires(a)iro.umontreal.ca> -------------------------------------------------------------------------- Vous êtes tous cordialement invités au prédoc oral de *Olivier Delalleau*, étudiant au DIRO sous la direction de Yoshua Bengio. Date: mardi 19 aout Heure: 14h00 Salle: 3195, Pav. Aisenstadt Titre : Problématiques de l'extraction de caractéristiques hiérarchique Résumé: De nombreux algorithmes d'apprentissage machine voient leurs performances se dégrader lorsqu'appliqués à des données complexes en haute dimension, alors qu'ils peuvent être très efficaces sur des données plus simples. L'extraction de caractéristiques peut être vue comme une étape de "simplification" des données, cruciale pour les performances. Lors de cette présentation, je passerai en revue les principales techniques d'extraction de caractéristiques et leurs limitations. Je motiverai ainsi l'utilisation de méthodes hiérarchiques, i.e. où les caractéristiques extraites à chaque niveau se basent sur des caractéristiques extraites aux niveaux précédents. J'aborderai plus en détails le problème de l'évaluation de l'extraction de caractéristiques, en particulier dans le cas des modèles génératifs à énergie comme les machines de Boltzmann restreintes. Finalement, je présenterai les axes de recherche privilégiés dans cette thèse, dont le but est d'obtenir des algorithmes d'extraction de caractéristiques hiérarchique plus performants. -- Pascal Vincent, Professeur adjoint / Assistant Professor, Département d'Informatique et Recherche Opérationnelle, Université de Montréal.

1 0

[Fwd: Predoc oral de James Bergstra, jeudi 21 aout]
by Dumitru Erhan 18 Aoû '08

18 Aoû '08

---------------------------- Message original ---------------------------- Objet: Predoc oral de James Bergstra, jeudi 21 aout De: "Pierre L'Ecuyer" <lecuyer(a)iro.umontreal.ca> Date: Ven 15 août 2008 15:40 À: "seminaires" <seminaires(a)iro.umontreal.ca> "Pierre L'Ecuyer" <lecuyer(a)iro.umontreal.ca> -------------------------------------------------------------------------- Bonjour. Vous êtes tous cordialement invités au predoc oral de *James Bergstra*, etudiant au DIRO sous la direction de Yoshua Bengio. Date: jeudi 21 aout Heure: 16h30 Salle: 3195, Pav. Aisenstadt Titre: Object Recognition and Multiscale prediction with Recurrent Neural Networks Résumé / Abstract: Object recognition rates in humans are much higher when subjects are allowed to look around an image and reflect a little, but shallow feedforward models are unable to exhibit this sort of behaviour. Deep models with multiple layers of nonlinear processing may be said to reflect a little, but my doctoral work will look at strategies for training recurrent neural networks with this capacity. Deep neural networks are much like recurrent networks: an unfolded recurrent architecture is a deep network with tied weights, and any deep network can be transformed into a recurrent architecture by adding hidden units to the recurrent state. I will to translate recent techniques for learning deep neural networks into the domain of recurrent networks. My thesis proposal will outline new recurrent models, new training strategies, and a new way to perform static classification with temporal models. My doctoral work so far has centered on the use of physiologically-motivated cell models in artificial neural networks, and on techniques for training recurrent networks to learn over short and medium-term timescales. -- Pierre L'Ecuyer, Professeur Titulaire Chaire du Canada en Simulation et Optimisation Stochastique CIRRELT, GERAD, and DIRO, Université de Montréal, Canada http://www.iro.umontreal.ca/~lecuyer

1 0

Fwd: Predoc oral de Dumitru Erhan
by Dumitru Erhan 14 Aoû '08

14 Aoû '08

Reminder: Begin forwarded message: > From: mignotte(a)iro.umontreal.ca > Date: August 8, 2008 5:57:20 PM EDT (CA) > To: seminaires(a)iro.umontreal.ca > Subject: Predoc oral de Dumitru Erhan > > > Bonjour. > > Vous êtes tous cordialement invités au predoc oral > de Dumitru Erhan (dont le directeur de these est Yoshua Bengio) > au DIRO, le jeudi 14 aout a 15:00 en salle 3195 > (Pavillon Aisenstadt). > > > Voici le titre de sa presentation: > > "Heuristics for Optimizing Deep Architectures > > Deep architectures are viewed by many as an eﬃcient new way of > tackling diﬃcult Artiﬁcial Intelligence problems, such as vision > or > natural language processing. A ﬂurry of models and algorithms to > eﬃciently train them have sprung over the last several years. The > key > ingredient seems to be greedy, layer-wise initialization of such > models. I will review the state-of-the-art and identify some of the > open issues with deep architectures. I will then present our work that > tackled some of these issues and propose a series of theoretical > extensions and experimental work that suggest new directions for > research on deep architectures --- such as learning multimodal > representations of visual, textual and musical data as well as > collective training of networks. I argue that tackling these problem > will give us new insights into the inner workings of these models and > could help the optimization process of deep architectures. " > > Max Mignotte >

1 0

Fwd: Predoc oral de Dumitru Erhan
by Dumitru Erhan 09 Aoû '08

09 Aoû '08

Begin forwarded message: > From: mignotte(a)iro.umontreal.ca > Date: August 8, 2008 5:57:20 PM EDT (CA) > To: seminaires(a)iro.umontreal.ca > Subject: Predoc oral de Dumitru Erhan > > ---------------------------- Message original > ---------------------------- > Objet: > De: mignotte(a)iro.umontreal.ca > Date: Ven 8 août 2008 17:56 > À: seminaires(a)iro.umontreal.ca > -------------------------------------------------------------------------- > > > > Bonjour. > > Vous êtes tous cordialement invités au predoc oral > de Dumitru Erhan (dont le directeur de these est Yoshua Bengio) > au DIRO, le jeudi 14 aout a 15:00 en salle 3195 > (Pavillon Aisenstadt). > > > Voici le titre de sa presentation: > > "Heuristics for Optimizing Deep Architectures > > Deep architectures are viewed by many as an eﬃcient new way of > tackling diﬃcult Artiﬁcial Intelligence problems, such as vision > or > natural language processing. A ﬂurry of models and algorithms to > eﬃciently train them have sprung over the last several years. The > key > ingredient seems to be greedy, layer-wise initialization of such > models. I will review the state-of-the-art and identify some of the > open issues with deep architectures. I will then present our work that > tackled some of these issues and propose a series of theoretical > extensions and experimental work that suggest new directions for > research on deep architectures --- such as learning multimodal > representations of visual, textual and musical data as well as > collective training of networks. I argue that tackling these problem > will give us new insights into the inner workings of these models and > could help the optimization process of deep architectures. " > > Max Mignotte >

1 0

Upcoming Seminar - Stuart Russell, Berkeley: June 30/08
by Joelle Pineau 29 Jui '08

29 Jui '08

Hello everyone, Please note this important seminar. Stuart Russell is a world-renowned researcher in Artificial Intelligence, and author of the leading textbook on the topic. Everyone is encouraged to attend! Monday June 30th, 2008 McConnell Engineering Rm 12 2:00pm Hierarchical learning and decision making Stuart Russell Computer Science Division University of California, Berkeley Effective behaviour in complex environments requires some form of hierarchical structure. The first part of the talk covers hierarchical reinforcement learning, wherein guidance as to the correct hierarchical structure can be supplied to an agent in the form of a *partial program* in which choices may be left unspecified. The ALisp language - an partial programming extension of Lisp - includes reinforcement learning algorithms that converge to the optimal complete program consistent with the given partial program. ALisp includes constructs for specifying asynchronous concurrent activities. ALisp shows promise as a tool for scaling up reinforcement learning. The second part of the talk covers lookahead decision making with hierarchical constraints on behavior. Here the setting is classical hierarchical planning. We resolve a longstanding open problem by defining *angelic* semantics for high-level actions and providing the first algorithms capable of proving that a high-level plan is correct and optimal without considering its concrete implementations. We also consider online hierarchical search with bounded lookahead. [Joint work with Ron Parr, David Andre, Bhaskara Marthi, Andy Zimdars, David Latham, Carlos Guestrin, Jason Wolfe] Stuart Russell was born in Portsmouth, England. He received his B.A. with first-class honours in Physics from Wadham College, Oxford University in 1982, and his Ph.D. in Computer Science from Stanford in 1986. He then joined the faculty of the University of California, Berkeley, where he is currently Professor and Chair of Computer Science. In 2003 he was inducted as a Fellow of the Association for Computing Machinery. Stuart Russell is an author of the textbook Artificial Intelligence: A Modern Approach (together with Peter Norvig).

2 1

UdeM-McGill-MITACS machine learning seminar: Thu June 26th, 14:00, AA-3195
by Dumitru Erhan 26 Jui '08

26 Jui '08

TODAY's UdeM-McGill-MITACS seminar (see http://www.iro.umontreal.ca/article.php3?id_article=107&lang=en) : Artificial Intelligence in Seven Years? by Léon Bottou NEC Labs America Location : Pavillon André-Aisenstadt (UdeM) room 3195 Date and Time : Thursday, June 26 2008, 14h00 This highly informal and highly prospective presentation discusses an ensemble of techniques which I believe form a plausible path to achieve artificial intelligence in the next decade. The discussion includes: available data and transfer learning, statistical learning theory is not yet ready to help, recursive representations in finite dimensional spaces, a general algorithm for parsing text and images, reasoning with neurons (artificial neurons !) Of course each of these points contain potential stumbling blocks. The purpose of this presentation is therefore to invite a discussion. P.S. - This is a whiteboard talk.

1 0

UdeM-McGill-MITACS machine learning seminar: Thu June 26th, 14:00, AA-3195
by Dumitru Erhan 24 Jui '08

24 Jui '08

THURSDAY's UdeM-McGill-MITACS seminar (see http://www.iro.umontreal.ca/article.php3?id_article=107&lang=en) : Artificial Intelligence in Seven Years? by Léon Bottou NEC Labs America Location : Pavillon André-Aisenstadt (UdeM) room 3195 Date and Time : Thursday, June 26 2008, 14h00 This highly informal and highly prospective presentation discusses an ensemble of techniques which I believe form a plausible path to achieve artificial intelligence in the next decade. The discussion includes: available data and transfer learning, statistical learning theory is not yet ready to help, recursive representations in finite dimensional spaces, a general algorithm for parsing text and images, reasoning with neurons (artificial neurons !) Of course each of these points contain potential stumbling blocks. The purpose of this presentation is therefore to invite a discussion. P.S. - This is a whiteboard talk.

1 0