- Lisa_seminaires - mailman.iro.umontreal.ca

REMINDER: UdeM-McGill-MITACS machine learning seminar: Fri June 20th, 15:30, AA-3195
by Dumitru Erhan 19 Jui '08

19 Jui '08

TOMORROW's UdeM-McGill-MITACS seminar (see http://www.iro.umontreal.ca/article.php3?id_article=107&lang=en) Learning hierarchical representations of natural images by Mike Lewicki, Computer Science Department and Center for the Neural Basis of Cognition Carnegie Mellon University Location : Pavillon André-Aisenstadt (UdeM) room 3195 Date and Time : June 20 2008, 15h30 What are the underlying computational principles that biology uses to transform the raw sensory signal into a hierarchy of representations that subserve higher-level perceptual tasks ? One hypothesis in this work is that biological representations are optimal from the viewpoint of statistical information processing, and adapt to the statistics of the natural sensory environment. In vision, we perceive complex natural edges, contours, and textures, whose retinal image is inherently highly variable. For accurate perception, the visual system must form abstract representations that remain invariant across individual fixations, but how this is achieved is poorly understood. In this talk, I will discuss an approach that is based on learning statistical distributions of local regions in a visual scene. The central hypothesis is learning these local distributions allows the visual system to generalize across similar images. I will present a model in which the joint activity of neurons encodes the probability distribution over their inputs and forms stable representations across complex patterns of variation. Trained on natural images, the model learns a compact set of functions that act as dictionary elements for image distributions typically encountered in natural scenes. Neurons in the model exhibit a wide range of properties observed in cortical neurons. These results provide a novel functional explanation for non-linear effects in complex cells in the primary visual cortex (V1) and make predictions about coding in higher visual areas, such as V2 and V4. This is joint work with Yan Karklin. Bio : Dr. Lewicki an associate professor in the Computer Science Department at Carnegie Mellon University and in the CMU-University of Pittsburgh Center for the Neural Basis of Cognition. He received his BS degree in mathematics and cognitive science from Carnegie Mellon University, his PhD degree in computation and neural systems from the California Institute of Technology, and did postdoctoral studies in the Computational Neurobiology Laboratory at the Salk Institute.

1 0

UdeM-McGill-MITACS machine learning seminar: Fri June 20th, 15:30, AA-3195
by Dumitru Erhan 16 Jui '08

16 Jui '08

Next week's UdeM-McGill-MITACS seminar (see http://www.iro.umontreal.ca/article.php3?id_article=107&lang=en) Learning hierarchical representations of natural images by Mike Lewicki, Computer Science Department and Center for the Neural Basis of Cognition Carnegie Mellon University Location : Pavillon André-Aisenstadt (UdeM) room 3195 Date and Time : June 20 2008, 15h30 What are the underlying computational principles that biology uses to transform the raw sensory signal into a hierarchy of representations that subserve higher-level perceptual tasks ? One hypothesis in this work is that biological representations are optimal from the viewpoint of statistical information processing, and adapt to the statistics of the natural sensory environment. In vision, we perceive complex natural edges, contours, and textures, whose retinal image is inherently highly variable. For accurate perception, the visual system must form abstract representations that remain invariant across individual fixations, but how this is achieved is poorly understood. In this talk, I will discuss an approach that is based on learning statistical distributions of local regions in a visual scene. The central hypothesis is learning these local distributions allows the visual system to generalize across similar images. I will present a model in which the joint activity of neurons encodes the probability distribution over their inputs and forms stable representations across complex patterns of variation. Trained on natural images, the model learns a compact set of functions that act as dictionary elements for image distributions typically encountered in natural scenes. Neurons in the model exhibit a wide range of properties observed in cortical neurons. These results provide a novel functional explanation for non-linear effects in complex cells in the primary visual cortex (V1) and make predictions about coding in higher visual areas, such as V2 and V4. This is joint work with Yan Karklin. Bio : Dr. Lewicki an associate professor in the Computer Science Department at Carnegie Mellon University and in the CMU-University of Pittsburgh Center for the Neural Basis of Cognition. He received his BS degree in mathematics and cognitive science from Carnegie Mellon University, his PhD degree in computation and neural systems from the California Institute of Technology, and did postdoctoral studies in the Computational Neurobiology Laboratory at the Salk Institute.

1 0

REMINDER: UdeM-McGill-MITACS machine learning seminar Tue May 27th, 10:00am, MC437
by Dumitru Erhan 26 Mai '08

26 Mai '08

This week's UdeM-McGill-MITACS seminar (see http://www.iro.umontreal.ca/article.php3?id_article=107&lang=en ): Learning in the Sample Compression Framework by Mohak Shah, Probabilistic Vision Group Centre for Intelligent Machines McGill University Location: McConnell Engineering (McGill), room 437 Time: May 27th 2008, 10am This talk will focus on the algorithms’ learning capabilities under the Sample Compression framework highlighting some of the advantages over other learning paradigms. We will show how theoretically motivated algorithms with practical realizable generalization error bounds can be obtained and further how these risk bounds can be utilized to guide the model selection. As a special case, we will focus on decision based classifiers (e.g. learning conjunctions or disjunctions of data-dependent features) and show how performances comparable to and in some cases better than the state-of-the-art learning algorithms such as SVM can be obtained.

1 0

UdeM-McGill-MITACS machine learning seminar Tue May 27th, 10:00am, MC437
by Dumitru Erhan 21 Mai '08

21 Mai '08

Next week's UdeM-McGill-MITACS seminar (see http://www.iro.umontreal.ca/article.php3?id_article=107&lang=en) : Learning in the Sample Compression Framework by Mohak Shah, Probabilistic Vision Group Centre for Intelligent Machines McGill University Location: McConnell Engineering (McGill), room 437 Time: May 27th 2008, 10am This talk will focus on the algorithms’ learning capabilities under the Sample Compression framework highlighting some of the advantages over other learning paradigms. We will show how theoretically motivated algorithms with practical realizable generalization error bounds can be obtained and further how these risk bounds can be utilized to guide the model selection. As a special case, we will focus on decision based classifiers (e.g. learning conjunctions or disjunctions of data-dependent features) and show how performances comparable to and in some cases better than the state-of-the-art learning algorithms such as SVM can be obtained.

1 0

[Fwd: RE: Montreal CIS Chapter Seminar: May 20, 2008 at 16:00 at ETS (Auditorium A-1600)]
by Yoshua Bengio 16 Mai '08

16 Mai '08

-------- Original Message -------- *From:* Cheriet, Mohamed *Sent:* May 1, 2008 3:32 PM *_______ ** **MONTREAL IEEE CIS MEETING / SEMINARS __________________*** * * *Date:** **Tuesday** Ma**y 20^th ,** 2008*** *Time:** 1**6**h* *Location:** **Auditorium A-1600, École de technologie supérieure*** *1100-Notre Dame W. (Corner Peel & Notre-Dame, Metro Bonaventure)* * *** *________________________________________________________________**____________________**_______* * * * * *Indexing and Searching Unconstrained Handwritten Documents* *Prof. **Venu Govindaraju* *University at Buffalo (SUNY Buffalo)*** *Dear Colleagues* *On behalf of the ExCom of Montreal IEEE CIS Chapter, I welcome you to attend the following exciting talk by a renowned researcher in the field. Please distribute the announcement among your students and colleagues. *** * * *Good News:** more exiting talks follow this year. I encourage you to attend, make suggestions, in order to make this Chapter a successful one. Thanks.* * * *I am looking forward to seeing you at the seminar.* *Best wishes,* *--- Prof. Mohamed Cheriet**, **Eng., Ph.D., SMIEEE* *Montreal IEEE CIS Chair* *ABSTRACT*** * * Recognition of handwritten documents has been a success in well-constrained domains, such as automatic recognition of handwritten postal addresses and validating bank check amounts. However, unconstrained offline handwriting recognition is still a very challenging task when lexicon size is very large. Thus, the large amount of handwritten material of the world, historical manuscripts and contemporary documents, are still inaccessible by search engines. We will describe our seminal work in real-time handwriting recognition which paved the way for the first deployment of end-to-end handwritten address interpretation system. We will also present loosely constrained applications such as indexing and searching hand-filled forms and unconstrained handwritten documents. We will conclude with an overview of our current research in using handwriting for web security. *Short BIO* Dr. Venu Govindaraju is a Professor of Computer Science and Engineering at the University at Buffalo (SUNY Buffalo). He received his B-Tech (Honors) from the Indian Institute of Technology (IIT), Kharagpur, India in 1986, and his Ph.D. from UB in 1992. Dr. Govindaraju is a Fellow of the IEEE (Institute of Electrical and Electronics Engineers) and a Fellow of the IAPR (International Association of Pattern Recognition). -- Yoshua Bengio

1 0

REMINDER: UdeM-McGill-MITACS machine learning seminar Tues May 13th, 10:00am, PAA3195
by Hugo Larochelle 12 Mai '08

12 Mai '08

This week's seminar (see http://www.iro.umontreal.ca/article.php3? id_article=107&lang=en): Deep belief networks are universal approximators and a recurrent neural network that learns to remember by Ilya Sutskever, Department of Computer Science University of Toronto Location: Pavillon André Aisenstadt (UdeM), room 3195 Time: May 13th 2008, 10:00am The talk will consist of two parts. In the first part I will prove that narrow deep belief networks can approximate any distribution over binary vectors to arbitrary accuracy, and describe a simple greedy algorithm that can learn such networks in an impractical manner. In the second part I will introduce a new kind of recurrent neural networks that can learn long term dependencies much better than standard recurrent neural networks.

1 0

UdeM-McGill-MITACS machine learning seminar Wed May 13th, 10:00am, PAA3195
by Hugo Larochelle 08 Mai '08

08 Mai '08

Next week's seminar (see http://www.iro.umontreal.ca/article.php3? id_article=107&lang=en): Deep belief networks are universal approximators and a recurrent neural network that learns to remember by Ilya Sutskever, Department of Computer Science University of Toronto Location: Pavillon André Aisenstadt (UdeM), room 3195 Time: May 13th 2008, 10:00am The talk will consist of two parts. In the first part I will prove that narrow deep belief networks can approximate any distribution over binary vectors to arbitrary accuracy, and describe a simple greedy algorithm that can learn such networks in an impractical manner. In the second part I will introduce a new kind of recurrent neural networks that can learn long term dependencies much better than standard recurrent neural networks.

1 0

SPECIAL TIME: UdeM-McGill-MITACS machine learning seminar Wed April 30th, 14:00, PAA3195
by Hugo Larochelle 29 Avr '08

29 Avr '08

This week's seminar (see http://www.iro.umontreal.ca/article.php3? id_article=107&lang=en). Also, you are all invited to the thesis defense of Nicolas Le Roux, right after this MITACS seminar, in the same room (see abstract below): ************************* Learning Deep Hierarchies of Sparse and Invariant Features by Yann LeCun, Courant Institute of Mathematical Science New York University Location: Pavillon André Aisenstadt (UdeM), room 3195 Time: April 30th 2008, 14:00 A long-term goal of Machine Learning research is to solve highy complex "intelligent" tasks, such as visual perception auditory perception, and language understanding. To reach that goal, the ML community must solve two problems: the Deep Learning Problem, and the Partition Function Problem. There is considerable theoretical and empirical evidence that complex tasks, such as invariant object recognition in vision, require "deep" architectures, composed of multiple layers of trainable non-linear modules. The Deep Learning Problem is related to the difficulty of training such deep architectures. Several methods have recently been proposed to train (or pre-train) deep architectures in an unsupervised fashion. Each layer of the deep architecture is composed of an encoder which computes a feature vector from the input, and a decoder which reconstructs the input from the features. A large number of such layers can be stacked and trained sequentially, thereby learning a deep hierarchy of features with increasing levels of abstraction. The training of each layer can be seen as shaping an energy landscape with low valleys around the training samples and high plateaus everywhere else. Forming these high plateaus constitute the so-called Partition Function problem. A particular class of methods for deep energy-based unsupervised learning will be described that solves the Partition Function problem by imposing sparsity constraints on the features. The method can learn multiple levels of sparse and overcomplete representations of data. When applied to natural image patches, the method produces hierarchies of filters similar to those found in the mammalian visual cortex. An application to category-level object recognition with invariance to pose and illumination will be described (with a live demo). Another application to vision-based navigation for off-road mobile robots will be described (with videos). The system autonomously learns to discriminate obstacles from traversable areas at long range. This is joint work with Y-Lan Boureau, Sumit Chopra, Raia Hadsell, Fu- Jie Huang, Koray Kavakcuoglu, and Marc’Aurelio Ranzato. ************************ Défense de thèse de Nicolas Le Roux: Titre: Avancées théoriques sur la représentation et l’optimisation des réseaux de neurones Quand et où: Mercredi le 30 avril, 15:30, Pavillon Aisenstadt, salle 3195 Directeur: Yoshua Bengio Jury: Pierre L'Ecuyer, Pascal Vincent Examinateur: Yann Le Cun Résumé: Les réseaux de neurones sont une classe d'algorithmes d'apprentissage très répandue dans le domaine de l'intelligence artificielle. L'engouement général qu'ils suscitèrent dans les années 80 s'estompa malheureusement du fait de la difficulté de leur optimisation. L'apparition des méthodes à noyau dans les années 90 accéléra ce phénomène. Après avoir mis en exergue les limitations des méthodes à noyau et des algorithmes peu profonds en général, je présenterai plusieurs extensions des réseaux de neurones étendant leurs possibilités et facilitant leur optimisation. Enfin, j'effectuerai une analyse détaillée des algorithmes profonds, tout en présentant un algorithme de descente de gradient rapide permettant leur utilisation dans des applications où la vitesse de traitement est essentielle.

1 0

SPECIAL TIME: UdeM-McGill-MITACS machine learning seminar Wed April 30th, 14:00, PAA3195
by Hugo Larochelle 23 Avr '08

23 Avr '08

Next week's seminar (see http://www.iro.umontreal.ca/article.php3? id_article=107&lang=en): Learning Deep Hierarchies of Sparse and Invariant Features by Yann LeCun, Courant Institute of Mathematical Science New York University Location: Pavillon André Aisenstadt (UdeM), room 3195 Time: April 30th 2008, 14:00 A long-term goal of Machine Learning research is to solve highy complex "intelligent" tasks, such as visual perception auditory perception, and language understanding. To reach that goal, the ML community must solve two problems: the Deep Learning Problem, and the Partition Function Problem. There is considerable theoretical and empirical evidence that complex tasks, such as invariant object recognition in vision, require "deep" architectures, composed of multiple layers of trainable non-linear modules. The Deep Learning Problem is related to the difficulty of training such deep architectures. Several methods have recently been proposed to train (or pre-train) deep architectures in an unsupervised fashion. Each layer of the deep architecture is composed of an encoder which computes a feature vector from the input, and a decoder which reconstructs the input from the features. A large number of such layers can be stacked and trained sequentially, thereby learning a deep hierarchy of features with increasing levels of abstraction. The training of each layer can be seen as shaping an energy landscape with low valleys around the training samples and high plateaus everywhere else. Forming these high plateaus constitute the so-called Partition Function problem. A particular class of methods for deep energy-based unsupervised learning will be described that solves the Partition Function problem by imposing sparsity constraints on the features. The method can learn multiple levels of sparse and overcomplete representations of data. When applied to natural image patches, the method produces hierarchies of filters similar to those found in the mammalian visual cortex. An application to category-level object recognition with invariance to pose and illumination will be described (with a live demo). Another application to vision-based navigation for off-road mobile robots will be described (with videos). The system autonomously learns to discriminate obstacles from traversable areas at long range. This is joint work with Y-Lan Boureau, Sumit Chopra, Raia Hadsell, Fu- Jie Huang, Koray Kavakcuoglu, and Marc’Aurelio Ranzato.

1 0

NEW TIME: UdeM-McGill-MITACS machine learning seminar Tues April 14th, 11:30am, PAA3195
by Hugo Larochelle 10 Avr '08

10 Avr '08

Next week's seminar (see http://www.iro.umontreal.ca/article.php3? id_article=107&lang=en): Restricted Boltzmann machines: Performance and behavior on image databases and future plans by Karol Gregor, California Institute of Technology Location: Pavillon André Aisenstadt (UdeM), room 3195 Time: April 14th 2008, 11:30am Inspired by amazing capabilities of cortex and at the same time its relatively homogeneous hierarchical structure, it is a worthwhile pursuit to develop algorithms that can be repeated in a hierarchical fashion and applied to large class of problems including vision, speech recognition and motor control. Restricted Boltzmann machines are a very capable tool and a good step in this direction. First, I will discuss our study of the restricted Boltzmann machines on image databases of Scenes and Caltech 256 as applied to bag of words of sift features. I will show dependencies on various parameters and comparison of performance to other methods. There are other interesting conclusions too: For a given amount of labeled data, the performance improves if the system is pre-trained on a larger number of images; Neurons seem to appear after unsupervised pre-training that explicitly represent certain categories (to some extent). Next I will discuss a simple experiment with temporal sequences. Then I will put this problem into the context of cortical computations and outline a plan that I believe is very reasonable to undertake in the future.

1 0