- Lisa_teatalk - mailman.iro.umontreal.ca

Reminder: Tea Talk Today!
by Aaron Courville 05 Mai '11

05 Mai '11

Reminder: As suggested by Yoshua, we're going to have a run up to NIPS tea talk this Thursday. The idea is we give very brief pitches for the various projects underway with an eye to a NIPS submission. Students not on a project can shop around and get involved. Updated list of speakers / subjects: (YOU MIGHT BE ON THE LIST!) - Ian Goodfellow: new estimation criteria for energy based models and a new reconstruction semi-restricted Boltzmann machine - Guillaume Desjardins: tracking the partition function and bilinear RBMs - Xavier Glorot & Yann Dauphin: reconstruction sampling with sparse connections - Xavier Glorot & Antoine Bordes: word sense disambiguation with semantic triplet models - James Bergstra: I'd like to essentially scale up the mu-ssRBM to work on big images using pyramidal image representations. - Remi Bardenet & James Bergstra: adaptive model selection - François Savard: deep auto-encoders with (recurrent) top-down connections for coordinating internal representations - Salah Rifai: semantic hashing with contrastive auto-encoders - David Warde-Farley & Razvan Pascanu: sparse coding for animations compression - Eric Thibodeau-Laufer: hybrid non-parametric & parametric (embeddings-based) content-based collaborative filtering - Razvan Pascanu: long-term dependencies in recurrent networks - Gregoire Mesnil & Antoine Bordes: Learning to automatically label sub-parts of images without any direct supervision - Olivier Delalleau: Theoretical results on the expressive power of deep sum-product networks - Yann & Salah: Modeling manifold tangents with deep networks for better supervised learning - Philippe Hamel : Combining pooling functions in convolutional networks Time / Date: 15h00 / Thursday, May 5th, 2011 Location: LISA Lab (AA3256) Hope to see you there. Cheers, Aaron -- Aaron C. Courville Département d’Informatique et de recherche opérationnelle Université de Montréal email:Aaron.Courville@gmail.com

1 0

TEA TALK: run up to NIPS (NOTE NEW TIME)
by Aaron Courville 03 Mai '11

03 Mai '11

Greetings, As suggested by Yoshua, we're going to have a run up to NIPS tea talk this Thursday. The idea is we give very brief pitches for the various projects underway with an eye to a NIPS submission. Students not on a project can shop around and get involved. List of speakers / subjects: (YOU MIGHT BE ON THE LIST!) - Ian Goodfellow: new estimation criteria for energy based models and a new reconstruction semi-restricted Boltzmann machine - Guillaume Desjardins: tracking the partition function and bilinear RBMs - Xavier Glorot & Yann Dauphin: reconstruction sampling with sparse connections - Xavier Glorot & Antoine Bordes: word sense disambiguation with semantic triplet models - James Bergstra: I'd like to essentially scale up the mu-ssRBM to work on big images using pyramidal image representations. - Remi Bardenet & James Bergstra: adaptive model selection - François Savard: deep auto-encoders with (recurrent) top-down connections for coordinating internal representations - Salah Rifai: semantic hashing with contrastive auto-encoders - David Warde-Farley & Razvan Pascanu: sparse coding for animations compression - Eric Thibodeau-Laufer: hybrid non-parametric & parametric (embeddings-based) content-based collaborative filtering - Razvan Pascanu: long-term dependencies in recurrent networks Time / Date: 15h00 / Thursday, May 5th, 2011 Location: LISA Lab (AA3256) Hope to see you there. Cheers, Aaron -- Aaron C. Courville Département d’Informatique et de recherche opérationnelle Université de Montréal email:Aaron.Courville@gmail.com

2 1

Tea Talk Tomorrow!
by Aaron Courville 31 Mar '11

31 Mar '11

Hi All, This week Fred will tell us about live better through test-driven development. Hope to see you there. Time/Date: 14h30 Thursday March 31st Location: LISA Lab (AA3256) Abstract: I will explain why doing test before or at the same time as coding is something very usefull. In some case, it can save time globally. Very often, you catch bug earlier and this make there fixing easier(bug found in test, not in model). The TDD definition on Wikipedia: Test-driven development (TDD) is a software development process that relies on the repetition of a very short development cycle: first the developer writes a failing automated test case that defines a desired improvement or new function, then produces code to pass that test and finally refactors the new code to acceptable standards. Kent Beck, who is credited with having developed or 'rediscovered' the technique, stated in 2003 that TDD encourages simple designs and inspires confidence.[1] Cheers, Aaron -- Aaron C. Courville Département d’Informatique et de recherche opérationnelle Université de Montréal email:Aaron.Courville@gmail.com

1 0

Reminder: Tea Talk today!
by Aaron Courville 25 Mar '11

25 Mar '11

Hi All, Date: Friday, March 25th Time: 13h30 Location: LISA lab (AA3256) Abstract: La Chaire CRSNG-Ubisoft Yoshua et Olivier D. feront un survol rapide des projets en cours, envisagés à court terme et à long terme (la chaire est subventionnée pour 5 ans) pour la chaire CRSNG-Ubisoft en apprentissage de représentations pour des jeux vidéos immersifs. Les projets à plus court terme concernent le feedback donné à des joueurs néophytes, en se basant sur son comportement vs des techniques de bon jeu suggérées par des designers du jeu. Un des problèmes d'apprentissage impliqué est la prédiction de la dangerosité des situations. Un autre projet actuellement en cours est la compression d'animations. Nous avons aussi deux projets en cours concernant le filtrage collaboratif (recommender des maps ou des gadgets aux joueurs, selon les choix qu'ils ont fait dans le passé et leur profil) d'une part, et le 'match-making' d'autre part (en tenant compte des résultats des matchs et aussi des attributs des joueurs). Finalement, nous sommes en discussion avec Ubisoft, en collaboration avec Polytechnique, concernant l'utilisation des algorithmes d'apprentissage au niveau de la vidéo, à plus long terme. Par exemple Ubisoft serait intéressé à la capture de mouvement (en particulier des expressions du visage) sans marqueurs (en utilisant seulement l'image), pour les acteurs, où à partir d'une caméra 3D, pour les joueurs (par exemple pour détecter les émotions du joueur). Nous essaierons aussi de lancer une discussion sur les défis de recherche fondamentale que ces projets impliquent, en particulier au niveau de l'apprentissage (principalement non-supervisé) de représentations. Cheers, Aaron -- Aaron C. Courville Département d’Informatique et de recherche opérationnelle Université de Montréal email:Aaron.Courville@gmail.com

1 0

Tea Talk Friday 13h30
by Aaron Courville 23 Mar '11

23 Mar '11

Hi All, This week we will have a tea talk at a new day and time. Yoshua and Olivier D. will give an overview of current and future UBISOFT projects. Hope to see you all there. Date: Friday, March 25th Time: 13h30 Location: LISA lab (AA3256) Abstract: La Chaire CRSNG-Ubisoft Yoshua et Olivier D. feront un survol rapide des projets en cours, envisagés à court terme et à long terme (la chaire est subventionnée pour 5 ans) pour la chaire CRSNG-Ubisoft en apprentissage de représentations pour des jeux vidéos immersifs. Les projets à plus court terme concernent le feedback donné à des joueurs néophytes, en se basant sur son comportement vs des techniques de bon jeu suggérées par des designers du jeu. Un des problèmes d'apprentissage impliqué est la prédiction de la dangerosité des situations. Un autre projet actuellement en cours est la compression d'animations. Nous avons aussi deux projets en cours concernant le filtrage collaboratif (recommender des maps ou des gadgets aux joueurs, selon les choix qu'ils ont fait dans le passé et leur profil) d'une part, et le 'match-making' d'autre part (en tenant compte des résultats des matchs et aussi des attributs des joueurs). Finalement, nous sommes en discussion avec Ubisoft, en collaboration avec Polytechnique, concernant l'utilisation des algorithmes d'apprentissage au niveau de la vidéo, à plus long terme. Par exemple Ubisoft serait intéressé à la capture de mouvement (en particulier des expressions du visage) sans marqueurs (en utilisant seulement l'image), pour les acteurs, où à partir d'une caméra 3D, pour les joueurs (par exemple pour détecter les émotions du joueur). Nous essaierons aussi de lancer une discussion sur les défis de recherche fondamentale que ces projets impliquent, en particulier au niveau de l'apprentissage (principalement non-supervisé) de représentations. Cheers, Aaron -- Aaron C. Courville Département d’Informatique et de recherche opérationnelle Université de Montréal email:Aaron.Courville@gmail.com

1 0

Tea Talk Tomorrow -- NOTE: CHANGE OF TIME
by Aaron Courville 09 Mar '11

09 Mar '11

Hi All, Tomorrow we will have another tea talk from the LISA lab members at the core of our UTLC team. Note the change of time! Hope to see you there. Location: LISA lab (AA3256) Time: 13h00 Date: Thursday, March 10th, 2011 Title: Unsupervised and Transfer Learning Challenge (UTLC) Authors: Yann, Xavier, Salah, Grégoire Abstract: The phase 1 of unsupervised learning challenge aimed to test unsupervised learning methods on five different datasets (sparse/dense, high/low dimension, images/manuscripts/... recognition). Competitors were evaluated with respect to the ALC (Area under Learning Curve) of a linear classifier (Hebbian) trained with very few training examples on a subset of the classes present in the training set. This allowed us to test deep/shallow learning algorithms in new settings: one shot learning (one sample used for training) and transductive learning. We will present the different methods we used to achieve good performance (ranking 1st overall on the validation set) and our positive/negative feedback about the competition. Cheers, Aaron -- Aaron C. Courville Département d’Informatique et de recherche opérationnelle Université de Montréal email:Aaron.Courville@gmail.com

1 0

NO tea talk today
by Aaron Courville 03 Mar '11

03 Mar '11

As you probably know or have guessed, there is no Tea Talk this week due to the unsupervised learning challenge (go team!). We should be back on track next week with a tea talk dedicated to the challenge and the lessons we're learned. Cheers, Aaron -- Aaron C. Courville Département d’Informatique et de recherche opérationnelle Université de Montréal email:Aaron.Courville@gmail.com

1 0

Tea Talk Tomorrow!
by Aaron Courville 23 Fév '11

23 Fév '11

Hi Gang, Tomorrow we will have Razvan tell us about stuff he's been working on recently. Date: Thursday. Feb. 24th 2011 Time: 14h30 Location LISA lab (AA3256) Abstract: Recurrent Neural Networks are a perfect framework for modelling complex non-linear temporal information. Unfortunately all gradient based methods for training them suffer fro the "vanishing gradient" problem, which means that RNN can at most discover only short term temporal dependencies in the data. There have been a few attempts to address this problem, the most noteworthy being the Long-Short Term Memory network that solves the task by modifying the structure of the network. We will be looking in a different direction, namely how can this problem be addressed by looking at the optimization algorithm ( Back Propagation Through Time in this case). We propose a regularization term that forces RNN to look back in time and show a few results on synthetic data. Cheers, Aaron -- Aaron C. Courville Département d’Informatique et de recherche opérationnelle Université de Montréal email:Aaron.Courville@gmail.com

1 0

Re: [Lisa_teatalk] Tea Talk Today -- Delayed to 3pm
by Aaron Courville 17 Fév '11

17 Fév '11

Yoshua is in an overtime phone meeting with DARPA people. We will postpone the Tea-talk to 3pm. Cheers, Aaron On Thu, Feb 17, 2011 at 9:59 AM, Aaron Courville <aaron.courville(a)gmail.com> wrote: > We're kicking off our new (and belated) tea talk season with Ilaria > Castelli who will talk about her recent work building and using > massive quantities of Youtube data to learn models of faces. > > Date: Today, Thursday Feb. 17th 2011 > Time: 14h30 > Where: LISA lab (AA3256) > > Abstract: > I'll explain how the youtube dataset has been collected and how it is > organized, in case someone else would like to use it in the future. > I'll then speak about the experiments I have tried with denoising > autoencoders and the olivetti dataset on a pose regression task, and > the issues I encountered when training together with our youtube data. > > Hope to see you there. > Cheers, > Aaron > > > -- > Aaron C. Courville > Département d’Informatique et > de recherche opérationnelle > Université de Montréal > email:Aaron.Courville@gmail.com > -- Aaron C. Courville Département d’Informatique et de recherche opérationnelle Université de Montréal email:Aaron.Courville@gmail.com

1 0

Tea Talk Today!!!
by Aaron Courville 17 Fév '11

17 Fév '11

We're kicking off our new (and belated) tea talk season with Ilaria Castelli who will talk about her recent work building and using massive quantities of Youtube data to learn models of faces. Date: Today, Thursday Feb. 17th 2011 Time: 14h30 Where: LISA lab (AA3256) Abstract: I'll explain how the youtube dataset has been collected and how it is organized, in case someone else would like to use it in the future. I'll then speak about the experiments I have tried with denoising autoencoders and the olivetti dataset on a pose regression task, and the issues I encountered when training together with our youtube data. Hope to see you there. Cheers, Aaron -- Aaron C. Courville Département d’Informatique et de recherche opérationnelle Université de Montréal email:Aaron.Courville@gmail.com

1 0

Tea-Talk tomorrow 14h30
by Yoshua Bengio 16 Fév '11

16 Fév '11

Hi all, Ilaria Castelli is leaving next week and will describe her work here over the last few months, at the next LISA tea-talk tomorrow afternoon, 14h30, room 3256. Ilaria will talk about a dataset she and Pierre-Antoine Manzagol collected off of you tube. She will explain how it has been collected and how it is organized, in case someone else would like to use it in the future. She will then speak about the experiments she has started with denoising autoencoders and the olivetti dataset on a pose regression task, and the issues she encountered when training together with the youtube data. -- Yoshua

1 0

Tea Talk Today!
by Aaron Courville 02 Déc '10

02 Déc '10

Hi Gang, Today we have a special NIPS preview. Lab members have volunteered to give 5 minute spotlight talks on a NIPS2010 talk in which they are interested. Date and Time: Today, Thursday Dec. 2nd, 14h30 Location: LISA lab, AA3256 Speakers and Titles: Ian Tiled Convolutional Neural Networks Olivier Over-complete representations on recurrent neural networks can support persistent percepts Salah Sparse Coding for Learning Interpretable Spatio-Temporal Primitives Dumitru Label Embedding Trees for Large Multi-Class Tasks Grégoire Regularized estimation of image statistics by score matching Pierre-Antoine t-logistic regression Arnaud Learning Convolutional Features Hierarchies for Visual Recognition David Hallucinations in Charles Bonnet Syndrome Induced by Homeostasis: a Deep Boltzmann Machine Model Guillaume Improving the Asymptotic Performance of MCMC by Inserting Vortices OR Short-term memory in neuronal networks through dynamical compressed sensing Xavier G. An analysis on negative curvature induced by singularity in multi-layer neural-network learning (not sure yet) Cheers, Aaron -- Aaron C. Courville Département d’Informatique et de recherche opérationnelle Université de Montréal email:Aaron.Courville@gmail.com

1 0

Tea Talk Tomorrow!
by Aaron Courville 24 Nov '10

24 Nov '10

Hi, Tomorrow we will be treated to a talk by Pascal Vincent on his recent work on the theoretical underpinning of Denoising Autoencoders. Hope to see you there. Time &Date: Thursday Nov. 25th, 14h30 Location: Lisa Lab (AA3256) TITLE: A connection between Score Matching and Denoising Autoencoders ABSTRACT: In this tea talk I will present my recent findings on a connection between two apparently very different techniques: denoising autoencoders and score matching. Denoising autoencoders have been previously shown to be competitive al- ternatives to Restricted Boltzmann Machines for unsupervised pre-training of each layer of a deep architecture. I will show that a simple denoising au- toencoder training criterion is equivalent to matching the score of a speciﬁc energy based model to that of a non-parametric Parzen density estimator of the data. This yields several useful insights. It deﬁnes a proper probabilistic model for the denoising autoencoder technique which makes it in principle possible to sample from them or to rank examples by their energy. It suggests a different way to apply score matching that is related to learning to denoise and does not require computing second derivatives. It justiﬁes the use of tied weights between the encoder and decoder, and suggests ways to extend the success of denoising autoencoders to a larger family of energy-based models. Cheers, Aaron -- Aaron C. Courville Département d’Informatique et de recherche opérationnelle Université de Montréal email:Aaron.Courville@gmail.com

1 0

Tea Talk Tomorrow!
by Aaron Courville 10 Nov '10

10 Nov '10

Hi All, Tomorrow we will hear about recent (actually ongoing) work of Antoine Bordes, jointly with Jason Weston and Ronan Collobert. Hope to see you all there. When: November 11th 2010 @ 14h30 Where: LISA Lab (AA3256) Title and Abstract: How to embed any symbolic knowledge-base into a meaningful vector representation. ----------------- Since the 18th century and the first "Encyclopedie" of Diderot and D'Alembert, one has always wanted to group, organize, and classify knowledge. Recent developments of this trend concern the building of knowledge bases (KB) which are special kinds of relational database for knowledge management, collection, and retrieval. Examples of such KB involve Cyc, Freebase or WordNet. Even though these KB encompass colossal quantities of information, accessing or manipulating it can sometimes be complicated due to the limitations and rigidity of their underlying symbolic framework. In this talk, we present some preliminary work on how to embed these symbolic representations into a more flexible and complete continuous vector space in which the original knowledge is kept and enhanced. We illustrate this method on WordNet and sub-parts of Freebase. Cheers, Aaron -- Aaron C. Courville Département d’Informatique et de recherche opérationnelle Université de Montréal email:Aaron.Courville@gmail.com

1 0

Tea Talk Tomorrow!
by Aaron Courville 03 Nov '10

03 Nov '10

Hi All, We have a tea talk tomorrow by Ian and Yann on their AISTAT submission. Time/Date: 14h30, Thursday Nov. 4th. Location: LISA Lab (AA3256) A Probabilistic Framework for Multi-Instance Learning We present a probabilistic framework for multi-instance learning. Our framework, PMIL, allows a wide variety of supervised learning models to be easily adapted to the multi-instance setting. We demonstrate the effectiveness of our approach on real-world benchmark datasets, achieving state of the art peformance on several TREC tasks. We use controlled experiments with artificially constructed multi-instance problems in order to demonstrate the ability of our approach to learn a good model of individual instances and to empirically analyze its behavior under conditions that violate the assumptions justifying our framework. Cheers, Aaron -- Aaron C. Courville Département d’Informatique et de recherche opérationnelle Université de Montréal email:Aaron.Courville@gmail.com

1 0

Tea Talk tomorrow (possibly)!
by Aaron Courville 14 Oct '10

14 Oct '10

Hi Gang, Tomorrow, if he's feeling up to it, Dumitru Erhan will regale us with stories of working at Google. When: Thursday Oct 14th @ 14h30 (Unless Dumi is still sick) Where: LISA Lab (AA3256) Title: How to model high-resolution images by starting with smaller patches first? Abstract: I'll go over some experiments and explorations of this topic, which I've done during the summer. While I'll present some results, this will be more of an open discussion on how to actually tackle this problem in a large-scale setting. Cheers, Aaron -- Aaron C. Courville Département d’Informatique et de recherche opérationnelle Université de Montréal email:Aaron.Courville@gmail.com

1 0

Fwd: [Lisa_seminaires] UdeM-McGill-MITACS machine learning seminar Thu Oct. 7@14h00, AA-3195
by Aaron Courville 05 Oct '10

05 Oct '10

Hi All, Charles Cadieu is visiting the lab tomorrow and Thursday (see below for details of his talk on Thursday at 14h00). Please let me know when you are *not available* to meet with him over these two days. Also let me know if you would particularly like to meet with him so that I can be sure to fit you into his schedule. Cheers, Aaron ---------- Forwarded message ---------- From: Dumitru Erhan <erhandum(a)iro.umontreal.ca> Date: Fri, Oct 1, 2010 at 10:06 AM Subject: [Lisa_seminaires] UdeM-McGill-MITACS machine learning seminar Thu Oct. 7@14h00, AA-3195 To: lisa_seminaires(a)iro.umontreal.ca The UdeM-McGill-MITACS machine learning seminar series is back with an exciting fall schedule. Next week's seminar (see http://www.iro.umontreal.ca/article.php3?id_article=107&lang=en) Learning Spatial and Transformational Invariants for Visual Representation by Charles Cadieu Redwood Center for Theoretical Neuroscience University of California, Berkeley Location: Pavillon André-Aisenstadt (UdeM), room AA-3195 Time: Thursday, October 7, 14:00 Abstract: Learning abstract, invariant properties of the visual word is a key attribute of biological vision systems. I will describe a hierarchical, probabilistic model that learns to extract invariant spatial structure and invariant motion structure from movies of the natural environment. The first layer in the model produces a sparse decomposition of local edge and motion structure. This decomposition is achieved through a complex-valued sparse coding model in which amplitudes represent the presence of edge structure at specific positions, orientations, and spatial scales, and phases represent the precise positions of edges and how image structure changes through time. This decomposition into amplitude and phase exposes statistical dependencies that the top layer in the model captures as two types of invariances: spatial and transformational. The spatial invariants are a sparse code of the patterns in the first layer amplitude components. They code a rich set of multi-scale edges, textures, and texture defined edge boundaries. The transformational invariants are a sparse representation of patterns of change in the first layer phase components. They learn a multi-scale code of motion, spanning local and global motions, and are capable of learning complex motions such as zooming, rotation, and deformation. Besides extracting abstract, invariant properties of the visual world, I will show how the hierarchical model provides a concrete model of cortical feedback that is useful for perception under noisy or ambiguous conditions. _______________________________________________ Lisa_seminaires mailing list Lisa_seminaires(a)mercure.iro.umontreal.ca https://webmail.iro.umontreal.ca/mailman/listinfo/lisa_seminaires -- Aaron C. Courville Département d’Informatique et de recherche opérationnelle Université de Montréal email:Aaron.Courville@gmail.com

1 0

Tea Talk Tomorrow!
by Aaron Courville 29 Sep '10

29 Sep '10

Hi All, This week we have James Bergstra giving his invited talk a test drive. Hope to see you all there. Time: Thursday Sept. 30th 14h30 Location: AA3195 Title: Complex Cells in Neural Networks Abstract: This is a practice run for a talk I'm giving next week at the Rowland Institute in Boston. It's a survey of what I've done and where I'm going with the 'quadratic units'. I'll go over published work from the last two years (better generalization in images, pretraining with slow features, high-throughput analysis of the Canonical Circuit) and also talk about ongoing work related to random search, and energy-based training of complex-cell-like models. -- http://www-etud.iro.umontreal.ca/~bergstrj Cheers, Aaron -- Aaron C. Courville Département d’Informatique et de recherche opérationnelle Université de Montréal email:Aaron.Courville@gmail.com

1 0

Tea Talk Today!
by Aaron Courville 23 Sep '10

23 Sep '10

Hi All, Today we will hear from Olivier Delalleau. He'll be talking about his recent work with Ubisoft. Hope to see you there. Date and Time: Thursday September 23, 2010 (Today) at 14h30 Location: LISA lab (AA3195) Title: Player Modeling in a First Person Shooter: Bridging the Gap between Casual and Power Gamers Abstract: Player-vs-Player (PvP) in multiplayer online games typically appeals mostly to the more "hardcore" gaming audience. This is because reaching a skill level that makes you competitive vs. other dedicated gamers can require a lot of practice. More casual players who may not have a lot of previous online experience thus often give up on PvP, focusing instead on cooperative or solo gameplay. We present a player modeling approach for a First Person Shooter game where the goal is to provide the player with feedback on how to improve his playstyle. The objective is to make online play more "casual-friendly" by providing sensible tips and encouraging the player into not throwing the game out of the window. Other applications of this approach include analyzing player behavior for better level design, as well as for better categorization and understanding of the various playstyles. This ongoing research at Ubisoft has been performed in collaboration with Yoshua Bengio, Philippe Hamel, Guillaume Desjardins, Pierre-Antoine Manzagol, François Savard and Yann Dauphin, and a bunch of people from Ubisoft whose names are super-top-secret. Cheers, Aaron -- Aaron C. Courville Département d’Informatique et de recherche opérationnelle Université de Montréal email:Aaron.Courville@gmail.com

1 0

Tea Talk Tomorrow!
by Aaron Courville 15 Sep '10

15 Sep '10

Hi All, This week we return to our normal tea talk schedule with Antoine Bordes, who will tell us about some fascinating work he did before joining LISA. Title and abstract below. When: Thursday Sept. 16th, 2:30pm Where: LISA lab (AA3256) Hope to see you all there. Towards Understanding Situated Natural Language -- Joint work with J. Weston (Google), N. Usunier (Paris 6) and R. Collobert (NEC) -- We present a general framework and learning algorithm for the task of concept labeling: each word in a given sentence has to be tagged with the unique physical entity (e.g. person, object or location) or abstract concept it refers to. Our method allows both world knowledge and linguistic information to be used during learning and prediction. We show experimentally that we can learn to use world knowledge to resolve ambiguities in language, such as word senses or reference resolution, without the use of hand-crafted rules or features. Paper and data available from http://www-etud.iro.umontreal.ca/~bordesa/mywiki/doku.php?id=abstract_babi_… . Cheers, Aaron -- Aaron C. Courville Département d’Informatique et de recherche opérationnelle Université de Montréal email:Aaron.Courville@gmail.com

1 0

Tea Talk Thursday: Pylearning part 2
by Aaron Courville 07 Sep '10

07 Sep '10

Hi All, This week, once again, we will us our tea talk time to meet and further our discussion of our ongoing reorganization of our code and code building practices. When, Thursday Sept. 9th, 2pm Where LISA Lab. It would be helpful if all the committees have met at least once before the meeting. More guidance for the committees (PLEASE READ THE FOLLOWING): Yoshua: Here is how I would define the job of the committees that deal with software modules (most of them except to upcoming social engineering committee): * The end-product of the committee is both an API and its justification. * It's justification is grounded on - algorithms currently used in the lab and that can fit in one mold or in a shared view of what they view, in the sense of being interchangeable as seen from the outside world - forward looking perspective on other algorithms that we might want to fit in this mold in the future - computational issues: * think through how these might be typically implemented, and consider things like - internal state, persistent storage (what happens when a machine crashes), - parallelization (especially of the kind we currently do, i.e., for hyper-optimization) and asynchronous functionality - compatibility issues: * what if Theano were to change in the future? try to avoid dependency on Theano where possible, and where not try to depend only on the abstraction that we are manupilating symbolic expressions and keep very Theano-specific commands like compilation in one place * compatibility with other python packages, either for data, learning algorithms, visualization, etc. - factorization objectives (e.g. dependency on the filesystem, cluster type, GPU vs CPU, database type, parallelization ,etc. should be as much as limited to the smallest subset of the code as possible) * It may be that the family of algorithms considered don't all fit in one mold. That's ok. We need to know why and maybe there are sub-families that can each be grouped in different molds, that is ok too. So we could have different APIs, one for each of these sub-families. Or we could come to the conclusion that there is no common mold at all and the only sharing could be at the level of 'formulas' (this might be the case for inference). Cheers, Aaron -- Aaron C. Courville Département d’Informatique et de recherche opérationnelle Université de Montréal email:Aaron.Courville@gmail.com

1 0

Tea Talk tomorrow!
by Aaron Courville 01 Sep '10

01 Sep '10

Hi All, This week the tea talk will take the form of a lab-wide discussion of our future ... code-wise. Hope to see you there. When: Thursday Sept. 2nd, 14h30 - 15h30 (Tomorrow) Where: LISA Lab. Title/Abstract: Brainstorming on the future of shared machine learning code on top of Theano The objective is to hear every one that has some opinion on this subject, mostly people who have designed or thought about learning frameworks on top of Theano, or people who know about other machine learning libraries from which we could learn something. There are those that lean towards a well-organized *library* defining functional types (such as types of datasets, of learners, of optimizers, of performance statistics, etc.) and those that believe that a more organic sharing of pieces of code would work better. But if we are going to share code and make it easy to REUSE pieces already written by others, we have to start agreeing on conventions and protocols, at least, for the various kinds of pieces one may find in this repository, which would become the successor of Pylearn. Cheers, Aaron -- Aaron C. Courville Département d’Informatique et de recherche opérationnelle Université de Montréal email:Aaron.Courville@gmail.com

1 0

how brains learn to see
by Yoshua Bengio 28 Aoû '10

28 Aoû '10

Very interesting TED talk: http://www.ted.com/talks/pawan_sinha_on_how_brains_learn_to_see.html The supposed visual critical period is not that bad... Motion is required to learn to segment objects... -- Yoshua

1 0

Tea Talk Today! (NOTE: new time and place)
by Aaron Courville 26 Aoû '10

26 Aoû '10

Hi All, This week Lysiane Bouchard will be telling us about her recent work on machine learning applications to neuroimaging data. Where: AA1409 (tentative) When: Thursday August 26th, 14h30 Title/Abstract: Machine learning applications to neuroimaging data Neuroimaging techniques allow measurements at more and more reﬁned resolution through various imaging modalities. These technical advancements create a need for more and more sophisticated analysis tools and are certainly related to the growing interest for machine learning algorithms in the neuroscience community. The goal of this talk is to give an overview of machine learning applications to neuroimaging data. First, I will introduce you to neuroimaging data, more specifically to Magnetic Resonance Imaging (MRI) data. Second, I will present several examples of machine learning applications, including a study on auditory perception on which I’ve been working recently. Finally, I will discuss about a few popular research trends aiming to integrate a-priori knowledge and traditional preprocessing steps in the algorithms to increase their interpretability and efficiency. Cheers, Aaron -- Aaron C. Courville Département d’Informatique et de recherche opérationnelle Université de Montréal email:Aaron.Courville@gmail.com -- Aaron C. Courville Département d’Informatique et de recherche opérationnelle Université de Montréal email:Aaron.Courville@gmail.com

1 0

Tea Talk Tomorrow!
by Aaron Courville 19 Aoû '10

19 Aoû '10

Greeting, This week we will have a talk by Nicholas Morsillo, from the University of Rochester (working with Chris Pal) on his recent explorations of the Deep. Hope to see you all there. There will be snacks. Date and Time: Thursday August 19th, 15h00 Location: LISA lab (AA3256) Title and abstract: Extending Convolutional Deep Belief Models for Improved Task-Specific Feature Learning In this talk I will present a number of topics involved in my current work with convolutional deep belief models for visual feature construction. I will discuss issues related to implementation, visualization, and data preprocessing, and I will introduce a novel semi-supervised extension to the standard layerwise training schedule. Preliminary results favor careful preprocessing and semi-supervised training. Questions, comments, and discussion are welcome. Cheers, Aaron -- Aaron C. Courville Département d’Informatique et de recherche opérationnelle Université de Montréal email:Aaron.Courville@gmail.com

1 0