- Lisa_teatalk - mailman.iro.umontreal.ca

Tea Talk Today!
by Aaron Courville 02 Déc '10

02 Déc '10

Hi Gang, Today we have a special NIPS preview. Lab members have volunteered to give 5 minute spotlight talks on a NIPS2010 talk in which they are interested. Date and Time: Today, Thursday Dec. 2nd, 14h30 Location: LISA lab, AA3256 Speakers and Titles: Ian Tiled Convolutional Neural Networks Olivier Over-complete representations on recurrent neural networks can support persistent percepts Salah Sparse Coding for Learning Interpretable Spatio-Temporal Primitives Dumitru Label Embedding Trees for Large Multi-Class Tasks Grégoire Regularized estimation of image statistics by score matching Pierre-Antoine t-logistic regression Arnaud Learning Convolutional Features Hierarchies for Visual Recognition David Hallucinations in Charles Bonnet Syndrome Induced by Homeostasis: a Deep Boltzmann Machine Model Guillaume Improving the Asymptotic Performance of MCMC by Inserting Vortices OR Short-term memory in neuronal networks through dynamical compressed sensing Xavier G. An analysis on negative curvature induced by singularity in multi-layer neural-network learning (not sure yet) Cheers, Aaron -- Aaron C. Courville Département d’Informatique et de recherche opérationnelle Université de Montréal email:Aaron.Courville@gmail.com

1 0

Tea Talk Tomorrow!
by Aaron Courville 25 Nov '10

25 Nov '10

Hi, Tomorrow we will be treated to a talk by Pascal Vincent on his recent work on the theoretical underpinning of Denoising Autoencoders. Hope to see you there. Time &Date: Thursday Nov. 25th, 14h30 Location: Lisa Lab (AA3256) TITLE: A connection between Score Matching and Denoising Autoencoders ABSTRACT: In this tea talk I will present my recent findings on a connection between two apparently very different techniques: denoising autoencoders and score matching. Denoising autoencoders have been previously shown to be competitive al- ternatives to Restricted Boltzmann Machines for unsupervised pre-training of each layer of a deep architecture. I will show that a simple denoising au- toencoder training criterion is equivalent to matching the score of a speciﬁc energy based model to that of a non-parametric Parzen density estimator of the data. This yields several useful insights. It deﬁnes a proper probabilistic model for the denoising autoencoder technique which makes it in principle possible to sample from them or to rank examples by their energy. It suggests a different way to apply score matching that is related to learning to denoise and does not require computing second derivatives. It justiﬁes the use of tied weights between the encoder and decoder, and suggests ways to extend the success of denoising autoencoders to a larger family of energy-based models. Cheers, Aaron -- Aaron C. Courville Département d’Informatique et de recherche opérationnelle Université de Montréal email:Aaron.Courville@gmail.com

1 0

Tea Talk Tomorrow!
by Aaron Courville 11 Nov '10

11 Nov '10

Hi All, Tomorrow we will hear about recent (actually ongoing) work of Antoine Bordes, jointly with Jason Weston and Ronan Collobert. Hope to see you all there. When: November 11th 2010 @ 14h30 Where: LISA Lab (AA3256) Title and Abstract: How to embed any symbolic knowledge-base into a meaningful vector representation. ----------------- Since the 18th century and the first "Encyclopedie" of Diderot and D'Alembert, one has always wanted to group, organize, and classify knowledge. Recent developments of this trend concern the building of knowledge bases (KB) which are special kinds of relational database for knowledge management, collection, and retrieval. Examples of such KB involve Cyc, Freebase or WordNet. Even though these KB encompass colossal quantities of information, accessing or manipulating it can sometimes be complicated due to the limitations and rigidity of their underlying symbolic framework. In this talk, we present some preliminary work on how to embed these symbolic representations into a more flexible and complete continuous vector space in which the original knowledge is kept and enhanced. We illustrate this method on WordNet and sub-parts of Freebase. Cheers, Aaron -- Aaron C. Courville Département d’Informatique et de recherche opérationnelle Université de Montréal email:Aaron.Courville@gmail.com

1 0

Tea Talk Tomorrow!
by Aaron Courville 03 Nov '10

03 Nov '10

Hi All, We have a tea talk tomorrow by Ian and Yann on their AISTAT submission. Time/Date: 14h30, Thursday Nov. 4th. Location: LISA Lab (AA3256) A Probabilistic Framework for Multi-Instance Learning We present a probabilistic framework for multi-instance learning. Our framework, PMIL, allows a wide variety of supervised learning models to be easily adapted to the multi-instance setting. We demonstrate the effectiveness of our approach on real-world benchmark datasets, achieving state of the art peformance on several TREC tasks. We use controlled experiments with artificially constructed multi-instance problems in order to demonstrate the ability of our approach to learn a good model of individual instances and to empirically analyze its behavior under conditions that violate the assumptions justifying our framework. Cheers, Aaron -- Aaron C. Courville Département d’Informatique et de recherche opérationnelle Université de Montréal email:Aaron.Courville@gmail.com

1 0

Tea Talk tomorrow (possibly)!
by Aaron Courville 14 Oct '10

14 Oct '10

Hi Gang, Tomorrow, if he's feeling up to it, Dumitru Erhan will regale us with stories of working at Google. When: Thursday Oct 14th @ 14h30 (Unless Dumi is still sick) Where: LISA Lab (AA3256) Title: How to model high-resolution images by starting with smaller patches first? Abstract: I'll go over some experiments and explorations of this topic, which I've done during the summer. While I'll present some results, this will be more of an open discussion on how to actually tackle this problem in a large-scale setting. Cheers, Aaron -- Aaron C. Courville Département d’Informatique et de recherche opérationnelle Université de Montréal email:Aaron.Courville@gmail.com

1 0

Fwd: [Lisa_seminaires] UdeM-McGill-MITACS machine learning seminar Thu Oct. 7@14h00, AA-3195
by Aaron Courville 06 Oct '10

06 Oct '10

Hi All, Charles Cadieu is visiting the lab tomorrow and Thursday (see below for details of his talk on Thursday at 14h00). Please let me know when you are *not available* to meet with him over these two days. Also let me know if you would particularly like to meet with him so that I can be sure to fit you into his schedule. Cheers, Aaron ---------- Forwarded message ---------- From: Dumitru Erhan <erhandum(a)iro.umontreal.ca> Date: Fri, Oct 1, 2010 at 10:06 AM Subject: [Lisa_seminaires] UdeM-McGill-MITACS machine learning seminar Thu Oct. 7@14h00, AA-3195 To: lisa_seminaires(a)iro.umontreal.ca The UdeM-McGill-MITACS machine learning seminar series is back with an exciting fall schedule. Next week's seminar (see http://www.iro.umontreal.ca/article.php3?id_article=107&lang=en) Learning Spatial and Transformational Invariants for Visual Representation by Charles Cadieu Redwood Center for Theoretical Neuroscience University of California, Berkeley Location: Pavillon André-Aisenstadt (UdeM), room AA-3195 Time: Thursday, October 7, 14:00 Abstract: Learning abstract, invariant properties of the visual word is a key attribute of biological vision systems. I will describe a hierarchical, probabilistic model that learns to extract invariant spatial structure and invariant motion structure from movies of the natural environment. The first layer in the model produces a sparse decomposition of local edge and motion structure. This decomposition is achieved through a complex-valued sparse coding model in which amplitudes represent the presence of edge structure at specific positions, orientations, and spatial scales, and phases represent the precise positions of edges and how image structure changes through time. This decomposition into amplitude and phase exposes statistical dependencies that the top layer in the model captures as two types of invariances: spatial and transformational. The spatial invariants are a sparse code of the patterns in the first layer amplitude components. They code a rich set of multi-scale edges, textures, and texture defined edge boundaries. The transformational invariants are a sparse representation of patterns of change in the first layer phase components. They learn a multi-scale code of motion, spanning local and global motions, and are capable of learning complex motions such as zooming, rotation, and deformation. Besides extracting abstract, invariant properties of the visual world, I will show how the hierarchical model provides a concrete model of cortical feedback that is useful for perception under noisy or ambiguous conditions. _______________________________________________ Lisa_seminaires mailing list Lisa_seminaires(a)mercure.iro.umontreal.ca https://webmail.iro.umontreal.ca/mailman/listinfo/lisa_seminaires -- Aaron C. Courville Département d’Informatique et de recherche opérationnelle Université de Montréal email:Aaron.Courville@gmail.com

1 0

Tea Talk Tomorrow!
by Aaron Courville 30 Sep '10

30 Sep '10

Hi All, This week we have James Bergstra giving his invited talk a test drive. Hope to see you all there. Time: Thursday Sept. 30th 14h30 Location: AA3195 Title: Complex Cells in Neural Networks Abstract: This is a practice run for a talk I'm giving next week at the Rowland Institute in Boston. It's a survey of what I've done and where I'm going with the 'quadratic units'. I'll go over published work from the last two years (better generalization in images, pretraining with slow features, high-throughput analysis of the Canonical Circuit) and also talk about ongoing work related to random search, and energy-based training of complex-cell-like models. -- http://www-etud.iro.umontreal.ca/~bergstrj Cheers, Aaron -- Aaron C. Courville Département d’Informatique et de recherche opérationnelle Université de Montréal email:Aaron.Courville@gmail.com

1 0

Tea Talk Today!
by Aaron Courville 23 Sep '10

23 Sep '10

Hi All, Today we will hear from Olivier Delalleau. He'll be talking about his recent work with Ubisoft. Hope to see you there. Date and Time: Thursday September 23, 2010 (Today) at 14h30 Location: LISA lab (AA3195) Title: Player Modeling in a First Person Shooter: Bridging the Gap between Casual and Power Gamers Abstract: Player-vs-Player (PvP) in multiplayer online games typically appeals mostly to the more "hardcore" gaming audience. This is because reaching a skill level that makes you competitive vs. other dedicated gamers can require a lot of practice. More casual players who may not have a lot of previous online experience thus often give up on PvP, focusing instead on cooperative or solo gameplay. We present a player modeling approach for a First Person Shooter game where the goal is to provide the player with feedback on how to improve his playstyle. The objective is to make online play more "casual-friendly" by providing sensible tips and encouraging the player into not throwing the game out of the window. Other applications of this approach include analyzing player behavior for better level design, as well as for better categorization and understanding of the various playstyles. This ongoing research at Ubisoft has been performed in collaboration with Yoshua Bengio, Philippe Hamel, Guillaume Desjardins, Pierre-Antoine Manzagol, François Savard and Yann Dauphin, and a bunch of people from Ubisoft whose names are super-top-secret. Cheers, Aaron -- Aaron C. Courville Département d’Informatique et de recherche opérationnelle Université de Montréal email:Aaron.Courville@gmail.com

1 0

Tea Talk Tomorrow!
by Aaron Courville 15 Sep '10

15 Sep '10

Hi All, This week we return to our normal tea talk schedule with Antoine Bordes, who will tell us about some fascinating work he did before joining LISA. Title and abstract below. When: Thursday Sept. 16th, 2:30pm Where: LISA lab (AA3256) Hope to see you all there. Towards Understanding Situated Natural Language -- Joint work with J. Weston (Google), N. Usunier (Paris 6) and R. Collobert (NEC) -- We present a general framework and learning algorithm for the task of concept labeling: each word in a given sentence has to be tagged with the unique physical entity (e.g. person, object or location) or abstract concept it refers to. Our method allows both world knowledge and linguistic information to be used during learning and prediction. We show experimentally that we can learn to use world knowledge to resolve ambiguities in language, such as word senses or reference resolution, without the use of hand-crafted rules or features. Paper and data available from http://www-etud.iro.umontreal.ca/~bordesa/mywiki/doku.php?id=abstract_babi_… . Cheers, Aaron -- Aaron C. Courville Département d’Informatique et de recherche opérationnelle Université de Montréal email:Aaron.Courville@gmail.com

1 0

Tea Talk Thursday: Pylearning part 2
by Aaron Courville 08 Sep '10

08 Sep '10

Hi All, This week, once again, we will us our tea talk time to meet and further our discussion of our ongoing reorganization of our code and code building practices. When, Thursday Sept. 9th, 2pm Where LISA Lab. It would be helpful if all the committees have met at least once before the meeting. More guidance for the committees (PLEASE READ THE FOLLOWING): Yoshua: Here is how I would define the job of the committees that deal with software modules (most of them except to upcoming social engineering committee): * The end-product of the committee is both an API and its justification. * It's justification is grounded on - algorithms currently used in the lab and that can fit in one mold or in a shared view of what they view, in the sense of being interchangeable as seen from the outside world - forward looking perspective on other algorithms that we might want to fit in this mold in the future - computational issues: * think through how these might be typically implemented, and consider things like - internal state, persistent storage (what happens when a machine crashes), - parallelization (especially of the kind we currently do, i.e., for hyper-optimization) and asynchronous functionality - compatibility issues: * what if Theano were to change in the future? try to avoid dependency on Theano where possible, and where not try to depend only on the abstraction that we are manupilating symbolic expressions and keep very Theano-specific commands like compilation in one place * compatibility with other python packages, either for data, learning algorithms, visualization, etc. - factorization objectives (e.g. dependency on the filesystem, cluster type, GPU vs CPU, database type, parallelization ,etc. should be as much as limited to the smallest subset of the code as possible) * It may be that the family of algorithms considered don't all fit in one mold. That's ok. We need to know why and maybe there are sub-families that can each be grouped in different molds, that is ok too. So we could have different APIs, one for each of these sub-families. Or we could come to the conclusion that there is no common mold at all and the only sharing could be at the level of 'formulas' (this might be the case for inference). Cheers, Aaron -- Aaron C. Courville Département d’Informatique et de recherche opérationnelle Université de Montréal email:Aaron.Courville@gmail.com

1 0