Mars 2012 - Lisa_teatalk - mailman.iro.umontreal.ca

Tea Talk Tomorrow!
by Aaron Courville 14 Mar '12

14 Mar '12

Hi All, Our regularly scheduled tea talk this week with be given by Nicolas Boulanger-Lewandowski. Nicolas will tell about some of his recent work. Hope to see you all there. When: 15h00, Thursday March 15th, 2012 Where: LISA lab (AA3256) Title: Discriminative non-negative matrix factorization for polyphonic music transcription Non-negative matrix factorization (NMF) is an unsupervised technique to decompose data into additive-only components. It is useful to infer interpretable recurring note events in a power spectrogram, or facial features in emotion recognition data. After reviewing recently developed models for polyphonic music generation, I will use NMF to transcribe audio excerpts into musical notation. I will introduce two discriminative criteria, both in the latent variable and autoencoder perspectives, to influence the NMF decomposition and improve the resulting transcription. Cheers, Aaron -- Aaron C. Courville Département d’Informatique et de recherche opérationnelle Université de Montréal email:Aaron.Courville@gmail.com

1 0

Tea Talk Tomorrow!
by Aaron Courville 11 Mar '12

11 Mar '12

Hi, Tomorrow we have a special Tea Talk at a special Time. A visitor, Pardis Noorzad, will present some of his work with Prof. Bob L. Sturm. Hope to see you there. When: Tomorrow, Monday March 12th, 2012 Where: TBA "SPARROW: Sparse approximation weighted regression" Abstract: We propose sparse approximation weighted regression (SPARROW), which is a nonparametric method of regression that takes advantage of the sparse linear approximation of a query point. SPARROW employs weights based on sparse approximation in the context of locally constant, locally linear, and locally quadratic regression to generate better estimates than for e.g., k-nearest neighbor regression and more generally, kernel-weighted local polynomial regression. Our experimental results show that SPARROW performs competitively. This is joint work with Prof. Bob L. Sturm. Cheers, Aaron -- Aaron C. Courville Département d’Informatique et de recherche opérationnelle Université de Montréal email:Aaron.Courville@gmail.com

1 1

Tea Talk Tomorrow!
by Aaron Courville 07 Mar '12

07 Mar '12

Greetings, This week Razvan Pascanu will tell us about some of his recent work on optimization methods. Hope to see you there. When: 15h00 Tomorrow (Thursday March 8th 2012) Where: LIDA Lab, (AA3256) Title : Yet Another Optimization Technique for Machine Learning ? Abstract: I've been trying to understand what Hessian Free or related methods are doing, how and why they can claim to be getting better generalization error. Why they work for RNNs better then SGD in certain cases? I don't have all the answers, but I found a set of misnomers IMHO, and interesting connections between natural gradient and recently proposed methods. I want to provide a different interpretation of how fancier optimization should be done for complex non-linear models, and I want in the end to outline how one would go about improving proposed algorithms. Cheers, Aaron -- Aaron C. Courville Département d’Informatique et de recherche opérationnelle Université de Montréal email:Aaron.Courville@gmail.com

1 0

Tea Talk Tomorrow (NOTE: new location)
by Aaron Courville 01 Mar '12

01 Mar '12

After the ICML rush, we are restarting our tea talk series with some deep thoughts from Yoshua. See you there. When: 15h00 Thursday, March 1st. 2012 Where: AA1355 (NEW LOCATION) Title/Abstract: Evolving Culture vs Local Minima Yoshua Bengio We propose a theory that relates difficulty of learning in deep architectures to culture and language. It is articulated around the following hypotheses: (1) learning in an individual human brain is hampered by the presence of effective local minima; (2) this optimization difficulty is particularly important when it comes to learning higher-level abstractions, i.e., concept that cover a vast and highly-nonlinear span of sensory configurations; (3) such high-level abstractions are best represented in brains by the composition of many levels of representation, i.e., by deep architectures; (4) a human brain can learn such high-level abstractions if guided by the signals produced by other humans, which act as hints or indirect supervision for these high-level abstractions; and (5), language and the combination of old ideas into new ideas provide an efficient evolutionary recombination operator, and this allows rapid search in the space of communicable ideas that help humans build up better high-level internal representations of their world. These hypotheses put together imply that human culture and the evolution of ideas have been crucial to counter an optimization difficulty: this optimization difficulty would otherwise make it very difficult for human brains to capture high-level knowledge of the world. The theory is grounded in experimental observations of the difficulties of training deep artificial neural networks. Plausible consequences of this theory for the efficiency of cultural evolutions are sketched. Cheers, Aaron -- Aaron C. Courville Département d’Informatique et de recherche opérationnelle Université de Montréal email:Aaron.Courville@gmail.com

3 2

Re: [Lisa_teatalk] [Lisa_labo] Tea Talk Tomorrow (NOTE: new location)
by Yoshua Bengio 29 Fév '12

29 Fév '12

Note the unusual location! If you participate in the pylearn2 code sprint, please take a refreshing break at 3pm... Also, for those interested, I am writing a paper on this subject. The draft is in articles/2012/culture_vs_localmin/ (and I attach a pdf of the latest version for your convenience) and I am supposed to submit it this week to as a book chapter, so any comment is welcome. Note also that this is meant to be the starting point for Caglar's thesis (who joined us this term for a PhD). -- Yoshua On 2012-02-29, at 15:11, Aaron Courville wrote: > After the ICML rush, we are restarting our tea talk series with some > deep thoughts from Yoshua. See you there. > > When: 15h00 Thursday, March 1st. 2012 > Where: AA1355 (NEW LOCATION) > > Title/Abstract: > > Evolving Culture vs Local Minima > Yoshua Bengio > > We propose a theory that relates difficulty of learning in deep > architectures to culture and language. It is articulated around the > following hypotheses: > > (1) learning in an individual human brain is hampered by the presence > of effective local minima; > (2) this optimization difficulty is particularly important when it > comes to learning higher-level abstractions, i.e., concept that cover > a vast and highly-nonlinear span of sensory configurations; > (3) such high-level abstractions are best represented in brains by the > composition of many levels of representation, i.e., by deep > architectures; > (4) a human brain can learn such high-level abstractions if guided by > the signals produced by other humans, which act as hints or indirect > supervision for these high-level abstractions; and > (5), language and the combination of old ideas into new ideas provide > an efficient evolutionary recombination operator, and this allows > rapid search in the space of communicable ideas that help humans build > up better high-level internal representations of their world. > > These hypotheses put together imply that human culture and the > evolution of ideas have been crucial to counter an optimization > difficulty: this optimization difficulty would otherwise make it very > difficult for human brains to capture high-level knowledge of the > world. The theory is grounded in experimental observations of the > difficulties of training deep artificial neural networks. Plausible > consequences of this theory for the efficiency of cultural evolutions > are sketched. > > > Cheers, > Aaron > > -- > Aaron C. Courville > Département d’Informatique et > de recherche opérationnelle > Université de Montréal > email:Aaron.Courville@gmail.com > _______________________________________________ > Lisa_labo mailing list > Lisa_labo(a)iro.umontreal.ca > https://webmail.iro.umontreal.ca/mailman/listinfo/lisa_labo

1 0