[Lisa_teatalk] [Lisa_labo] Tea Talk Tomorrow (NOTE: new location)

Çağlar Gülçehre ca9lar at gmail.com
Thu Mar 1 16:46:26 EST 2012


If you are interested in exchanging ideas or have questions, you can
ask/talk with me as well.


2012/2/29 Yoshua Bengio <yoshua.bengio at gmail.com>:
>
> Note the unusual location!
>
> If you participate in the pylearn2 code sprint, please take a refreshing break at 3pm...
>
> Also, for those interested, I am writing a paper on this subject. The draft is in
> articles/2012/culture_vs_localmin/ (and I attach a pdf of the latest version
> for your convenience) and I am supposed to submit it this week to
> as a book chapter, so any comment is welcome. Note also that this
> is meant to be the starting point for Caglar's thesis (who joined us
> this term for a PhD).
>
> -- Yoshua
>
>
>
> On 2012-02-29, at 15:11, Aaron Courville wrote:
>
>> After the ICML rush, we are restarting our tea talk series with some
>> deep thoughts from Yoshua. See you there.
>>
>> When: 15h00 Thursday, March 1st. 2012
>> Where: AA1355 (NEW LOCATION)
>>
>> Title/Abstract:
>>
>> Evolving Culture vs Local Minima
>> Yoshua Bengio
>>
>> We propose a theory that relates difficulty of learning in deep
>> architectures to culture and language. It is articulated around the
>> following hypotheses:
>>
>> (1) learning in an individual  human brain is hampered by the presence
>> of effective local minima;
>> (2) this optimization difficulty is particularly important when it
>> comes to learning higher-level abstractions, i.e., concept that cover
>> a vast and highly-nonlinear span of sensory configurations;
>> (3) such high-level abstractions are best represented in brains by the
>> composition of many levels of representation, i.e., by deep
>> architectures;
>> (4) a human brain can learn such high-level abstractions if guided by
>> the signals produced by other humans, which act as hints or indirect
>> supervision for these high-level abstractions; and
>> (5), language and the combination of old ideas into new ideas provide
>> an efficient evolutionary recombination operator, and this allows
>> rapid search in the space of communicable ideas that help humans build
>> up better high-level internal representations of their world.
>>
>> These hypotheses put together imply that human culture and the
>> evolution of ideas have been crucial to counter an optimization
>> difficulty: this optimization difficulty would otherwise make it very
>> difficult for human brains to capture high-level knowledge of the
>> world.  The theory is grounded in experimental observations of the
>> difficulties of training deep artificial neural networks. Plausible
>> consequences of this theory for the efficiency of cultural evolutions
>> are sketched.
>>
>>
>> Cheers,
>> Aaron
>>
>> --
>> Aaron C. Courville
>> Département d’Informatique et
>> de recherche opérationnelle
>> Université de Montréal
>> email:Aaron.Courville at gmail.com
>> _______________________________________________
>> Lisa_labo mailing list
>> Lisa_labo at iro.umontreal.ca
>> https://webmail.iro.umontreal.ca/mailman/listinfo/lisa_labo
>
>
> _______________________________________________
> Lisa_labo mailing list
> Lisa_labo at iro.umontreal.ca
> https://webmail.iro.umontreal.ca/mailman/listinfo/lisa_labo
>


-- 
Caglar GULCEHRE


More information about the Lisa_teatalk mailing list