[Lisa_teatalk] [Lisa_labo] Tea Talk Tomorrow (NOTE: new location)

Yoshua Bengio bengioy at iro.umontreal.ca
Wed Feb 29 21:09:40 EST 2012


Note the unusual location! 

If you participate in the pylearn2 code sprint, please take a refreshing break at 3pm...

Also, for those interested, I am writing a paper on this subject. The draft is in
articles/2012/culture_vs_localmin/ (and I attach a pdf of the latest version
for your convenience) and I am supposed to submit it this week to
as a book chapter, so any comment is welcome. Note also that this
is meant to be the starting point for Caglar's thesis (who joined us
this term for a PhD).

-- Yoshua


On 2012-02-29, at 15:11, Aaron Courville wrote:

> After the ICML rush, we are restarting our tea talk series with some
> deep thoughts from Yoshua. See you there.
> 
> When: 15h00 Thursday, March 1st. 2012
> Where: AA1355 (NEW LOCATION)
> 
> Title/Abstract:
> 
> Evolving Culture vs Local Minima
> Yoshua Bengio
> 
> We propose a theory that relates difficulty of learning in deep
> architectures to culture and language. It is articulated around the
> following hypotheses:
> 
> (1) learning in an individual  human brain is hampered by the presence
> of effective local minima;
> (2) this optimization difficulty is particularly important when it
> comes to learning higher-level abstractions, i.e., concept that cover
> a vast and highly-nonlinear span of sensory configurations;
> (3) such high-level abstractions are best represented in brains by the
> composition of many levels of representation, i.e., by deep
> architectures;
> (4) a human brain can learn such high-level abstractions if guided by
> the signals produced by other humans, which act as hints or indirect
> supervision for these high-level abstractions; and
> (5), language and the combination of old ideas into new ideas provide
> an efficient evolutionary recombination operator, and this allows
> rapid search in the space of communicable ideas that help humans build
> up better high-level internal representations of their world.
> 
> These hypotheses put together imply that human culture and the
> evolution of ideas have been crucial to counter an optimization
> difficulty: this optimization difficulty would otherwise make it very
> difficult for human brains to capture high-level knowledge of the
> world.  The theory is grounded in experimental observations of the
> difficulties of training deep artificial neural networks. Plausible
> consequences of this theory for the efficiency of cultural evolutions
> are sketched.
> 
> 
> Cheers,
> Aaron
> 
> -- 
> Aaron C. Courville
> Département d’Informatique et
> de recherche opérationnelle
> Université de Montréal
> email:Aaron.Courville at gmail.com
> _______________________________________________
> Lisa_labo mailing list
> Lisa_labo at iro.umontreal.ca
> https://webmail.iro.umontreal.ca/mailman/listinfo/lisa_labo


-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://webmail.iro.umontreal.ca/pipermail/lisa_teatalk/attachments/20120229/2c996740/attachment-0002.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: chapterYB.pdf
Type: application/pdf
Size: 547135 bytes
Desc: not available
Url : http://webmail.iro.umontreal.ca/pipermail/lisa_teatalk/attachments/20120229/2c996740/attachment-0001.pdf 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://webmail.iro.umontreal.ca/pipermail/lisa_teatalk/attachments/20120229/2c996740/attachment-0003.html 


More information about the Lisa_teatalk mailing list