Note the unusual location!
If you participate in the pylearn2 code sprint, please take a refreshing break at 3pm...
Also, for those interested, I am writing a paper on this subject. The draft is in articles/2012/culture_vs_localmin/ (and I attach a pdf of the latest version for your convenience) and I am supposed to submit it this week to as a book chapter, so any comment is welcome. Note also that this is meant to be the starting point for Caglar's thesis (who joined us this term for a PhD).
-- Yoshua
On 2012-02-29, at 15:11, Aaron Courville wrote:
After the ICML rush, we are restarting our tea talk series with some deep thoughts from Yoshua. See you there.
When: 15h00 Thursday, March 1st. 2012 Where: AA1355 (NEW LOCATION)
Title/Abstract:
Evolving Culture vs Local Minima Yoshua Bengio
We propose a theory that relates difficulty of learning in deep architectures to culture and language. It is articulated around the following hypotheses:
(1) learning in an individual human brain is hampered by the presence of effective local minima; (2) this optimization difficulty is particularly important when it comes to learning higher-level abstractions, i.e., concept that cover a vast and highly-nonlinear span of sensory configurations; (3) such high-level abstractions are best represented in brains by the composition of many levels of representation, i.e., by deep architectures; (4) a human brain can learn such high-level abstractions if guided by the signals produced by other humans, which act as hints or indirect supervision for these high-level abstractions; and (5), language and the combination of old ideas into new ideas provide an efficient evolutionary recombination operator, and this allows rapid search in the space of communicable ideas that help humans build up better high-level internal representations of their world.
These hypotheses put together imply that human culture and the evolution of ideas have been crucial to counter an optimization difficulty: this optimization difficulty would otherwise make it very difficult for human brains to capture high-level knowledge of the world. The theory is grounded in experimental observations of the difficulties of training deep artificial neural networks. Plausible consequences of this theory for the efficiency of cultural evolutions are sketched.
Cheers, Aaron
-- Aaron C. Courville Département d’Informatique et de recherche opérationnelle Université de Montréal email:Aaron.Courville@gmail.com _______________________________________________ Lisa_labo mailing list Lisa_labo@iro.umontreal.ca https://webmail.iro.umontreal.ca/mailman/listinfo/lisa_labo
Afficher les réponses par date