[Lisa_teatalk] Tea Talk Tomorrow (NOTE: new location)

Aaron Courville aaron.courville at gmail.com
Wed Feb 29 15:11:04 EST 2012


After the ICML rush, we are restarting our tea talk series with some
deep thoughts from Yoshua. See you there.

When: 15h00 Thursday, March 1st. 2012
Where: AA1355 (NEW LOCATION)

Title/Abstract:

Evolving Culture vs Local Minima
Yoshua Bengio

 We propose a theory that relates difficulty of learning in deep
architectures to culture and language. It is articulated around the
following hypotheses:

(1) learning in an individual  human brain is hampered by the presence
of effective local minima;
(2) this optimization difficulty is particularly important when it
comes to learning higher-level abstractions, i.e., concept that cover
a vast and highly-nonlinear span of sensory configurations;
(3) such high-level abstractions are best represented in brains by the
composition of many levels of representation, i.e., by deep
architectures;
(4) a human brain can learn such high-level abstractions if guided by
the signals produced by other humans, which act as hints or indirect
supervision for these high-level abstractions; and
(5), language and the combination of old ideas into new ideas provide
an efficient evolutionary recombination operator, and this allows
rapid search in the space of communicable ideas that help humans build
up better high-level internal representations of their world.

These hypotheses put together imply that human culture and the
evolution of ideas have been crucial to counter an optimization
difficulty: this optimization difficulty would otherwise make it very
difficult for human brains to capture high-level knowledge of the
world.  The theory is grounded in experimental observations of the
difficulties of training deep artificial neural networks. Plausible
consequences of this theory for the efficiency of cultural evolutions
are sketched.


Cheers,
Aaron

-- 
Aaron C. Courville
Département d’Informatique et
de recherche opérationnelle
Université de Montréal
email:Aaron.Courville at gmail.com


More information about the Lisa_teatalk mailing list