[Lisa_teatalk] Tea Talk Thursday!

Aaron Courville aaron.courville at gmail.com
Tue Apr 24 14:37:57 EDT 2012


Hey,

The Tea Talk this week will be given by Yoshua Bengio, discussing his
latest ideas regarding density estimation with CAE-type models.

When: 15h00, Thursday, April 26th.
Where: LISA Lab.

Title/Abstract:

Density Function Induction by Estimating Local Density Structure

I am exploring a novel learning framework for unsupervised learning
which should be viewed as
an alternative to maximum likelihood training, score matching,
pseudo-likelihood, etc. It may help
to justify algorithms such as the Contracting Auto-Encoder and clarify
the corresponding density model
learned. The basic information learned by a CAE and a DAE are (1)
where to move in input space
so as to increase density (i.e. towards the reconstruction), and (2)
if near a manifold (i.e.
a mode of dimension > 0) in what directions (and how much) to move in
order to remain
in a high-density region. We have shown that one can use geometric
intuition to define a sampling
algorithm in the form of an MCMC using this information, and that it
mixes rather well and produces visually
and numerically good samples. I first show that this implicitly
defines a density learned by the model,
which is the asymptotic distribution of that MCMC. I then discuss
consistency, i.e., what it is that
(1) and (2) above should be such that this learned implicit density
function corresponds
(at least asymptotically) to the unknown generating process from which
the training data
were sampled. This is work in progress with yet many mysteries and
open questions.
In addition to providing a more solid probabilistic footing for
CAE-like or DAE-like algorithms
this work could potentially yield better training algorithms for this
type of learner as well
as better sampling algorithms for them.


Cheers,
Aaron

-- 
Aaron C. Courville
Département d’Informatique et
de recherche opérationnelle
Université de Montréal
email:Aaron.Courville at gmail.com


More information about the Lisa_teatalk mailing list