Hey,
The Tea Talk this week will be given by Yoshua Bengio, discussing his latest ideas regarding density estimation with CAE-type models.
When: 15h00, Thursday, April 26th. Where: LISA Lab.
Title/Abstract:
Density Function Induction by Estimating Local Density Structure
I am exploring a novel learning framework for unsupervised learning which should be viewed as an alternative to maximum likelihood training, score matching, pseudo-likelihood, etc. It may help to justify algorithms such as the Contracting Auto-Encoder and clarify the corresponding density model learned. The basic information learned by a CAE and a DAE are (1) where to move in input space so as to increase density (i.e. towards the reconstruction), and (2) if near a manifold (i.e. a mode of dimension > 0) in what directions (and how much) to move in order to remain in a high-density region. We have shown that one can use geometric intuition to define a sampling algorithm in the form of an MCMC using this information, and that it mixes rather well and produces visually and numerically good samples. I first show that this implicitly defines a density learned by the model, which is the asymptotic distribution of that MCMC. I then discuss consistency, i.e., what it is that (1) and (2) above should be such that this learned implicit density function corresponds (at least asymptotically) to the unknown generating process from which the training data were sampled. This is work in progress with yet many mysteries and open questions. In addition to providing a more solid probabilistic footing for CAE-like or DAE-like algorithms this work could potentially yield better training algorithms for this type of learner as well as better sampling algorithms for them.
Cheers, Aaron
Afficher les réponses par date