[Lisa_teatalk] Tea Talk Tomorrow!

Aaron Courville aaron.courville at gmail.com
Wed Nov 24 13:50:38 EST 2010


Hi,

Tomorrow we will be treated to a talk by Pascal Vincent on his recent
work on the theoretical underpinning of Denoising Autoencoders. Hope
to see you there.

Time &Date: Thursday Nov. 25th, 14h30
Location: Lisa Lab (AA3256)

TITLE: A connection between Score Matching and Denoising Autoencoders

ABSTRACT:

In this tea talk I will present my recent findings on a connection
between two apparently very different techniques: denoising
autoencoders and score matching.

Denoising autoencoders have been previously shown to be competitive al-
ternatives to Restricted Boltzmann Machines for unsupervised pre-training
of each layer of a deep architecture. I will show that a simple denoising au-
toencoder training criterion is equivalent to matching the score of a specific
energy based model to that of a non-parametric Parzen density estimator of
the data. This yields several useful insights. It defines a proper probabilistic
model for the denoising autoencoder technique which makes it in principle
possible to sample from them or to rank examples by their energy. It suggests
a different way to apply score matching that is related to learning to denoise
and does not require computing second derivatives. It justifies the use of tied
weights between the encoder and decoder, and suggests ways to extend the
success of denoising autoencoders to a larger family of energy-based models.


Cheers,
Aaron

-- 
Aaron C. Courville
Département d’Informatique et
de recherche opérationnelle
Université de Montréal
email:Aaron.Courville at gmail.com


More information about the Lisa_teatalk mailing list