[Lisa_seminaires] Tea talk Wednesday 19th @13:00 AA3195 by Roland Memisevic

Razvan Pascanu r.pascanu at gmail.com
Lun 17 Mar 13:42:12 EDT 2014


Hi all,

 I'm glad to announce our next tea talk this Wednesday (19th March), at the
usual place AA3195 at 13:00. Roland is going to talk to us about some of
his recent work. See title and abstract below. I hope to see many of you
there !

Title: Zero-bias autoencoders

Abstract:
We recently stumbled upon a hidden unit activation function that makes it
very easy to train an autoencoder by simply minimizing reconstruction
error, without using any additional regularization. The activation function
takes the following form: it is zero if the net input to the neuron is
smaller than some threshold and it is linear elsewhere. It seems important
for the linear part to be truly linear and not affine, in other words the
bias term for the linear part should be zero. I will describe some
experiments as well as some theoretical insights that help explain why this
activation function works so well. Joint work with Kishore Konda and David
Krueger.

In ASCII art:

                        /
                       /
                      /
                     /
                    /
                   /
                  |
                  |
                  |
--------0----------

hidden unit's net input on the x-axis, output on the y-axis. (this picture
may be messed up if your mail-reader is smart about font spacing)


Best,
Razvan
-------------- section suivante --------------
Une pièce jointe HTML a été nettoyée...
URL: https://webmail.iro.umontreal.ca/mailman/private/lisa_seminaires/attachments/20140317/da02c4af/attachment.html 


Plus d'informations sur la liste de diffusion Lisa_seminaires