[Lisa_teatalk] Tea Talks 12th Feb

Jörg Bornschein bornj at iro.umontreal.ca
Mon Feb 8 09:35:47 EST 2016


Hi everyone,


besides of David Duvenauds talk on Thursday, we will have two tea talks
this Friday:

First Sixin Zhang will talk about learning with elastic averaging SGD and
afterwards James Henderson will talk about Spike Event Based Learning in
Neural Networks

Here are the details:


Title: Deep learning with Elastic Averaging SGD
Who: Sixin Zhang
When: Fri 12th Feb, 14:30
Where: AA 3195

In this talk, I'll introduce Elastic Averaging SGD (EASGD). It is a
stochastic optimization method developed to train large deep learning
models in the parallel computing environment under communication
constraints. We start from the basic ideas of SGD and Average SGD to
motivate EASGD. The basic stability and averaging properties of synchronous
EASGD will be discussed. Then we accelerate EASGD with Nesterov's optimal
method, giving EAMSGD. The robustness of EASGD and EAMSGD to the
communication delay allows us to train CIFAR-10 and ImageNet-1k image
classification tasks with relatively large parameter communication period.
We will try to understand why the Downpour method does not work so well,
and see how to fix that. If time permits, I'll discuss the tree extension
of EASGD, and why it's a good or maybe a bad idea.

This is joint-work with Anna Choromanska and Yann LeCun.



Title: Spike Event Based Learning in Neural Networks
Who: James Henderson
When: Fri 12th, 15:30
Where: AA 3195

A scheme is derived for learning connectivity in spiking neural networks.
The scheme learns instantaneous firing rates that are conditional on the
activity in other parts of the network. The scheme is independent of the
choice of neuron dynamics or activation function, and network architecture.
It involves two simple, online, local learning rules that are applied only
in response to occurrences of spike events. This scheme provides a direct
method for transferring ideas between the fields of deep learning and
computational neuroscience. This learning scheme is demonstrated using a
layered feedforward spiking neural network trained self-supervised on a
prediction and classification task for moving MNIST images collected using
a Dynamic Vision Sensor.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://webmail.iro.umontreal.ca/pipermail/lisa_teatalk/attachments/20160208/0b61fc96/attachment.html 


More information about the Lisa_teatalk mailing list