Hi everyone,
besides of David Duvenauds talk on Thursday, we will have two tea talks this Friday:
First Sixin Zhang will talk about learning with elastic averaging SGD and afterwards James Henderson will talk about Spike Event Based Learning in Neural Networks
Here are the details:
Title: Deep learning with Elastic Averaging SGD
Who: Sixin Zhang
When: Fri 12th Feb, 14:30
Where: AA 3195
In this talk, I'll introduce Elastic Averaging SGD (EASGD). It is a stochastic optimization method developed to train large deep learning models in the parallel computing environment under communication constraints. We start from the basic ideas of SGD and Average SGD to motivate EASGD. The basic stability and averaging properties of synchronous EASGD will be discussed. Then we accelerate EASGD with Nesterov's optimal method, giving EAMSGD. The robustness of EASGD and EAMSGD to the communication delay allows us to train CIFAR-10 and ImageNet-1k image classification tasks with relatively large parameter communication period. We will try to understand why the Downpour method does not work so well, and see how to fix that. If time permits, I'll discuss the tree extension of EASGD, and why it's a good or maybe a bad idea.
This is joint-work with Anna Choromanska and Yann LeCun.
Title: Spike Event Based Learning in Neural Networks
Who: James Henderson
When: Fri 12th, 15:30
Where: AA 3195
A scheme is derived for learning connectivity in spiking neural networks. The scheme learns instantaneous firing rates that are conditional on the activity in other parts of the network. The scheme is independent of the choice of neuron dynamics or activation function, and network architecture. It involves two simple, online, local learning rules that are applied only in response to occurrences of spike events. This scheme provides a direct method for transferring ideas between the fields of deep learning and computational neuroscience. This learning scheme is demonstrated using a layered feedforward spiking neural network trained self-supervised on a prediction and classification task for moving MNIST images collected using a Dynamic Vision Sensor.