Février 2015 - Lisa_teatalk - mailman.iro.umontreal.ca

Tea Talk by Julian Serban this Friday (27 Feb) 14.00 - 15.00 @AA3195
by Kyung Hyun Cho 28 Fév '15

28 Fév '15

Dear all, Julian Serban will tell us about "Learning to Play Chess with Deep Learning" this Friday at the usual place starting from 14.00. Hope to see many of you at the tea talk! - Cho === Speaker: Julian Serban Date/Time: 27 Feb 2015 14.00-15.00 Place: AA3195 Title: Learning to Play Chess with Deep Learning Abstract: TBA

1 1

Tea Talk by Prof. Yoshua Bengio on Friday (20 Feb) at 14.00
by Kyung Hyun Cho 21 Fév '15

21 Fév '15

Dear all, We will be back to the usual tea talk schedule starting from next Friday with Yoshua introducing us a biologically plausible learning algorithm for training a deep generative neural network. The talk will start at 14.00 in the usual place of AA3195. Hope to see many of you there! - Cho === Speaker: Prof. Yoshua Bengio Date/Time: 14.00 - 15.00 on Friday (20 Feb) Place: AA3195 Title: Biologically Plausible Deep Learning Abstract: Neuroscientists have long criticised deep learning algorithms as incompatible with current knowledge of neurobiology. We explore more biologically plausible versions of deep representation learning, focusing here mostly on unsupervised learning but developing a learning mechanism that could account for supervised, unsupervised and reinforcement learning. The starting point is that the basic learning rule believed to govern synaptic weight updates (Spike-Timing Dependent Plasticity) can be interpreted as gradient descent on some objective function so long as the neuronal dynamics push firing rates towards better values of the objective function (be it supervised, unsupervised, or reward-driven). The second main idea is that this corresponds to a form of variational EM algorithm in which the above dynamics correspond to approximate inference. Another contribution of this paper is that the gradients required for updating the hidden states in the above variational interpretation can be estimated using an approximation that only requires propagating activations forward and backward, with pairs of layers learning to form a denoising auto-encoder. Finally, we extend the theory about the probabilistic interpretation of autoencoders to justify improved sampling schemes based on the generative interpretation of denoising auto-encoders, and we validate all these ideas on generative learning tasks.

2 1

Tea Talk by Prof. Yoshua Bengio on Friday (20 Feb) at 14.00
by Kyunghyun Cho 15 Fév '15

15 Fév '15

Dear all, We will be back to the usual tea talk schedule starting from next Friday with Yoshua introducing us a biologically plausible learning algorithm for training a deep generative neural network. The talk will start at 14.00 in the usual place of AA3195. Hope to see many of you there! - Cho === Speaker: Prof. Yoshua Bengio Date/Time: 14.00 - 15.00 on Friday (20 Feb) Place: AA3195 Title: Biologically Plausible Deep Learning Abstract: Neuroscientists have long criticised deep learning algorithms as incompatible with current knowledge of neurobiology. We explore more biologically plausible versions of deep representation learning, focusing here mostly on unsupervised learning but developing a learning mechanism that could account for supervised, unsupervised and reinforcement learning. The starting point is that the basic learning rule believed to govern synaptic weight updates (Spike-Timing Dependent Plasticity) can be interpreted as gradient descent on some objective function so long as the neuronal dynamics push firing rates towards better values of the objective function (be it supervised, unsupervised, or reward-driven). The second main idea is that this corresponds to a form of variational EM algorithm in which the above dynamics correspond to approximate inference. Another contribution of this paper is that the gradients required for updating the hidden states in the above variational interpretation can be estimated using an approximation that only requires propagating activations forward and backward, with pairs of layers learning to form a denoising auto-encoder. Finally, we extend the theory about the probabilistic interpretation of autoencoders to justify improved sampling schemes based on the generative interpretation of denoising auto-encoders, and we validate all these ideas on generative learning tasks.

1 0

Talk by Dr. Harri Valpola on Wednesday 11 Feb at 13.30 @AA3195
by Kyung Hyun Cho 07 Fév '15

07 Fév '15

Dear all, Next week, we will have a visitor from Finland. Dr. Harri Valpola is a machine learning researcher from ZenRobotics (Helsinki) and has been collaborating with Prof. Tapani Raiko on various topics including unsupervised learning. He will arrive at the lab in the afternoon on Tuesday (around 2pm). If anyone's interested in talking to him, please, let him know by replying to his email (harri.valpola(a)iki.fi) On the next day (Wednesday), he will tell us about his recent research on unsupervised neural networks at our usual seminar place starting at 13.30. Please, see below for Harri's message! - Cho === Title of the talk: Deep unsupervised and semi-supervised learning Speaker: Harri Valpola, ZenRobotics Ltd. I'm Harri Valpola, a machine learning researcher from Helsinki, currently working for ZenRobotics which I co-founded in 2007. Between 2010 and 2014 I was temporarily focused on product development (machine learning applied to robots) but now that our product (robotic waste sorting) is pretty much ready, I have been able to return back to research since summer 2014. My main research topic has been unsupervised learning ever since I started working in Kohonen's lab in 1993. Now I'm going to talk about a model which I've been working on in collaboration with Tapani Raiko and Antti Rasmus from Aalto University. The model combines denoising autoencoders and denoising source separation. The goal is to design an unsupervised learning method which supports supervised learning as well as possible. The network needs to be able to focus on abstract invariant features on the higher layers and learn deep hierarchies efficiently. This is achieved by shortcut connections from the encoder to the decoder and cost functions on many layers of the network (not just reconstruction of observations and prediction of targets). There are published results in two arXiv papers ( https://arxiv.org/abs/1411.7783 and https://arxiv.org/abs/1412.7210) but I'm also going to talk a bit about our very preliminary results on semi-supervised learning. I'm visiting Montreal between Tuesday and Thursday (Feb 10-12) and I hope to meet many of you. In general, I'm interested in anything that helps me in my pursue: I want to build a brain for autonomous robots. Part of the solution, I believe, is a "cortical algorithm". Currently there are good solutions for some subtasks but not for all (or even most) of them at the same time: * deep hierarchy of abstract invariant features * semi-supervised learning * efficient inference (state estimation which integrates information from all modalities and across time) * segmentation * attention * simulation and planning * representation, detection and learning of relations Perhaps there are solutions that I'm not aware of -- I hope someone gives me some tips in Montreal :-) The model I'm pursuing should be able to handle time-series data and work efficiently so optimization is also among the interesting topics. The sampling aspect of GSN is definitely interesting, too. I don't know if anybody is doing robotics-related work in Montreal but I'll mention a few topics that I'm interested in, just in case: * hierarchical control (especially unsupervised and reinforcement learning of hc) and decision making * working memory and attention (and control of them) * cerebellar learning (related to iterative learning control) In case someone is interested to hear my comments about your own work, I can advertise my own background (chronological order so starts more than 20 years back): * invariant features from temporal slowness * sparse coding * variational unsupervised learning of generative models * denoising source separation * hierarchical latent variable models * cerebellar learning * hierarchical control * higher-order models * segmentation and attention * representation, detection and learning of relations * reinforcement learning I've worked with various types of data, including: * speech * images * industrial processes * astrophysics * climate * neuroimaging * various robotics-related I'm looking forward to meeting you in Montreal, - Harri -

1 0