Next week, we will have a visitor from Finland. Dr. Harri Valpola is a machine learning researcher from ZenRobotics (Helsinki) and has been collaborating with Prof. Tapani Raiko on various topics including unsupervised learning.
Title of the talk: Deep unsupervised and semi-supervised learning
Speaker: Harri Valpola, ZenRobotics Ltd.
I'm Harri Valpola, a machine learning researcher from Helsinki, currently working for ZenRobotics which I co-founded in 2007. Between 2010 and 2014 I was temporarily focused on product development (machine learning applied to robots) but now that our product (robotic waste sorting) is pretty much ready, I have been able to return back to research since summer 2014.
My main research topic has been unsupervised learning ever since I started working in Kohonen's lab in 1993. Now I'm going to talk about a model which I've been working on in collaboration with Tapani Raiko and Antti Rasmus from Aalto University. The model combines denoising autoencoders and denoising source separation. The goal is to design an unsupervised learning method which supports supervised learning as well as possible. The network needs to be able to focus on abstract invariant features on the higher layers and learn deep hierarchies efficiently. This is achieved by shortcut connections from the encoder to the decoder and cost functions on many layers of the network (not just reconstruction of observations and prediction of targets). There are published results in two arXiv papers (
https://arxiv.org/abs/1411.7783 and
https://arxiv.org/abs/1412.7210) but I'm also going to talk a bit about our very preliminary results on semi-supervised learning.
I'm visiting Montreal between Tuesday and Thursday (Feb 10-12) and I hope to meet many of you. In general, I'm interested in anything that helps me in my pursue: I want to build a brain for autonomous robots. Part of the solution, I believe, is a "cortical algorithm". Currently there are good solutions for some subtasks but not for all (or even most) of them at the same time:
* deep hierarchy of abstract invariant features
* semi-supervised learning
* efficient inference (state estimation which integrates information from all modalities and across time)
* segmentation
* attention
* simulation and planning
* representation, detection and learning of relations
Perhaps there are solutions that I'm not aware of -- I hope someone gives me some tips in Montreal :-) The model I'm pursuing should be able to handle time-series data and work efficiently so optimization is also among the interesting topics. The sampling aspect of GSN is definitely interesting, too.
I don't know if anybody is doing robotics-related work in Montreal but I'll mention a few topics that I'm interested in, just in case:
* hierarchical control (especially unsupervised and reinforcement learning of hc) and decision making
* working memory and attention (and control of them)
* cerebellar learning (related to iterative learning control)
In case someone is interested to hear my comments about your own work, I can advertise my own background (chronological order so starts more than 20 years back):
* invariant features from temporal slowness
* sparse coding
* variational unsupervised learning of generative models
* denoising source separation
* hierarchical latent variable models
* cerebellar learning
* hierarchical control
* higher-order models
* segmentation and attention
* representation, detection and learning of relations
* reinforcement learning
I've worked with various types of data, including:
* speech
* images
* industrial processes
* astrophysics
* climate
* neuroimaging
* various robotics-related
I'm looking forward to meeting you in Montreal,
- Harri -