Dear all,
Next week, we will have a visitor from Finland. Dr. Harri Valpola is a
machine learning researcher from ZenRobotics (Helsinki) and has been
collaborating with Prof. Tapani Raiko on various topics including
unsupervised learning.
He will arrive at the lab in the afternoon on Tuesday (around 2pm). If
anyone's interested in talking to him, please, let him know by replying to
his email (harri.valpola(a)iki.fi). On the next day (Wednesday), he will tell
us about his recent research on unsupervised neural networks at our usual
seminar place starting at 13.30.
Please, see below for Harri's message!
- Cho
===
Title of the talk: Deep unsupervised and semi-supervised learning
Speaker: Harri Valpola, ZenRobotics Ltd.
I'm Harri Valpola, a machine learning researcher from Helsinki, currently
working for ZenRobotics which I co-founded in 2007. Between 2010 and 2014 I
was temporarily focused on product development (machine learning applied to
robots) but now that our product (robotic waste sorting) is pretty much
ready, I have been able to return back to research since summer 2014.
My main research topic has been unsupervised learning ever since I started
working in Kohonen's lab in 1993. Now I'm going to talk about a model which
I've been working on in collaboration with Tapani Raiko and Antti Rasmus
from Aalto University. The model combines denoising autoencoders and
denoising source separation. The goal is to design an unsupervised learning
method which supports supervised learning as well as possible. The network
needs to be able to focus on abstract invariant features on the higher
layers and learn deep hierarchies efficiently. This is achieved by shortcut
connections from the encoder to the decoder and cost functions on many
layers of the network (not just reconstruction of observations and
prediction of targets). There are published results in two arXiv papers (
https://arxiv.org/abs/1411.7783 and https://arxiv.org/abs/1412.7210) but
I'm also going to talk a bit about our very preliminary results on
semi-supervised learning.
I'm visiting Montreal between Tuesday and Thursday (Feb 10-12) and I hope
to meet many of you. In general, I'm interested in anything that helps me
in my pursue: I want to build a brain for autonomous robots. Part of the
solution, I believe, is a "cortical algorithm". Currently there are good
solutions for some subtasks but not for all (or even most) of them at the
same time:
* deep hierarchy of abstract invariant features
* semi-supervised learning
* efficient inference (state estimation which integrates information from
all modalities and across time)
* segmentation
* attention
* simulation and planning
* representation, detection and learning of relations
Perhaps there are solutions that I'm not aware of -- I hope someone gives
me some tips in Montreal :-) The model I'm pursuing should be able to
handle time-series data and work efficiently so optimization is also among
the interesting topics. The sampling aspect of GSN is definitely
interesting, too.
I don't know if anybody is doing robotics-related work in Montreal but I'll
mention a few topics that I'm interested in, just in case:
* hierarchical control (especially unsupervised and reinforcement learning
of hc) and decision making
* working memory and attention (and control of them)
* cerebellar learning (related to iterative learning control)
In case someone is interested to hear my comments about your own work, I
can advertise my own background (chronological order so starts more than 20
years back):
* invariant features from temporal slowness
* sparse coding
* variational unsupervised learning of generative models
* denoising source separation
* hierarchical latent variable models
* cerebellar learning
* hierarchical control
* higher-order models
* segmentation and attention
* representation, detection and learning of relations
* reinforcement learning
I've worked with various types of data, including:
* speech
* images
* industrial processes
* astrophysics
* climate
* neuroimaging
* various robotics-related
I'm looking forward to meeting you in Montreal,
- Harri -
