[Lisa_seminaires] Eugene Vorontsov, July 21, AA6214, 13:45

Junyoung Chung elecegg at gmail.com
Dim 16 Juil 23:48:57 EDT 2017


Hi all,

our next speaker is Eugene Vorontsov. Hope to see you many of you there.
When: 13:45, July 21
Where: AA6214

Title: On orthogonality and learning recurrent networks with long term
dependencies

Abstract:

It is well known that it is challenging to train deep neural networks and
recurrent neural networks for tasks that exhibit long term dependencies.
The vanishing or exploding gradient problem is a well known issue
associated with these challenges. One approach to addressing vanishing and
exploding gradients is to use either soft or hard constraints on weight
matrices so as to encourage or enforce orthogonality. Orthogonal matrices
preserve gradient norm during backpropagation and may therefore be a
desirable property. This paper explores issues with optimization
convergence, speed and gradient stability when encouraging or enforcing
orthogonality. To perform this analysis, we propose a weight matrix
factorization and parameterization strategy through which we can bound
matrix norms and therein control the degree of expansivity induced during
backpropagation. We find that hard constraints on orthogonality can
negatively affect the speed of convergence and model performance.

Bio:
I am a PhD student with professors Chris Pal and Samuel Kadoury at École
Polytechnique de Montréal and MILA. I am working on medical image
segmentation models and have recently begun developing an interest in
optimization and regularization of deep neural networks. Prior to MILA, I
studied Engineering Science at the University of Toronto, specializing in
biomedical engineering. An aspect of computer science that appealed to me
is the fast production of experimental results -- unfortunately, I also
like big models.

--Junyoung
-------------- section suivante --------------
Une pièce jointe HTML a été nettoyée...
URL: http://webmail.iro.umontreal.ca/pipermail/lisa_seminaires/attachments/20170716/747da0e4/attachment.html 


Plus d'informations sur la liste de diffusion Lisa_seminaires