[Lisa_seminaires] Eugene Vorontsov, July 21, AA6214, 13:45

Junyoung Chung elecegg at gmail.com
Jeu 20 Juil 18:09:16 EDT 2017


Hi all,

we will have a tea-talk tomorrow.
Eugene will present his recent work on RNNs that was accepted to this ICML.

Best,
--Junyoung

On Sun, Jul 16, 2017 at 11:48 PM, Junyoung Chung <elecegg at gmail.com> wrote:

> Hi all,
>
> our next speaker is Eugene Vorontsov. Hope to see you many of you there.
> When: 13:45, July 21
> Where: AA6214
>
> Title: On orthogonality and learning recurrent networks with long term
> dependencies
>
> Abstract:
>
> It is well known that it is challenging to train deep neural networks and
> recurrent neural networks for tasks that exhibit long term dependencies.
> The vanishing or exploding gradient problem is a well known issue
> associated with these challenges. One approach to addressing vanishing and
> exploding gradients is to use either soft or hard constraints on weight
> matrices so as to encourage or enforce orthogonality. Orthogonal matrices
> preserve gradient norm during backpropagation and may therefore be a
> desirable property. This paper explores issues with optimization
> convergence, speed and gradient stability when encouraging or enforcing
> orthogonality. To perform this analysis, we propose a weight matrix
> factorization and parameterization strategy through which we can bound
> matrix norms and therein control the degree of expansivity induced during
> backpropagation. We find that hard constraints on orthogonality can
> negatively affect the speed of convergence and model performance.
>
> Bio:
> I am a PhD student with professors Chris Pal and Samuel Kadoury at École
> Polytechnique de Montréal and MILA. I am working on medical image
> segmentation models and have recently begun developing an interest in
> optimization and regularization of deep neural networks. Prior to MILA, I
> studied Engineering Science at the University of Toronto, specializing in
> biomedical engineering. An aspect of computer science that appealed to me
> is the fast production of experimental results -- unfortunately, I also
> like big models.
>
> --Junyoung
>
-------------- section suivante --------------
Une pièce jointe HTML a été nettoyée...
URL: http://webmail.iro.umontreal.ca/pipermail/lisa_seminaires/attachments/20170720/c67bd774/attachment.html 


Plus d'informations sur la liste de diffusion Lisa_seminaires