[Lisa_seminaires] Eugene Vorontsov, July 21, AA6214, 13:45

Junyoung Chung elecegg at gmail.com
Ven 21 Juil 13:39:11 EDT 2017


It's starting soon!

On Fri, Jul 21, 2017 at 1:15 PM Junyoung Chung <elecegg at gmail.com> wrote:

> We will have the tea talk in 30 minutes!
>
> On Thu, Jul 20, 2017 at 6:09 PM Junyoung Chung <elecegg at gmail.com> wrote:
>
>> Hi all,
>>
>> we will have a tea-talk tomorrow.
>> Eugene will present his recent work on RNNs that was accepted to this
>> ICML.
>>
>> Best,
>> --Junyoung
>>
>> On Sun, Jul 16, 2017 at 11:48 PM, Junyoung Chung <elecegg at gmail.com>
>> wrote:
>>
>>> Hi all,
>>>
>>> our next speaker is Eugene Vorontsov. Hope to see you many of you there.
>>> When: 13:45, July 21
>>> Where: AA6214
>>>
>>> Title: On orthogonality and learning recurrent networks with long term
>>> dependencies
>>>
>>> Abstract:
>>>
>>> It is well known that it is challenging to train deep neural networks
>>> and recurrent neural networks for tasks that exhibit long term
>>> dependencies. The vanishing or exploding gradient problem is a well known
>>> issue associated with these challenges. One approach to addressing
>>> vanishing and exploding gradients is to use either soft or hard constraints
>>> on weight matrices so as to encourage or enforce orthogonality. Orthogonal
>>> matrices preserve gradient norm during backpropagation and may therefore be
>>> a desirable property. This paper explores issues with optimization
>>> convergence, speed and gradient stability when encouraging or enforcing
>>> orthogonality. To perform this analysis, we propose a weight matrix
>>> factorization and parameterization strategy through which we can bound
>>> matrix norms and therein control the degree of expansivity induced during
>>> backpropagation. We find that hard constraints on orthogonality can
>>> negatively affect the speed of convergence and model performance.
>>>
>>> Bio:
>>> I am a PhD student with professors Chris Pal and Samuel Kadoury at École
>>> Polytechnique de Montréal and MILA. I am working on medical image
>>> segmentation models and have recently begun developing an interest in
>>> optimization and regularization of deep neural networks. Prior to MILA, I
>>> studied Engineering Science at the University of Toronto, specializing in
>>> biomedical engineering. An aspect of computer science that appealed to me
>>> is the fast production of experimental results -- unfortunately, I also
>>> like big models.
>>>
>>> --Junyoung
>>>
>>
>> --
> --Junyoung
>
-- 
--Junyoung
-------------- section suivante --------------
Une pièce jointe HTML a été nettoyée...
URL: http://webmail.iro.umontreal.ca/pipermail/lisa_seminaires/attachments/20170721/ccd61a3d/attachment.html 


Plus d'informations sur la liste de diffusion Lisa_seminaires