[Lisa_seminaires] [Lisa_teatalk] [Tea Talk] Nicolas Le Roux (Google) May 25 2018 10:30 AM AA3195

Michael Noukhovitch mnoukhov at gmail.com
Mar 22 Mai 13:09:17 EDT 2018


Also, if you'd like to meet Nicolas for a 1 on 1, sign up for a slot on the
Google Calendar Appointment:
https://calendar.google.com/calendar/selfsched?sstoken=UUUxb01teGhLckE3fGRlZmF1bHR8ZjRkYmRlYjFkYzE3YzY4MDg1YzE1NTM2YzNmMGE2OGU

Sorry for the extra emails,
Michael

On Tue, May 22, 2018 at 12:45 PM Nicolas Le Roux <nlr at google.com> wrote:

> For those of you who attended my talk at the Workshop on learning theory,
> this one will fill in some of the gaps I mentioned.
>
> On Tue, 22 May 2018 at 12:17, Michael Noukhovitch <mnoukhov at gmail.com>
> wrote:
>
>> This week we will have two tea talks (morning and afternoon) on Friday!
>> Wow!
>>
>> The first talk will be *Nicolas Le* *Roux * on *May 25 2018* at *10:30
>> AM* in room *AA3195* (note the room change, we'll most likely be in 3195
>> for the summer).
>>
>> Streaming is, as always, at this link:
>> https://bluejeans.com/809027115/webrtc but watch it live because there
>> will be no recordings!
>>
>> Don't waver on going to this talk, it is sure to be invariantly excellent!
>> Michael
>>
>> *TITLE* An exploration of variance reduction techniques in stochastic
>> optimization
>>
>> *KEYWORDS *deep learning theory, optimization
>>
>> *ABSTRACT*
>> I will present recent and ongoing work on reducing the variance in
>> stochastic optimization techniques to speed-up and simplify the resulting
>> algorithms. In particular, stochastic gradient methods can suffer from high
>> variance, limiting their convergence speed. While variance reduction
>> techniques exist in the finite case, they are rarer in the online case. We
>> demonstrate how an increasing momentum offers variance reduction in the
>> online case, at the expense of bias, and how that bias can be countered by
>> an extrapolation step. The resulting algorithm differs from iterate
>> averaging in only a factor, but, in the context of the minimization of a
>> quadratic function, this difference is enough to lead to the first
>> algorithm converging both linearly in the noiseless and sublinearly in the
>> homoscedastic noise case when using a constant stepsize.
>>
>> *BIO*
>> *Nicolas Le Roux got an MSc in Applied Maths from Ecole Centrale Paris
>> and an MSc in Maths, Learning and Vision from ENS Cachan. He got his PhD in
>> 2008 from University of Montreal where he worked with Yoshua Bengio on
>> neural networks in general and their optimisation in particular. He then
>> moved to Microsoft Research Cambridge to work on generative models of
>> images with John Winn. In 2010, he joined Inria in Francis Bach's team to
>> work on large-scale convex optimisation. From 2012 to 2017, he created and
>> managed the research team at Criteo in Paris. He joined Google Brain
>> Montreal in 2017 where he now works on large-scale optimization and
>> reinforcement learning.*
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "brain-montreal-office-hours" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to brain-montreal-office-hours+unsubscribe at google.com.
>> To post to this group, send email to
>> brain-montreal-office-hours at google.com.
>> To view this discussion on the web visit
>> https://groups.google.com/a/google.com/d/msgid/brain-montreal-office-hours/CAMVr0Vy4KsyukWMA_7jfFzebONgVbD3Oh7wa%2Bk5eVxta8KYUVg%40mail.gmail.com
>> <https://groups.google.com/a/google.com/d/msgid/brain-montreal-office-hours/CAMVr0Vy4KsyukWMA_7jfFzebONgVbD3Oh7wa%2Bk5eVxta8KYUVg%40mail.gmail.com?utm_medium=email&utm_source=footer>
>> .
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "brain-mon-fte" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to brain-mon-fte+unsubscribe at google.com.
>> To post to this group, send email to brain-mon-fte at google.com.
>> To view this discussion on the web visit
>> https://groups.google.com/a/google.com/d/msgid/brain-mon-fte/CAMVr0Vy4KsyukWMA_7jfFzebONgVbD3Oh7wa%2Bk5eVxta8KYUVg%40mail.gmail.com
>> <https://groups.google.com/a/google.com/d/msgid/brain-mon-fte/CAMVr0Vy4KsyukWMA_7jfFzebONgVbD3Oh7wa%2Bk5eVxta8KYUVg%40mail.gmail.com?utm_medium=email&utm_source=footer>
>> .
>> _______________________________________________
>> Lisa_teatalk mailing list
>> Lisa_teatalk at iro.umontreal.ca
>> https://webmail.iro.umontreal.ca/mailman/listinfo/lisa_teatalk
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "brain-montreal-office-hours" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to brain-montreal-office-hours+unsubscribe at google.com.
>> To post to this group, send email to
>> brain-montreal-office-hours at google.com.
>> To view this discussion on the web visit
>> https://groups.google.com/a/google.com/d/msgid/brain-montreal-office-hours/CAMVr0Vy4KsyukWMA_7jfFzebONgVbD3Oh7wa%2Bk5eVxta8KYUVg%40mail.gmail.com
>> .
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "brain-mon-fte" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to brain-mon-fte+unsubscribe at google.com.
>> To post to this group, send email to brain-mon-fte at google.com.
>> To view this discussion on the web visit
>> https://groups.google.com/a/google.com/d/msgid/brain-mon-fte/CAMVr0Vy4KsyukWMA_7jfFzebONgVbD3Oh7wa%2Bk5eVxta8KYUVg%40mail.gmail.com
>> .
>>
>
>
> --
> Nicolas Le Roux
>
-------------- section suivante --------------
Une pièce jointe HTML a été nettoyée...
URL: http://webmail.iro.umontreal.ca/pipermail/lisa_seminaires/attachments/20180522/369234cb/attachment.html 


Plus d'informations sur la liste de diffusion Lisa_seminaires