[Lisa_seminaires] [Lisa_teatalk] [Tea Talk] Nicolas Le Roux (Google) May 25 2018 10:30 AM AA3195

Nicolas Le Roux nlr at google.com
Mar 22 Mai 12:45:40 EDT 2018


For those of you who attended my talk at the Workshop on learning theory,
this one will fill in some of the gaps I mentioned.

On Tue, 22 May 2018 at 12:17, Michael Noukhovitch <mnoukhov at gmail.com>
wrote:

> This week we will have two tea talks (morning and afternoon) on Friday!
> Wow!
>
> The first talk will be *Nicolas Le* *Roux * on *May 25 2018* at *10:30 AM*
> in room *AA3195* (note the room change, we'll most likely be in 3195 for
> the summer).
>
> Streaming is, as always, at this link:
> https://bluejeans.com/809027115/webrtc but watch it live because there
> will be no recordings!
>
> Don't waver on going to this talk, it is sure to be invariantly excellent!
> Michael
>
> *TITLE* An exploration of variance reduction techniques in stochastic
> optimization
>
> *KEYWORDS *deep learning theory, optimization
>
> *ABSTRACT*
> I will present recent and ongoing work on reducing the variance in
> stochastic optimization techniques to speed-up and simplify the resulting
> algorithms. In particular, stochastic gradient methods can suffer from high
> variance, limiting their convergence speed. While variance reduction
> techniques exist in the finite case, they are rarer in the online case. We
> demonstrate how an increasing momentum offers variance reduction in the
> online case, at the expense of bias, and how that bias can be countered by
> an extrapolation step. The resulting algorithm differs from iterate
> averaging in only a factor, but, in the context of the minimization of a
> quadratic function, this difference is enough to lead to the first
> algorithm converging both linearly in the noiseless and sublinearly in the
> homoscedastic noise case when using a constant stepsize.
>
> *BIO*
> *Nicolas Le Roux got an MSc in Applied Maths from Ecole Centrale Paris and
> an MSc in Maths, Learning and Vision from ENS Cachan. He got his PhD in
> 2008 from University of Montreal where he worked with Yoshua Bengio on
> neural networks in general and their optimisation in particular. He then
> moved to Microsoft Research Cambridge to work on generative models of
> images with John Winn. In 2010, he joined Inria in Francis Bach's team to
> work on large-scale convex optimisation. From 2012 to 2017, he created and
> managed the research team at Criteo in Paris. He joined Google Brain
> Montreal in 2017 where he now works on large-scale optimization and
> reinforcement learning.*
>
> --
> You received this message because you are subscribed to the Google Groups
> "brain-montreal-office-hours" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to brain-montreal-office-hours+unsubscribe at google.com.
> To post to this group, send email to
> brain-montreal-office-hours at google.com.
> To view this discussion on the web visit
> https://groups.google.com/a/google.com/d/msgid/brain-montreal-office-hours/CAMVr0Vy4KsyukWMA_7jfFzebONgVbD3Oh7wa%2Bk5eVxta8KYUVg%40mail.gmail.com
> <https://groups.google.com/a/google.com/d/msgid/brain-montreal-office-hours/CAMVr0Vy4KsyukWMA_7jfFzebONgVbD3Oh7wa%2Bk5eVxta8KYUVg%40mail.gmail.com?utm_medium=email&utm_source=footer>
> .
>
> --
> You received this message because you are subscribed to the Google Groups
> "brain-mon-fte" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to brain-mon-fte+unsubscribe at google.com.
> To post to this group, send email to brain-mon-fte at google.com.
> To view this discussion on the web visit
> https://groups.google.com/a/google.com/d/msgid/brain-mon-fte/CAMVr0Vy4KsyukWMA_7jfFzebONgVbD3Oh7wa%2Bk5eVxta8KYUVg%40mail.gmail.com
> <https://groups.google.com/a/google.com/d/msgid/brain-mon-fte/CAMVr0Vy4KsyukWMA_7jfFzebONgVbD3Oh7wa%2Bk5eVxta8KYUVg%40mail.gmail.com?utm_medium=email&utm_source=footer>
> .
> _______________________________________________
> Lisa_teatalk mailing list
> Lisa_teatalk at iro.umontreal.ca
> https://webmail.iro.umontreal.ca/mailman/listinfo/lisa_teatalk
>
> --
> You received this message because you are subscribed to the Google Groups
> "brain-montreal-office-hours" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to brain-montreal-office-hours+unsubscribe at google.com.
> To post to this group, send email to
> brain-montreal-office-hours at google.com.
> To view this discussion on the web visit
> https://groups.google.com/a/google.com/d/msgid/brain-montreal-office-hours/CAMVr0Vy4KsyukWMA_7jfFzebONgVbD3Oh7wa%2Bk5eVxta8KYUVg%40mail.gmail.com
> .
>
> --
> You received this message because you are subscribed to the Google Groups
> "brain-mon-fte" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to brain-mon-fte+unsubscribe at google.com.
> To post to this group, send email to brain-mon-fte at google.com.
> To view this discussion on the web visit
> https://groups.google.com/a/google.com/d/msgid/brain-mon-fte/CAMVr0Vy4KsyukWMA_7jfFzebONgVbD3Oh7wa%2Bk5eVxta8KYUVg%40mail.gmail.com
> .
>


-- 
Nicolas Le Roux
-------------- section suivante --------------
Une pièce jointe HTML a été nettoyée...
URL: http://webmail.iro.umontreal.ca/pipermail/lisa_seminaires/attachments/20180522/fb0921c4/attachment-0001.html 


Plus d'informations sur la liste de diffusion Lisa_seminaires