Lisa_seminaires

lisa_seminaires@iro.umontreal.ca

396 discussions

[Tea Talk] Jeffrey Pennington (Google Brain NYC) Fri 29 March 2019 10h30 Mila Auditorium
by Rim Assouel 29 Mar '19

29 Mar '19

This week we have Jeffrey Pennington from Google Brain NYC giving a talk on Are Overparameterized Neural Networks Actually Just Linear Models? at 10h30 in room Mila Auditorium. Will this talk be streamed <https://mila.bluejeans.com/809027115/webrtc>? yes See you there! The Tea Talk Team TITLE Are Overparameterized Neural Networks Actually Just Linear Models? ABSTRACT Neural networks define a rich and expressive class of functions whose properties and behaviors are very hard to describe from a theoretical perspective. Nevertheless, when these functions become highly overparameterized, a surprisingly simple characterization emerges. In this talk, I will discuss several perspectives on this characterization: 1) I will examine the prior over functions induced by common weight initialization schemes and show that it corresponds to a Gaussian process with a well-defined compositional kernel; 2) I will show that by tuning initialization hyperparameters, this kernel can be optimized for signal propagation, yielding networks that are trainable to enormous depths (10k+ layers); and 3) I will demonstrate that the learning dynamics of such overparameterized neural networks are governed by a linear model obtained from the first-order Taylor expansion of the network around its initial parameters. BIO Jeffrey Pennington is a Research Scientist at Google Brain, NYC. Prior to this, he was a postdoctoral fellow at Stanford University, as a member of the Stanford Artificial Intelligence Laboratory in the Natural Language Processing (NLP) group, where he studied the unsupervised learning of word representations. He received his Ph.D. in theoretical particle physics from Stanford University while working at the SLAC National Accelerator Laboratory, with a main focus on the development of calculational techniques in perturbative quantum field theory. Jeffrey’s current research interests center on the theory of deep learning, and include topics such as: trainability and expressivity, the dynamics of learning, the role of overparameterization, stochastic networks and random matrix theory, and the geometry of high-dimensional loss surfaces

2 2

[Invited Talk] Jan van Rijn (Columbia University) Monday March 25 2019 10h00 Mila Auditorium
by Rim Assouel 25 Mar '19

25 Mar '19

Hi everyone, Next week we will have an “extraordinary” invited talk ( organized by Christopher Beckham) on Monday :) The regular tea talk will also be back on Friday ( announcement to come!) See you there, The Tea Talk Team TITLE Automated Machine Learning using OpenML.org <http://openml.org/> ABTRACT Algorithm Selection and Hyperparameter Optimization are two laborious but viable components of the Machine Learning pipeline. The field of Automated Machine Learning (AutoML) focusses, as the name suggests, on automating these processes. Automated Machine Learning builds upon knowledge from previous experiments. In this talk I will introduce OpenML.org <http://openml.org/>, an on line platform containing almost 10 million Machine Learning experiments and 20,000 monthly unique visitors. Furthermore, I will talk about some of the projects that utilize this knowledge, in particular for determining which hyperparameters are most important and how to optimize them based on prior knowledge. BIO Jan van Rijn is founder of the OpenML Foundation, for Open Machine Learning research. He has authored several publications related to Data Science, Automated Machine Learning (AutoML) and Artificial Intelligence. He is currently a post-doctoral researcher at Columbia University in New York. For a complete and up to date list of his publications, please see his Google Scholar profile: https://scholar.google.com/citations?user=O4X5CpwAAAAJ <https://scholar.google.com/citations?user=O4X5CpwAAAAJ>

2 2

[Tea Talk] David Blei (Columbia University) Fri March 8 2019 10h30 Mila Auditorium
by Rim Assouel 06 Mar '19

06 Mar '19

This week we have David Blei from Columbia University giving a talk on The Blessings of Multiple Causes at 10h30 at Mila Auditorium. Will this talk be streamed <https://mila.bluejeans.com/809027115/webrtc>? Yes David will also be visiting Mila on Thursday (7th March). To that end we organized a discussion group on Causal Inference from 2pm to 3pm in room A14. A bunch of you asked for 1-1 and 2-1 meetings : The schedule is also set during that day, I’ll send the invites by tomorrow :) See you there! The Tea Talk Team TITLE The Blessings of Multiple Causes ABSTRACT Causal inference from observational data is a vital problem, but it comes with strong assumptions. Most methods require that we observe all confounders, variables that correlate to both the causal variables (the treatment) and the effect of those variables (how well the treatment works). But whether we have observed all confounders is a famously untestable assumption. We describe the deconfounder, a way to do causal inference from observational data with weaker assumptions that the classical methods require. How does the deconfounder work? While traditional causal methods measure the effect of a single cause on an outcome, many modern scientific studies involve multiple causes, different variables whose effects are simultaneously of interest. The deconfounder uses the multiple causes as a signal for unobserved confounders, combining unsupervised machine learning and predictive model checking to perform causal inference. We describe the theoretical requirements for the deconfounder to provide unbiased causal estimates, and show that it requires weaker assumptions than classical causal inference. We analyze the deconfounder's performance in three types of studies: semi-simulated data around smoking and lung cancer, semi-simulated data around genomewide association studies, and a real dataset about actors and movie revenue. The deconfounder provides a checkable approach to estimating close-to-truth causal effects. This is joint work with Yixin Wang. [*] https://arxiv.org/abs/1805.06826 BIO David Blei is a Professor of Statistics and Computer Science at Columbia University, and a member of the Columbia Data Science Institute. He studies probabilistic machine learning, including its theory, algorithms, and application. David has received several awards for his research, including a Sloan Fellowship (2010), Office of Naval Research Young Investigator Award (2011), Presidential Early Career Award for Scientists and Engineers (2011), Blavatnik Faculty Award (2013), ACM-Infosys Foundation Award (2013), and a Guggenheim fellowship (2017). He is the co-editor-in-chief of the Journal of Machine Learning Research. He is a fellow of the ACM and the IMS. He is a fellow of the ACM and the IMS.

3 3

[NO Tea Talk this week]
by Rim Assouel 07 Fév '19

07 Fév '19

Hi everyone, To let people attend the scheduled job talk, there will be no tea talk this week. Sai, Pablo and Rim

1 0

[Tea Talk] Isabela Albuquerque (INRS) Fri February 1 2019 10h30 Mila Auditorium
by Rim Assouel 01 Fév '19

01 Fév '19

This week we have Isabela Albuquerque from INRS giving a talk on Multi-objective training of Generative Adversarial Networks with multiple discriminators at 10h30 in room Mila Auditorium. Will this talk be streamed <https://mila.bluejeans.com/809027115/webrtc>? yes PS : Slides and recordings of previous talks are available at :https://sites.google.com/lisa.iro.umontreal.ca/tea-talk-recordings/home?authuser=0 <https://sites.google.com/lisa.iro.umontreal.ca/tea-talk-recordings/home?aut…> See you there! Pablo, Sai and Rim TITLE Multi-objective training of Generative Adversarial Networks with multiple discriminators ABSTRACT Recent literature has demonstrated promising results for training Generative Adversarial Networks by employing a set of discriminators, in contrast to the traditional game involving one generator against a single adversary. Such methods perform single-objective optimization on some simple consolidation of the losses, e.g. an average. In this work, we revisit the multiple-discriminator setting by framing the simultaneous minimization of losses provided by different models as a multi-objective optimization problem. Specifically, we evaluate the performance of multiple gradient descent and the hypervolume maximization algorithm on a number of different datasets. Moreover, we argue that the previously proposed methods and hypervolume maximization can all be seen as variations of multiple gradient descent in which the update direction can be computed efficiently. Our results indicate that hypervolume maximization presents a better compromise between sample quality and computational cost than previous methods. BIO Isabela Albuquerque is a PhD candidate at the Institut National de la Recherche Scientifique (INRS), Université du Québec. Her main research interests are machine learning and cognitive neuroscience. The primary goal of her current research is improving the generalization across different individuals of electroencephalogram-based mental workload models. Prior to joining INRS, she received a Bachelor's degree in Electrical Engineering from the Federal University of Pernambuco and a Master's degree in Computer Engineering from the University of Pernambuco, both in Recife, Brazil.

2 2

[Tea Talk] Hugo Larochelle (Mila + Brain) Fri January 25 2019 10h30 Mila Auditorium
by Rim Assouel 25 Jan '19

25 Jan '19

This week we have Hugo Larochelle from Mila + Brain giving a talk on (Part II) Few-Shot Learning with Meta-Learning: Progress Made and Challenges Ahead at 10h30 in room Mila Auditorium. Will this talk be streamed <https://mila.bluejeans.com/809027115/webrtc>? yes Feel like learning to learn again ? Hugo will teach you another way, this Friday, one shot at a time ;) See you there! Pablo, Sai and Rim Tea Talks website : https://sites.google.com/lisa.iro.umontreal.ca/tea-talk-recordings <https://sites.google.com/lisa.iro.umontreal.ca/tea-talk-recordings> TITLE (Part II) Few-Shot Learning with Meta-Learning: Progress Made and Challenges Ahead ABSTRACT A lot of the recent progress on many AI tasks was enable in part by the availability of large quantities of labeled data. Yet, humans are able to learn concepts from as little as a handful of examples. Meta-learning is a very promising framework for addressing the problem of generalizing from small amounts of data, known as few-shot learning. In meta-learning, our model is itself a learning algorithm: it takes as input a training set and outputs a classifier. For few-shot learning, it is (meta-)trained directly to produce classifiers with good generalization performance for problems with very little labeled data. In this talk, I'll present an overview of the recent research that has made exciting progress on this topic (including my own) and will discuss the challenges as well as research opportunities that remain. BIO Hugo Larochelle is Research Scientist at Google Brain and lead of the Montreal Google Brain team. He is also a member of Yoshua Bengio's Mila and an Adjunct Professor at the Université de Montréal. Previously, he was Associate Professor at the University of Sherbrooke. He also co-founded Whetlab, which was acquired in 2015 by Twitter, where he then worked as a Research Scientist in the Twitter Cortex group. From 2009 to 2011, he was also a member of the machine learning group at the University of Toronto, as a postdoctoral fellow under the supervision of Geoffrey Hinton. He obtained his Ph.D. at the Université de Montréal, under the supervision of Yoshua Bengio. He has the best hair in machine learning academia (not his words). Finally, he has a popular online course on deep learning and neural networks, freely accessible on YouTube.

2 1

[No Tea Talk][ICML deadline]
by Rim Assouel 14 Jan '19

14 Jan '19

There will be no Tea Talk this week because of the ICML deadline coming soon. Next talk will be on the 25th for Hugo Larochelle’s part 2 on Few-Shot Learning with Meta-Learning. See you next week, Pablo, Sai and Rim

1 0

[Tea Talk] [New Website]Maxime Chevalier-Boisvert (Mila) Fri January 11 2019 10h30 Mila Auditorium
by Rim Assouel 11 Jan '19

11 Jan '19

This week we have Maxime Chevalier-Boisvert from Mila giving a talk on BabyAI: A Platform to Study the Sample Efficiency of Grounded Language Learning at 10h30 in room Mila Auditorium. Will this talk be streamed <https://mila.bluejeans.com/809027115/webrtc>? yes We are also pleased to share the new « website » where we will keep track of the talks slides/recordings : https://sites.google.com/lisa.iro.umontreal.ca/tea-talk-recordings/home?aut… <https://sites.google.com/lisa.iro.umontreal.ca/tea-talk-recordings/home?aut…> It’s accessible with your Mila google account :) See you there and Happy New Year! Pablo, Sai and Rim TITLE BabyAI: A Platform to Study the Sample Efficiency of Grounded Language Learning ABSTRACT This talk will present BabyAI, a research project based at Mila with the long-term goal of creating agents which we can communicate with and teach new concepts using natural language. We will begin by discussing why language learning is a hard problem, one to which deep learning doesn't yet have a satisfying answer for. We will then introduce the BabyAI platform, created to study the sample efficiency of grounded language learning in the context of embodied agents and instruction following (see paper accepted at ICLR19, https://openreview.net/forum?id=rJeXCo0cYX). Lastly, we will discuss multiple promising research directions we have identified with the goal of improving the sample efficiency of grounded language learning. BIO Maxime has completed a PhD in computer science at the Université de Montréal in early 2016. She has since worked at Apple as part of the mobile GPU team, and then joined Mila as part of the research engineering staff. At Mila, she has been co-directing the BabyAI project with Dzmitry Bahdanau, and took part in creating the Duckietown Simulator. Her main research interests are robotics, embodied agents and reinforcement learning.

3 3

Re: [Lisa_seminaires] [TODAY][Tea Talk] Praneeth (MSR India) Thursday December 13 2018 15:30 AA3195
by Rim Assouel 13 Déc '18

13 Déc '18

Reminder : this happens in 1h ! > Le 11 déc. 2018 à 17:20, rim.assouel(a)gmail.com a écrit : > > This week we have Praneeth from MSR India giving a talk on Thursday December 13 2018 at 15:30 in room AA3195 > > Will this talk be streamed <https://mila.bluejeans.com/4255239897/webrtc>? Yes > > Pay attention to the day change !! This tea talk will happen on THURSDAY :) > > As it will be the last tea talk of the year, it will be featured with actual tea, talk and snacks :) > > See you there! > Rim and Sai > > TITLE On momentum methods and acceleration in stochastic optimization > > ABSTRACT > > It is well known that momentum gradient methods (e.g., Polyak's heavy ball, Nesterov's acceleration) yield significant improvements over vanilla gradient descent in deterministic optimization (i.e., where we have access to exact gradient of the function to be minimized). However, there is widespread sentiment that these momentum methods are not effective for the purposes of stochastic optimization due to their instability and error accumulation. Numerous works have attempted to quantify these instabilities in the face of either statistical or non-statistical errors (Paige, 1971; Proakis, 1974; Polyak, 1987; Greenbaum, 1989; Roy and Shynk, 1990; Sharma et al., 1998; d’Aspremont, 2008; Devolder et al., 2013, 2014; Yuan et al., 2016) but a precise understanding is lacking. This work considers these issues for the special case of stochastic approximation for the linear least squares regression problem, and shows that: > > 1. classical momentum methods (heavy ball and Nesterov's acceleration) indeed do not offer any improvement over stochastic gradient descent, and > 2. introduces an accelerated stochatic gradient method that provably achieves the minimax optimal statistical risk faster than stochastic gradient descent (and classical momentum methods). > > Critical to the analysis is a sharp characterization of accelerated stochastic gradient descent as a stochastic process. While the results are rigorously established for the special case of linear least squares regression, experiments suggest that the conclusions hold for the training of deep neural networks. > BIO > Praneeth Netrapalli is a researcher at Microsoft Research India, Bengaluru since August 2016. Prior to this, he was a postdoctoral researcher at Microsoft Research New England in Cambridge, MA. He obtained MS and PhD from UT Austin and B-Tech from IIT Bombay all in Electrical Engineering. His research focuses on designing efficient algorithms for machine learning problems primarily via stochastic and nonconvex optimization. More information about his research is available on his home page http://praneethnetrapalli.org/

2 1

[DAY CHANGE][Tea Talk] Praneeth (MSR India) Thursday December 14 2018 15:30 AA3195
by rim.assouel＠gmail.com 12 Déc '18

12 Déc '18

This week we have Praneeth from MSR India giving a talk on Thursday December 13 2018 at 15:30 in room AA3195 Will this talk be streamed <https://mila.bluejeans.com/4255239897/webrtc>? Yes Pay attention to the day change !! This tea talk will happen on THURSDAY :) As it will be the last tea talk of the year, it will be featured with actual tea, talk and snacks :) See you there! Rim and Sai TITLE On momentum methods and acceleration in stochastic optimization ABSTRACT It is well known that momentum gradient methods (e.g., Polyak's heavy ball, Nesterov's acceleration) yield significant improvements over vanilla gradient descent in deterministic optimization (i.e., where we have access to exact gradient of the function to be minimized). However, there is widespread sentiment that these momentum methods are not effective for the purposes of stochastic optimization due to their instability and error accumulation. Numerous works have attempted to quantify these instabilities in the face of either statistical or non-statistical errors (Paige, 1971; Proakis, 1974; Polyak, 1987; Greenbaum, 1989; Roy and Shynk, 1990; Sharma et al., 1998; d’Aspremont, 2008; Devolder et al., 2013, 2014; Yuan et al., 2016) but a precise understanding is lacking. This work considers these issues for the special case of stochastic approximation for the linear least squares regression problem, and shows that: 1. classical momentum methods (heavy ball and Nesterov's acceleration) indeed do not offer any improvement over stochastic gradient descent, and 2. introduces an accelerated stochatic gradient method that provably achieves the minimax optimal statistical risk faster than stochastic gradient descent (and classical momentum methods). Critical to the analysis is a sharp characterization of accelerated stochastic gradient descent as a stochastic process. While the results are rigorously established for the special case of linear least squares regression, experiments suggest that the conclusions hold for the training of deep neural networks. BIO Praneeth Netrapalli is a researcher at Microsoft Research India, Bengaluru since August 2016. Prior to this, he was a postdoctoral researcher at Microsoft Research New England in Cambridge, MA. He obtained MS and PhD from UT Austin and B-Tech from IIT Bombay all in Electrical Engineering. His research focuses on designing efficient algorithms for machine learning problems primarily via stochastic and nonconvex optimization. More information about his research is available on his home page http://praneethnetrapalli.org/

3 2

Aller à la page :

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

Lisa_seminaires