April 2018 - Lisa_seminaires - mailman.iro.umontreal.ca

[Tea Talk] ICLR Lightning Talks! (MILA) Fri Apr 27 10:30AM AA1360
by Michael Noukhovitch 27 Apr '18

27 Apr '18

This week we have something special. *MILA students who have papers accepted at ICLR * will each be giving a 5-10 minutes talk on their work this *Friday Apr 27* at *10:30AM* in room *AA1360*. This should be fun and light, giving everyone a chance to see what your friends and colleagues have been up to, as well as congratulate them on their submissions! Make sure not to miss these electrifying talks! Michael Here's the list of papers being presented but there are even more (wow!) that made it into ICLR that aren't being presented *Fraternal Dropout* Konrad Zolna · Devansh Arpit · Dendi Suhubdy · Yoshua Bengio *Residual Connections Encourage Iterative Inference* Stanislaw Jastrzebski · Devansh Arpit · Nicolas Ballas · Vikas Verma · Tong Che · Yoshua Bengio *Deep Complex Networks* Chiheb Trabelsi · Olexa Bilaniuk · Ying Zhang · Dmitriy Serdyuk · Sandeep Subramanian · Joao Felipe Santos · Soroush Mehri · Negar Rostamzadeh · Yoshua Bengio · Christopher Pal *MaskGAN: Better Text Generation via Filling in the _______* William Fedus · Ian Goodfellow · Andrew Dai *Many Paths to Equilibrium: GANs Do Not Need to Decrease a Divergence At Every Step* William Fedus · Mihaela Rosca · Balaji Lakshminarayanan · Andrew Dai · Shakir Mohamed · Ian Goodfellow *Boundary Seeking GANs* R Devon Hjelm · Athul P Jacob · Adam Trischler · Tong Che · Kyunghyun Cho · Yoshua Bengio *Parametric Adversarial Divergences are Good Task Losses for Generative Modeling* Gabriel Huang · Hugo Berard · Ahmed Touati · Gauthier Gidel · Pascal Vincent · Simon Lacoste-Julien *Twin Networks: Matching the Future for Sequence Generation* Dmitriy Serdyuk · Nan Rosemary Ke · Alessandro Sordoni · Adam Trischler · Christopher Pal · Yoshua Bengio *ChatPainter: Improving Text to Image Generation using Dialogue* Shikhar Sharma · Dendi Suhubdy · Vincent Michalski · Samira Ebrahimi Kahou · Yoshua Bengio *FigureQA: An Annotated Figure Dataset for Visual Reasoning* Samira Ebrahimi Kahou · Vincent Michalski · Adam Atkinson · Ákos Kádár · Adam Trischler · Yoshua Bengio *Learning General Purpose Distributed Sentence Representations via Large Scale Multi-task Learning** Sandeep Subramanian · Adam Trischler · Yoshua Bengio · Christopher Pal *Learning Awareness Models** Brandon Amos · Laurent Dinh · Serkan Cabi · Thomas Rothörl · Sergio Gómez Colmenarejo · Alistair M Muldal · Tom Erez · Yuval Tassa · Nando d Freitas · Misha Denil *slides shown for presenters in absentia

1 1

[Extra Tea Talk] Masashi Sugiyama (RIKEN / U Tokyo) Thurs Apr 26 10:30AM AA3195
by Michael Noukhovitch 27 Apr '18

27 Apr '18

Next week we have an extra tea talk presented by *Masashi Sugiyama* from *RIKEN and University of Tokyo* giving a talk on *Thursday April 26* at *10:30AM* in room *AA3195* *note the different room* Want to meet Prof. Sugiyama? Sign up on the doodle <https://doodle.com/poll/b48c5gru3e89vpmc> We've only had great talks and I'm positive this one will be great too! Michael *TITLE* Machine learning from weak supervision - Towards accurate classification with low labeling costs. *KEYWORDS *semi-supervised learning, learning with biased data *ABSTRACT*Recent advances in machine learning with big labeled data allow us to achieve human-level performance in various tasks such as speech recognition, image understanding, and natural language translation. On the other hand, there are still many application domains where human labor is involved in the data acquisition process and thus the use of massive labeled data is prohibited. In this talk, I will introduce our recent advances in classification techniques from weak supervision, including classification from positive and unlabeled data, a novel approach to semi-supervised classification, classification from positive-confidence data, and classification from complementary labels *BIO* Masashi Sugiyama received the PhD degree in Computer Science from Tokyo Institute of Technology, Japan in 2001. He has been Professor at the University of Tokyo since 2014 and concurrently appointed as Director of RIKEN Center for Advanced Intelligence Project in 2016. His research interests include theory, algorithms, and applications of machine learning. He (co)-authored several books such as Density Ratio Estimation in Machine Learning (Cambridge University Press, 2012), Machine Learning in Non-Stationary Environments (MIT Press, 2012), Statistical Reinforcement Learning (Chapman and Hall, 2015), and Introduction to Statistical Machine Learning (Morgan Kaufmann, 2015). He served as a Program Co-chair and General Co-chair for the Neural Information Processing Systems conference in 2015 and 2016, respectively, and he will be a Program Co-chair for AISTATS2019. Masashi Sugiyama received the Japan Society for the Promotion of Science Award and the Japan Academy Medal in 2017.

2 2

Re: [Lisa_seminaires] [Extra Tea Talk] Masashi Sugiyama (RIKEN / U Tokyo) Thurs Apr 26 10:30AM AA3195
by Michael Noukhovitch 26 Apr '18

26 Apr '18

Reminder this is in 15 minutes! Steam is always here: https://bluejeans.com/809027115/webrtc On Thu, Apr 19, 2018, 10:53 Michael Noukhovitch <mnoukhov(a)gmail.com> wrote: > Next week we have an extra tea talk presented by *Masashi Sugiyama* from *RIKEN > and University of Tokyo* giving a talk on *Thursday April 26* at *10:30AM* > in room *AA3195* *note the different room* > > Want to meet Prof. Sugiyama? Sign up on the doodle > <https://doodle.com/poll/b48c5gru3e89vpmc> > > We've only had great talks and I'm positive this > Michael > > *TITLE* Machine learning from weak supervision - Towards accurate > classification with low labeling costs. > > *KEYWORDS *semi-supervised learning, learning with biased data > > > *ABSTRACT*Recent advances in machine learning with big labeled data allow > us to > achieve human-level performance in various tasks such as speech > recognition, image understanding, and natural language translation. On > the other hand, there are still many application domains where human > labor is involved in the data acquisition process and thus the use of > massive labeled data is prohibited. In this talk, I will introduce > our recent advances in classification techniques from weak > supervision, including classification from positive and unlabeled > data, a novel approach to semi-supervised classification, > classification from positive-confidence data, and classification from > complementary labels > > *BIO* > Masashi Sugiyama received the PhD degree in Computer Science from > Tokyo Institute of Technology, Japan in 2001. He has been Professor at > the University of Tokyo since 2014 and concurrently appointed as > Director of RIKEN Center for Advanced Intelligence Project in 2016. > His research interests include theory, algorithms, and applications of > machine learning. He (co)-authored several books such as Density Ratio > Estimation in Machine Learning (Cambridge University Press, 2012), > Machine Learning in Non-Stationary Environments (MIT Press, 2012), > Statistical Reinforcement Learning (Chapman and Hall, 2015), and > Introduction to Statistical Machine Learning (Morgan Kaufmann, 2015). > He served as a Program Co-chair and General Co-chair for the Neural > Information Processing Systems conference in 2015 and 2016, > respectively, and he will be a Program Co-chair for AISTATS2019. > Masashi Sugiyama received the Japan Society for the Promotion of > Science Award and the Japan Academy Medal in 2017. >

1 0

[Tea Talk] Ankit Anand (IIT Delhi) Fri Apr 20 10:30AM AA1360
by Michael Noukhovitch 20 Apr '18

20 Apr '18

This week we have *Ankit Anand* from *IIT Delhi* giving a talk on *Friday April 20* at *10:30AM* in room *AA1360*. ¡ʞlɐʇ ǝɥʇ oʇ ǝɯoɔ Michael *TITLE* Symmetry Aware Decision Making and Inference *KEYWORDS *AI Planning, Graphical Models, Image Segmentation/Stereovision *ABSTRACT* Many traditional AI algorithms fail to scale as the problem size increases exponentially with the number of features. This work focuses on using symmetries and abstractions to reduce computation and memory overhead in state-of-the-art AI and ML problems. Specifically, we exploit symmetries in sequential decision making under uncertainty (Monte Carlo Tree Search algorithms like UCT) and probabilistic inference. Abstractions and symmetries though well studied in the past have mostly been used as a preprocessing step. Our work focuses to learn symmetries and abstractions from scratch within the algorithm itself and use the learned abstractions and symmetries for further speeding up the state-of-the-art algorithms. Specifically, a) We propose ASAP-UCT and OGA-UCT which uses novel notion of Abstractions in Monte Carlo Tree Search Algorithms like UCT b) We apply symmetry aware graphical model inference in close to state-of-the-art Stereovision and Image Segmentation algorithms to gain impressive speed-ups without loss in quality c) We propose novel notion of symmetries -"Contextual Symmetries" and "Non-Count Symmetries" and use those for faster mixing in MCMC algorithms Moreover, our strategies are computationally efficient and provide gains up to 25% in International Probabilistic Planning Competition(IPPC) domains while speeding up MAP inference in Graphical Models by 10x for Stereovision and Segmentation without much loss in quality. *BIO* Ankit Anand <http://www.cse.iitd.ernet.in/~ankitanand/> is a PhD Student at Computer Science and Engineering department of IIT Delhi. He is advised by Dr. Parag Singla <http://www.cse.iitd.ac.in/~parags/> and Dr. Mausam <http://homes.cs.washington.edu/~mausam/>. His primary research interests are Artificial Intelligence Planning, Reinforcement Learning and Probabilistic Graphical Models. He has worked extensively on applications of Image Segmentation, Stereovision and Sequential Decision Making tasks. He is a recipient of Tata Consultancy Services Ph.D Fellowship from 2013-2016 and has published in top tier conferences like IJCAI, AAMAS, ICAPS and AISTATS. He is also a winner of best paper award at StarAI workshop at IJCAI'16.

2 3

[Tea Talk] Guillaume Lajoie (UdeM) Fri April 13 10:30AM AA1360
by Michael Noukhovitch 17 Apr '18

17 Apr '18

This week we have *Guillaume Lajoie* from *DMS *at *UdeM *giving a talk on *Friday April 13* at *10:30AM* in room *AA1360*. Make sure to come to this dynamic talk and your research will spike! Michael *TITLE *Dynamics of high-dimensional recurrent networks: how chaos shapes computation in biological and artificial neural networks *KEYWORDS *Dynamic Systems Analysis, Neuroscience, DL Theory *ABSTRACT*Networks of neurons —either biological or artificial— are called recurrent if their connections are distributed and contain feedback loops. Such networks can perform remarkably complex computations, as evidenced by their ubiquity throughout the brain and ever-increasing use in machine learning. They are, however, notoriously hard to control and their dynamics are generally poorly understood, especially in the presence of external forcing. This is because recurrent networks are typically chaotic systems, meaning they have rich and sensitive dynamics leading to variable responses to inputs. How does this chaos manifest in the neural code of the brain? How might we tame sensitivity to exploit complexity when training artificial recurrent networks for machine learning? Understanding how the dynamics of large driven networks shape their capacity to encode and process information presents a sizeable challenge. In this talk, I will discuss the use of Random Dynamical Systems Theory as a framework to study information processing in high-dimensional, signal-driven networks. I will present an overview of recent results linking chaotic attractors to entropy production, dimensionality and input discrimination of dynamical observables. I will outline insights this theory provides on how cortex performs complex computations using sparsely connected inhibitory and excitatory neurons, as well as implications for gradient-based optimization methods for artificial networks. *BIO*Since Jan 2018, Guillaume Lajoie is an Assistant Professor at the Math & Stats Dept. of UdeM. He obtained his PhD in applied mathematics from the University of Washington in 2013 and carried on research as an independent Bernstein Fellow at the Max Planck Institute for Dynamics and Self-Organization, as a visiting Scholar at the Courant Institute for Mathematical Sciences at NYU, and as a Washington Research Foundation Innovation Fellow at the University of Washington's Institute for Neuroengineering. His research interests lie at the intersection of applied mathematics, neuroscience and AI, where he works at leveraging tools from Dynamical Systems, Stochastic Processes, Information Theory and Machine Learning to better understand how recurrent network dynamics --either biological or artificial-- support computations. Applications of his research range from contributions to a theory of computation in recurrent networks, to the development of brain-computer interfaces.

2 2

[Tea Talk] Mailing List Change
by Michael Noukhovitch 16 Apr '18

16 Apr '18

*** Français ci-dessous *** Starting next week, MILA tea talks will not be sent to this mailing list ( lisa_seminaires(a)iro.umontreal.ca) anymore. If you're not part of MILA and you'd like to keep receiving MILA tea talk emails please subscribe to the lisa_teatalks mailing list by emailing lisa_teatalk-join(a)iro.umontreal.ca Thanks! Michael ---------------- À partir de la semaine prochaine, les courriels relatifs aux MILA tea talks ne seront plus envoyés à cette adresse (lisa_seminaires(a)iro.umontreal.ca) Si vous ne faites pas partie du MILA mais voudriez continuer à recevoir ces courriels, veuillez vous inscrire sur la liste lisa_teatalk en envoyant un courriel à lisa_teatalk-join(a)iro.umontreal.ca Merci! Michael

1 0

Fwd: Conférence de Yoshua Bengio - lundi 16 avril - Programme du 50e anniversaire du CRM - Les mathématique de l'apprentissage machine
by Yoshua Bengio 10 Apr '18

10 Apr '18

I will give an opening keynote lecture next Monday, Rm 1360, as part of the CRM 50th anniversary: ---------- Forwarded message ---------- From: Centre de recherches mathematiques <crm(a)crm.umontreal.ca> Date: 2018-04-10 15:18 GMT-04:00 Subject: Conférence de Yoshua Bengio - lundi 16 avril - Programme du 50e anniversaire du CRM - Les mathématique de l'apprentissage machine To: Activités CRM <activites(a)crm.umontreal.ca> ****************************************************************** Programme du 50e anniversaire du CRM Les mathématique de l'apprentissage machine 14 avril - 11 mai 2018 / April 14 - May 11, 2018 50th anniversary program Mathematics of machine learning ****************************************************************** Conférence inaugurale / Opening keynote lecture lundi 16 avril / Monday, April 16 11:30 - 12:30 Université de Montréal, Pavillon André-Aisenstadt, salle / room 1360 Yoshua Bengio (Université de Montréal) "Deep Learning for AI" There has been rather impressive progress recently with brain-inspired statistical learning algorithms based on the idea of learning multiple levels of representation, also known as neural networks or deep learning. They shine in artificial intelligence tasks involving perception and generation of sensory data like images or sounds and to some extent in understanding and generating natural language. We have proposed new generative models which lead to training frameworks very different from the traditional maximum likelihood framework, and borrowing from game theory. Theoretical understanding of the success of deep learning is work in progress but relies on representation aspects as well as optimization aspects, which interact. At the heart is the ability of these learning mechanisms to capitalize on the compositional nature of the underlying data distributions, meaning that some functions can be represented exponentially more efficiently with deep distributed networks compared to approaches like standard non-parametric methods which lack both depth and distributed representations. On the optimization side, we now have evidence that local minima (due to the highly non-convex nature of the training objective) may not be as much of a problem as thought a few years ago, and that training with variants of stochastic gradient descent actually helps to quickly find better-generalizing solutions. Finally, new interesting questions and answers are arising regarding learning theory for deep networks, why even very large networks do not necessarily overfit and how the representation-forming structure of these networks may give rise to better error bounds which do not absolutely depend on the iid data hypothesis. ****************************************************************** http://www.crm.math.ca/crm50/ ******************************************************************

1 0

[Tea Talk] Brady Neal (MILA) Fri Apr 4 10:30AM AA1360
by Michael Noukhovitch 06 Apr '18

06 Apr '18

This week we have our very own *Brady Neal* giving a talk this *Friday* at *10:30AM* in room *AA1360*. I may be a little bit biased, but coming to this talk will be at least one standard deviation better than your usual Friday! Michael *TITLE *Towards Understanding Generalization in Deep Learning by Revisiting the Bias-Variance Decomposition *KEYWORDS *DL Theory, ML Theory *ABSTRACT*Generalization is at the very core of machine learning. The bias-variance decomposition in machine learning is an underused lens to view generalization through. While it’s more common to derive upper bounds on the generalization gap via more complicated measures of complexity such as the VC dimension and Rademacher complexity, the bias-variance decomposition is an *equality* that is noticeably simpler. Looking through this lens, we can quickly get to partial explanations for why larger neural networks seem to generalize better than their smaller counterparts, despite the fact that bounds based on VC dimension and Rademacher complexity suggest the opposite. We appeal to some of the blessings of high dimensionality to do this. While Zhang et al. (2017) were quite surprised by the results of their experiments in “Understanding deep learning requires rethinking generalization,” they would have been much less surprised if they were looking at their results through the lens of the bias-variance decomposition. *BIO*Brady was an intern at MILA and is now a Masters student at MILA with Ioannis Mitliagkas. He organizes the DL Theory Reading Group and is currently focused on generalization and optimization in deep learning

2 3