- Lisa_teatalk - mailman.iro.umontreal.ca

[Tea Talk] Martin Gilbert (IVADO) Fri May 11 2018 10:30 AM AA1360
by Michael Noukhovitch 08 Mai '18

08 Mai '18

This week we have our very own, Martin Gilbert from IVADO giving a follow-up on his previous Intro to Ethics talk on *Friday May 11 2018* at *10:30 AM* in room *AA1360* Come to this talk because you'll learn and improve your life, it's your duty as MILA students, and because going to tea talks would be very gracious of you! Michael *TITLE* Intro to Ethics 2 *ABSTRACT* In this presentation, I will introduce to the three main families of normative moral theories: consequentialism, deontologism and virtue ethics.

1 0

[Tea Talk] is cancelled for ICLR
by Michael Noukhovitch 02 Mai '18

02 Mai '18

No tea talk this week! Instead find someone to talk to about tea!

1 0

[Tea Talk] ICLR Lightning Talks! (MILA) Fri Apr 27 10:30AM AA1360
by Michael Noukhovitch 27 Avr '18

27 Avr '18

This week we have something special. *MILA students who have papers accepted at ICLR * will each be giving a 5-10 minutes talk on their work this *Friday Apr 27* at *10:30AM* in room *AA1360*. This should be fun and light, giving everyone a chance to see what your friends and colleagues have been up to, as well as congratulate them on their submissions! Make sure not to miss these electrifying talks! Michael Here's the list of papers being presented but there are even more (wow!) that made it into ICLR that aren't being presented *Fraternal Dropout* Konrad Zolna · Devansh Arpit · Dendi Suhubdy · Yoshua Bengio *Residual Connections Encourage Iterative Inference* Stanislaw Jastrzebski · Devansh Arpit · Nicolas Ballas · Vikas Verma · Tong Che · Yoshua Bengio *Deep Complex Networks* Chiheb Trabelsi · Olexa Bilaniuk · Ying Zhang · Dmitriy Serdyuk · Sandeep Subramanian · Joao Felipe Santos · Soroush Mehri · Negar Rostamzadeh · Yoshua Bengio · Christopher Pal *MaskGAN: Better Text Generation via Filling in the _______* William Fedus · Ian Goodfellow · Andrew Dai *Many Paths to Equilibrium: GANs Do Not Need to Decrease a Divergence At Every Step* William Fedus · Mihaela Rosca · Balaji Lakshminarayanan · Andrew Dai · Shakir Mohamed · Ian Goodfellow *Boundary Seeking GANs* R Devon Hjelm · Athul P Jacob · Adam Trischler · Tong Che · Kyunghyun Cho · Yoshua Bengio *Parametric Adversarial Divergences are Good Task Losses for Generative Modeling* Gabriel Huang · Hugo Berard · Ahmed Touati · Gauthier Gidel · Pascal Vincent · Simon Lacoste-Julien *Twin Networks: Matching the Future for Sequence Generation* Dmitriy Serdyuk · Nan Rosemary Ke · Alessandro Sordoni · Adam Trischler · Christopher Pal · Yoshua Bengio *ChatPainter: Improving Text to Image Generation using Dialogue* Shikhar Sharma · Dendi Suhubdy · Vincent Michalski · Samira Ebrahimi Kahou · Yoshua Bengio *FigureQA: An Annotated Figure Dataset for Visual Reasoning* Samira Ebrahimi Kahou · Vincent Michalski · Adam Atkinson · Ákos Kádár · Adam Trischler · Yoshua Bengio *Learning General Purpose Distributed Sentence Representations via Large Scale Multi-task Learning** Sandeep Subramanian · Adam Trischler · Yoshua Bengio · Christopher Pal *Learning Awareness Models** Brandon Amos · Laurent Dinh · Serkan Cabi · Thomas Rothörl · Sergio Gómez Colmenarejo · Alistair M Muldal · Tom Erez · Yuval Tassa · Nando d Freitas · Misha Denil *slides shown for presenters in absentia

1 1

[Extra Tea Talk] Masashi Sugiyama (RIKEN / U Tokyo) Thurs Apr 26 10:30AM AA3195
by Michael Noukhovitch 27 Avr '18

27 Avr '18

Next week we have an extra tea talk presented by *Masashi Sugiyama* from *RIKEN and University of Tokyo* giving a talk on *Thursday April 26* at *10:30AM* in room *AA3195* *note the different room* Want to meet Prof. Sugiyama? Sign up on the doodle <https://doodle.com/poll/b48c5gru3e89vpmc> We've only had great talks and I'm positive this one will be great too! Michael *TITLE* Machine learning from weak supervision - Towards accurate classification with low labeling costs. *KEYWORDS *semi-supervised learning, learning with biased data *ABSTRACT*Recent advances in machine learning with big labeled data allow us to achieve human-level performance in various tasks such as speech recognition, image understanding, and natural language translation. On the other hand, there are still many application domains where human labor is involved in the data acquisition process and thus the use of massive labeled data is prohibited. In this talk, I will introduce our recent advances in classification techniques from weak supervision, including classification from positive and unlabeled data, a novel approach to semi-supervised classification, classification from positive-confidence data, and classification from complementary labels *BIO* Masashi Sugiyama received the PhD degree in Computer Science from Tokyo Institute of Technology, Japan in 2001. He has been Professor at the University of Tokyo since 2014 and concurrently appointed as Director of RIKEN Center for Advanced Intelligence Project in 2016. His research interests include theory, algorithms, and applications of machine learning. He (co)-authored several books such as Density Ratio Estimation in Machine Learning (Cambridge University Press, 2012), Machine Learning in Non-Stationary Environments (MIT Press, 2012), Statistical Reinforcement Learning (Chapman and Hall, 2015), and Introduction to Statistical Machine Learning (Morgan Kaufmann, 2015). He served as a Program Co-chair and General Co-chair for the Neural Information Processing Systems conference in 2015 and 2016, respectively, and he will be a Program Co-chair for AISTATS2019. Masashi Sugiyama received the Japan Society for the Promotion of Science Award and the Japan Academy Medal in 2017.

2 2

Re: [Lisa_teatalk] [Extra Tea Talk] Masashi Sugiyama (RIKEN / U Tokyo) Thurs Apr 26 10:30AM AA3195
by Michael Noukhovitch 26 Avr '18

26 Avr '18

Reminder this is in 15 minutes! Steam is always here: https://bluejeans.com/809027115/webrtc On Thu, Apr 19, 2018, 10:53 Michael Noukhovitch <mnoukhov(a)gmail.com> wrote: > Next week we have an extra tea talk presented by *Masashi Sugiyama* from *RIKEN > and University of Tokyo* giving a talk on *Thursday April 26* at *10:30AM* > in room *AA3195* *note the different room* > > Want to meet Prof. Sugiyama? Sign up on the doodle > <https://doodle.com/poll/b48c5gru3e89vpmc> > > We've only had great talks and I'm positive this > Michael > > *TITLE* Machine learning from weak supervision - Towards accurate > classification with low labeling costs. > > *KEYWORDS *semi-supervised learning, learning with biased data > > > *ABSTRACT*Recent advances in machine learning with big labeled data allow > us to > achieve human-level performance in various tasks such as speech > recognition, image understanding, and natural language translation. On > the other hand, there are still many application domains where human > labor is involved in the data acquisition process and thus the use of > massive labeled data is prohibited. In this talk, I will introduce > our recent advances in classification techniques from weak > supervision, including classification from positive and unlabeled > data, a novel approach to semi-supervised classification, > classification from positive-confidence data, and classification from > complementary labels > > *BIO* > Masashi Sugiyama received the PhD degree in Computer Science from > Tokyo Institute of Technology, Japan in 2001. He has been Professor at > the University of Tokyo since 2014 and concurrently appointed as > Director of RIKEN Center for Advanced Intelligence Project in 2016. > His research interests include theory, algorithms, and applications of > machine learning. He (co)-authored several books such as Density Ratio > Estimation in Machine Learning (Cambridge University Press, 2012), > Machine Learning in Non-Stationary Environments (MIT Press, 2012), > Statistical Reinforcement Learning (Chapman and Hall, 2015), and > Introduction to Statistical Machine Learning (Morgan Kaufmann, 2015). > He served as a Program Co-chair and General Co-chair for the Neural > Information Processing Systems conference in 2015 and 2016, > respectively, and he will be a Program Co-chair for AISTATS2019. > Masashi Sugiyama received the Japan Society for the Promotion of > Science Award and the Japan Academy Medal in 2017. >

1 0

[Tea Talk] Ankit Anand (IIT Delhi) Fri Apr 20 10:30AM AA1360
by Michael Noukhovitch 20 Avr '18

20 Avr '18

This week we have *Ankit Anand* from *IIT Delhi* giving a talk on *Friday April 20* at *10:30AM* in room *AA1360*. ¡ʞlɐʇ ǝɥʇ oʇ ǝɯoɔ Michael *TITLE* Symmetry Aware Decision Making and Inference *KEYWORDS *AI Planning, Graphical Models, Image Segmentation/Stereovision *ABSTRACT* Many traditional AI algorithms fail to scale as the problem size increases exponentially with the number of features. This work focuses on using symmetries and abstractions to reduce computation and memory overhead in state-of-the-art AI and ML problems. Specifically, we exploit symmetries in sequential decision making under uncertainty (Monte Carlo Tree Search algorithms like UCT) and probabilistic inference. Abstractions and symmetries though well studied in the past have mostly been used as a preprocessing step. Our work focuses to learn symmetries and abstractions from scratch within the algorithm itself and use the learned abstractions and symmetries for further speeding up the state-of-the-art algorithms. Specifically, a) We propose ASAP-UCT and OGA-UCT which uses novel notion of Abstractions in Monte Carlo Tree Search Algorithms like UCT b) We apply symmetry aware graphical model inference in close to state-of-the-art Stereovision and Image Segmentation algorithms to gain impressive speed-ups without loss in quality c) We propose novel notion of symmetries -"Contextual Symmetries" and "Non-Count Symmetries" and use those for faster mixing in MCMC algorithms Moreover, our strategies are computationally efficient and provide gains up to 25% in International Probabilistic Planning Competition(IPPC) domains while speeding up MAP inference in Graphical Models by 10x for Stereovision and Segmentation without much loss in quality. *BIO* Ankit Anand <http://www.cse.iitd.ernet.in/~ankitanand/> is a PhD Student at Computer Science and Engineering department of IIT Delhi. He is advised by Dr. Parag Singla <http://www.cse.iitd.ac.in/~parags/> and Dr. Mausam <http://homes.cs.washington.edu/~mausam/>. His primary research interests are Artificial Intelligence Planning, Reinforcement Learning and Probabilistic Graphical Models. He has worked extensively on applications of Image Segmentation, Stereovision and Sequential Decision Making tasks. He is a recipient of Tata Consultancy Services Ph.D Fellowship from 2013-2016 and has published in top tier conferences like IJCAI, AAMAS, ICAPS and AISTATS. He is also a winner of best paper award at StarAI workshop at IJCAI'16.

2 3

[Tea Talk] Guillaume Lajoie (UdeM) Fri April 13 10:30AM AA1360
by Michael Noukhovitch 18 Avr '18

18 Avr '18

This week we have *Guillaume Lajoie* from *DMS *at *UdeM *giving a talk on *Friday April 13* at *10:30AM* in room *AA1360*. Make sure to come to this dynamic talk and your research will spike! Michael *TITLE *Dynamics of high-dimensional recurrent networks: how chaos shapes computation in biological and artificial neural networks *KEYWORDS *Dynamic Systems Analysis, Neuroscience, DL Theory *ABSTRACT*Networks of neurons —either biological or artificial— are called recurrent if their connections are distributed and contain feedback loops. Such networks can perform remarkably complex computations, as evidenced by their ubiquity throughout the brain and ever-increasing use in machine learning. They are, however, notoriously hard to control and their dynamics are generally poorly understood, especially in the presence of external forcing. This is because recurrent networks are typically chaotic systems, meaning they have rich and sensitive dynamics leading to variable responses to inputs. How does this chaos manifest in the neural code of the brain? How might we tame sensitivity to exploit complexity when training artificial recurrent networks for machine learning? Understanding how the dynamics of large driven networks shape their capacity to encode and process information presents a sizeable challenge. In this talk, I will discuss the use of Random Dynamical Systems Theory as a framework to study information processing in high-dimensional, signal-driven networks. I will present an overview of recent results linking chaotic attractors to entropy production, dimensionality and input discrimination of dynamical observables. I will outline insights this theory provides on how cortex performs complex computations using sparsely connected inhibitory and excitatory neurons, as well as implications for gradient-based optimization methods for artificial networks. *BIO*Since Jan 2018, Guillaume Lajoie is an Assistant Professor at the Math & Stats Dept. of UdeM. He obtained his PhD in applied mathematics from the University of Washington in 2013 and carried on research as an independent Bernstein Fellow at the Max Planck Institute for Dynamics and Self-Organization, as a visiting Scholar at the Courant Institute for Mathematical Sciences at NYU, and as a Washington Research Foundation Innovation Fellow at the University of Washington's Institute for Neuroengineering. His research interests lie at the intersection of applied mathematics, neuroscience and AI, where he works at leveraging tools from Dynamical Systems, Stochastic Processes, Information Theory and Machine Learning to better understand how recurrent network dynamics --either biological or artificial-- support computations. Applications of his research range from contributions to a theory of computation in recurrent networks, to the development of brain-computer interfaces.

2 2

[Tea Talk] Brady Neal (MILA) Fri Apr 4 10:30AM AA1360
by Michael Noukhovitch 07 Avr '18

07 Avr '18

This week we have our very own *Brady Neal* giving a talk this *Friday* at *10:30AM* in room *AA1360*. I may be a little bit biased, but coming to this talk will be at least one standard deviation better than your usual Friday! Michael *TITLE *Towards Understanding Generalization in Deep Learning by Revisiting the Bias-Variance Decomposition *KEYWORDS *DL Theory, ML Theory *ABSTRACT*Generalization is at the very core of machine learning. The bias-variance decomposition in machine learning is an underused lens to view generalization through. While it’s more common to derive upper bounds on the generalization gap via more complicated measures of complexity such as the VC dimension and Rademacher complexity, the bias-variance decomposition is an *equality* that is noticeably simpler. Looking through this lens, we can quickly get to partial explanations for why larger neural networks seem to generalize better than their smaller counterparts, despite the fact that bounds based on VC dimension and Rademacher complexity suggest the opposite. We appeal to some of the blessings of high dimensionality to do this. While Zhang et al. (2017) were quite surprised by the results of their experiments in “Understanding deep learning requires rethinking generalization,” they would have been much less surprised if they were looking at their results through the lens of the bias-variance decomposition. *BIO*Brady was an intern at MILA and is now a Masters student at MILA with Ioannis Mitliagkas. He organizes the DL Theory Reading Group and is currently focused on generalization and optimization in deep learning

2 3

No [Tea Talk] this Friday
by Michael Noukhovitch 27 Mar '18

27 Mar '18

For those of you who can't imagine how a Friday can be good without a research talk, I recommend watching this talk <https://www.youtube.com/watch?v=sT_-owjKIbA> on *How to Give a Great Research Talk* by Microsoft's *Simon Peyton Jones*. I think the talk is quite good and has some great ideas for when you all come to do your own tea talk (sign up for summer tea talks by emailing me!) Have a great Friday! Michael

1 0

[Tea Talk] Alex Ororbia (Penn State) Fri Mar 23 10:30AM AA1360
by Michael Noukhovitch 23 Mar '18

23 Mar '18

This week we have *Alexander G. Ororbia II* from *Penn State* giving a talk on *Friday March 23* at *10:30AM* in room *AA1360*. *Want to meet with Alex?* Fill out this Doodle <https://doodle.com/poll/6nkharwxw8ya26ys> and we'll find a spot! This talk should present some new ideas that will really challenge us to never stop learning, so you should definitely come! Michael *TITLE *Adaptation in the Face of Evolving Distributions: Towards Continual Learning of Connectionist Architectures *KEYWORDS *continual/lifelong/meta learning, optimization, biological-inspired RNN *ABSTRACT*It is a common statistical learning practice to build models on (very large) static datasets of identically and independently distributed samples. But what if the distribution to be learned is dynamic and samples are drawn from it over time? In this setting, these traditional learning approaches no longer directly apply. Motivated by this issue, we must look to continual, or lifelong learning, and the nature of systems that adapt themselves to such distributions. With the goal of creating robust and scalable continual learning systems, my work can be decomposed into two main threads: 1) creating architectures with longer-term memory and 2) developing better learning algorithms. In this talk, I will focus on the second thread, specifically on my efforts in developing alternatives to back-propagation of errors. Specifically, I will present, Local Representation Alignment (LRA), a training procedure that is much less sensitive to bad initializations, does not require modifications to the network architecture, can be readily adapted to networks with highly nonlinear and discrete-valued activation functions and stochastic sampling operations, and can even train networks with various kinds of activation functions from zero initialization. Results on classification benchmarks will be discussed and recent results in the lifelong learning setting, where the model must learn across tasks, will be presented. I will finally talk about my proposed neural architecture, the Temporal Neural Coding Network (TNCN), which builds on concepts from predictive coding, an important theory of the brain that has gained influence in cognitive science. Using the proposed Discrepancy Reduction learning procedure, which can be viewed as a special variation of LRA, the TNCN can learn from sequential data, such as videos or text documents. More importantly, initial results show promise that the TNCN can be competitive with models that learn using back-propagation through time--the popular, biologically implausible algorithm used to "unroll" and train recurrent neural networks. *BIO* Alex is currently a Ph.D. student at The Pennsylvania State University in Information Sciences & Technology. In 2013, he obtained his Bachelor of Science degree in Computer Science & Engineering at Bucknell University, minoring in Philosophy and Mathematics. The focus of his work is on lifelong learning--an important and challenging open problem in machine learning. He studies representation learning and draws inspiration from ideas in cognitive science and neuroscience. Alex's mission is to develop the learning procedures and architectures needed to create general-purpose, adaptive agents that can learn from data of multiple modalities and operate in messy, non-stationary environments.

1 1