Lisa_seminaires Février 2018

lisa_seminaires@iro.umontreal.ca

3 participants
11 discussions

[DIRO Talk] Devon Hjelm (MILA) Fri Mar 2 10:30AM AA1360
by Michael Noukhovitch 02 Mar '18

02 Mar '18

For our last special DIRO talk, we have our very own *Devon Hjelm* giving a talk on *Friday March 2* at *10:30AM* in room *AA1360*. This talk has generated a lot of interest, so come take a GANder and judge for yourself if it's the real deal! Michael *TITLE *Research in Generative Adversarial Learning *KEYWORDS* Unsupervised learning, generative models, representation learning, adversarial learning, mutual information *ABSTRACT* Since their inception, generative adversarial networks (GANs) have emerged as a state-of-the-art approach for generating high-dimensional continuous data. As the field has grown, so has its applications, with the underlying principles of GANs having been extended into numerous problems in unsupervised learning. This talk will cover the basic principles of GANs, as well as briefly summarize our three recent works that cover different types of unsupervised problems using these principles: 1) Discrete generation in GANs (Boundary seeking GANs, ICLR 2018): Training GANs with discrete data (e.g., natural language as realized as a sequence of character or word tokens) is normally not possible due to being unable to backprop. We introduce a principled approach for training GANs with discrete data that draws from likelihood ratio estimation, importance sampling, and policy gradients. 2) Learning richer representations in bidirectional adversarial models (GibbsNet, NIPS 2017). Undirected graphical models (e.g., RBMs, DBMs) can provide richer representations than those available from directed graphical models (e.g., VAEs). We draw from ideas in undirected graphical models to formulate an adversarial model that learns a representation that is richer than competing adversarial models. 3) Neural mutual information estimation (MINE, in review): Mutual information is notoriously difficult to compute, especially in the high-dimensional continuous setting. We introduce a general-purpose neural estimator for mutual information which is scalable, flexible, and completely trainable via back-prop. *BIO* R Devon Hjelm earned his PhD at the University of New Mexico under the supervision of Vince Calhoun at the Mind Research Network, a research institute dedicated to neuro-diagnostic discovery. Prior to this, he acquired a Master’s degree in Physics (with a focus on Quantum Information) and Linguistics. He joined the Montréal Institute for Learning Algorithms (MILA) at the University of Montréal in January 2017 as an IVADO “distinguished researcher” postdoctoral fellow under Yoshua Bengio. There, his research focus became adversarial learning (GANs), notably applying ideas from GANs to solving a broader class of unsupervised learning problems. Ultimately, he is primarily interested in research on training an agent that is able to reason about the natural world from evidence and communicate its understanding to humans. *PHOTO* *(real sample, not generated)* *[image: CD2F92BF-5584-473C-BE37-5D616C37B293.png]*

1 1

[Tea Talk] Maximilian Puelma Touzel (ENS Paris) Thur Mar 1 10:30AM AA1360
by Michael Noukhovitch 01 Mar '18

01 Mar '18

This week we have *Maximilian Puelma Touzel *from* Ecole Normale Supérieure in Paris *giving a talk on *Thursday March 1* at *10:30AM *in room *AA1360*. **New* *If you'd like to meet with the presenter, choose which time slots work for you here (https://doodle.com/poll/2mxtkei7bnznht82) and we'll try to make it work! Guaranteed to be more fun than unboxing, come to the talk and be illuminated! Michael *TITLE *Don’t paint the box black: Using dynamical systems to understand the high-dimensional activity of cortex-inspired neural networks *KEYWORDS *biological NNs, learning dynamics, systems analysis, statistical physics *ABSTRACT*The dynamics of cortical circuits are thought to be responsible for much of the learning in higher vertebrates. Over the last decades, a data-driven theoretical effort in neuroscience has sought to understand the collective dynamics of 'task-ready' cortical activity. In this talk, I will first recount the work that led up to the first self-consistent mean-field theory of such activity in 2010. Employing a long-standing framework from dynamical systems and statistical physics suited for characterizing information processing in high-dimensional dynamics, subsequent work has focussed on understanding the properties of the underlying full spiking network dynamics. I will describe some of these approaches, and present original work that analytically treats the complex phase space geometry of a multi-stable regime exhibited by these models. The outlook for this general framework looks bright as we more broadly seek to elucidate high-dimensional learning dynamics in a variety of systems, both biological and machine. *BIO* Maximilian Puelma Touzel is a theoretical biophysicist interested in understanding how learning systems build and adapt representations of their environment. He studied math and physics at the University of Toronto and then theoretical neuroscience at the Max Planck Institute for Dynamics and Self-Organization in Göttingen, Germany, receiving a PhD there under the supervision of Fred Wolf. Currently, he is a post-doctoral fellow working on inference of probabilistic models of immune repertoire change with Aleksandra Walczak and Thierry Mora in the Laboratoire de Physique Théorique at the Ecole Normale Supérieure in Paris.

1 1

[DIRO Talk] Jacob Andreas (UC Berkeley) Mon Feb 26 11AM Z240
by Michael Noukhovitch 27 Fév '18

27 Fév '18

Kicking off next week's special DIRO talks we have *Jacob Andreas* from *UC Berkeley* giving a talk on *Monday Feb 26* at *11:00AM* in room *Claire-McNicoll Z240*. You should be able to learn quite a bit from this presentation just by listening! Michael *TITLE *Learning from Language *KEYWORDS *NLP, compositional/modular learning, reinforcement learning, interpretable ML *ABSTRACT*Natural language is built from a library of concepts and compositional operators that provide a rich source of information about how humans understand the world. Can this information help us build better machine learning models? In this talk, we'll explore three ways of integrating compositional linguistic structure and learning: using language as a source of modular reasoning operators for question answering, as a scaffold for fast and generalizable reinforcement learning, and as a tool for understanding representations in neural networks. *BIO* Jacob Andreas is a fifth-year PhD student at UC Berkeley working in natural language processing. He holds a B.S. from Columbia and an M.Phil. from Cambridge, where he studied as a Churchill scholar. His papers were recognized at NAACL 2016 and ICML 2017. Jacob has been an NSF graduate fellow, a Huawei--Berkeley AI fellow, and a Facebook fellow.

1 2

Fwd: Autonomous Weapons talk by Jean-Baptiste Jeangène Vilmer (French Ministry of Defense) Fri Mar 2 3PM Z345
by Michael Noukhovitch 26 Fév '18

26 Fév '18

CRE and Martin Gilbert are hosting a talk on the subject of autonomous weapons from a different perspective this Friday for all those interested (note that the talk will be given in english but the QA afterwards is bilingual). The speaker, *Jean-Baptiste Jeangène Vilmer*, is the director of* Institute for Strategic Research* (IRSEM, French Ministry of Defense). It will be this *Friday March 2* at *3:00PM* in Claire-McNicoll *Z-345*. ABSTRACT Autonomous weapon systems – known as “killer robots” in popular culture – are weapon systems that can select and attack targets without human involvement. While they do not yet exist, they are already being discussed before the UN, as some NGOs and states have begun demanding a preventive ban against such weapons. This presentation will expose the difficulty of defining them (the notion of autonomy, the typology of more or less autonomous weapons, the notion of “meaningful human control”), the diplomatic debate going on at the UN and the underlying moral debate (responding to deontologist and consequentialist objections to autonomous weapons). After such consideration, this presentation ultimately opposes a preventive ban and proposes, instead, a code of conduct. BIO A Graduate of the Université de Montréal and McGill University, Dr Jean-Baptiste Jeangène Vilmer is the director of the Institute for Strategic Research (IRSEM, French Ministry of Defense). Previously a member of the Policy Planning Staff (Ministry of Foreign Affairs), he was a member of the French delegation at the UN Meeting of Experts on Lethal Autonomous Weapons Systems in 2014, 2015 and 2016. Last article on the topic: “Autonomous Weapon Diplomacy: The Geneva Debates”, Ethics & International Affairs, Online Exclusive, 27 September 2016 ( https://www.ethicsandinternationalaffairs.org/2016/autonomous-weapon-diplom… )

1 0

[DIRO Talk] Guillaume Rabusseau (McGill) Tue Feb 27 11AM Z317
by Michael Noukhovitch 23 Fév '18

23 Fév '18

Our second DIRO talk for next week will be *Guillaume Rabusseau*, currently a postdoc at *McGill*, giving a talk on *Tuesday Feb 27* at *11:00AM* in room *Claire-McNicoll Z317* Structure your schedules around this talk, it should be intense! Michael *TITLE *Machine Learning with Tensors for Structured Data *KEYWORDS* Structured/Graph Learning, Weighted Automata, Multilinear Algebra, ML Theory *ABSTRACT*Over the last few years, the machine learning community has shown a growing interest towards tensors. The most obvious connection between tensors and machine learning appears when the data has a natural tensor structure (e.g. videos, hyperspectral images...) but tensors can also arise as model parameters or as powerful tools to design efficient and consistent learning algorithms. In this talk, I will show how tensor methods can be leveraged for learning with structured data. I will first consider the case of tensor structured data and show how taking the tensor structure into account, rather than vectorizing the data before feeding it to the learning algorithm, can lead to better computational and generalization performances. The second part of the talk will be focused on learning with discrete structured data such as sequences, trees and graphs. I will first argue that classical computational models such as weighted automata are particularly suited to represent functions defined over syntactic objects in the context of machine learning. After outlining the fundamental connections between weighted automata, tensors, and more classical learning models such as recurrent neural networks, I will present recent contributions showcasing the relevance of weighted automata for learning with structured data. To conclude the talk, I will give an overall perspective on how multilinear algebra and tensor methods can be beneficially used in machine learning by presenting future research directions. *BIO*Guillaume Rabusseau is an IVADO postdoctoral research fellow in the Reasoning and Learning Lab at McGill University, where he works with Prakash Panangaden, Joelle Pineau and Doina Precup. His research interests lie at the intersection of theoretical computer science and machine learning. He obtained his PhD in computer science in 2016 at Aix-Marseille University under the supervision of François Denis and Hachem Kadri. His work revolves around exploring inter-connections between tensors and machine learning and developing efficient learning methods for structured data relying on linear and multilinear algebra.

1 0

[DIRO Talk] Ina Fiterau (Stanford) Fri Feb 23 10:30AM AA1360
by Michael Noukhovitch 23 Fév '18

23 Fév '18

For this week's last DIRO talk, we have *Ina Fiterau*, a post-doc from *Stanford* giving a talk on *Friday Feb 23* at *10:30AM* in room *AA1360*. This talk should combine a variety of different modal topics in a seamless way, which sounds bloody exciting! Michael *TITLE *Hybrid Machine Learning Methods for the Interpretation and Integration of Heterogeneous Multimodal Data *KEYWORDS *Biomedical ML, Deep + Shallow Learning, Human-in-the-loop, Mulit-modal *ABSTRACT*The prevalence of smartphones and wearable devices and the widespread use of electronic health records have led to a surge in multimodal health data that is noisy, non-uniform, and collected at an unprecedented scale. This talk focuses on machine learning models that learn expressive representations of multimodal, heterogeneous data for biomedical predictive models designed to interact with domain experts. In the first part of the talk, the focus is on techniques for partitioning data and leveraging low-dimensional structure to enable visualization and annotation by humans. The latter part addresses the construction of hybrid models that combine deep learning with random forests, and the fusing of structured information into temporal representation learning. This array of methods obviates the need for feature engineering while improving on the state of the art for diverse biomedical applications. Use cases include the classification of alerts in a vital sign monitoring system, the prediction of surgical outcomes in children with cerebral palsy, and forecasting the progression of osteoarthritis from subjects' physical activity. *BIO* Madalina Fiterau is a Postdoctoral Fellow in the Computer Science Department at Stanford University, working with Professors Chris Ré and Scott Delp in the Mobilize Center. She has obtained a PhD in Machine Learning from Carnegie Mellon University in September 2015, advised by Professor Artur Dubrawski. The focus of her PhD thesis, entitled “Discovering Compact and Informative Structures through Data Partitioning”, is learning interpretable ensembles, with applicability ranging from image classification to clinical alert prediction. Madalina is currently expanding her research on interpretable models, in part by applying deep learning to obtain salient representations from biomedical “deep” data, including time series, text and images. Madalina is the recipient of the Marr Prize for Best Paper at ICCV 2015 and of Star Research Award at the Annual Congress of the Society of Critical Care Medicine 2016. She has co-organized NIPS workshops on the topic of Machine Learning in Healthcare in 2013, 2014, 2016 and 2017.

1 1

[Tea Talk] Bradly Stadie (UC Berkeley) Wed Feb 21 3:00PM AA1360
by Michael Noukhovitch 21 Fév '18

21 Fév '18

Sorry for the lateness, please note this talk is a normal tea talk and it is *tomorrow!* If you want to meet with the speaker, please email me and I'll set it up! This week we have *Bradly Stadie* from *UC Berkeley* giving a talk on *Wednesday Feb 21* at *3:00PM* in room *AA1360*. See you there! Michael *TITLE* Towards Quickly Learning Agents *KEYWORDS *Meta-learning, Reinforcement Learning, Imitation Learning *ABSTRACT*We consider the problems of meta reinforcement learning, wherein the goal is to train agents that can learn new tasks quickly. We suggest 5 major sub-problems that are crucial for enabling quickly learning agents: Exploration, Imitation, Hierarchy/Planning, Credit Assignment, and Control. The talk will examine the sub-problems of exploration and imitation and their interplay with quickly learning agents in more depth. In particular, we examine: 1) Third Person Imitation: which enables agents to imitate other agents that differ significantly from themselves. 2) One Shot Imitation: which enables a robot to see a single video of a demonstration and copy the demonstrated behavior. 3) E-MAML: a meta-RL algorithm that achieves significantly better exploration than previous meta-RL algorithms. *BIO* Bradly Stadie is a PhD student at UC Berkeley in Pieter Abbeel’s lab. Previously, he spent 2 years working at Open AI under Ilya Sutskever. Bradly’s work has focused on the fields of exploration, imitation, and meta learning with applications to games, robotics, and causal inference. Prior to his interest in AI, Bradly studied mathematics at the University of Chicago, where his work focused on representation theory of locally compact groups and stochastic differential equations.

2 2

[DIRO Talk] Pieter-Jan Kindermans (Google/Ghent) Thur Feb 22 11:00AM AA1355
by Michael Noukhovitch 20 Fév '18

20 Fév '18

Continuing our special DIRO talk week, we have *Pieter-Jan Kindermans* from *Ghent University and Google Brain* giving a talk on *Thursday Feb 22* at *11:00AM* in room *AA1355*. As always, email me if you'd be interested in speaking with him! This talk should be good because I've heard the speaker is great at explaining ML! Michael *TITLE *Towards a reliably understanding and visualization of deep neural networks *KEYWORDS *Explainable ML, Testing Deep Learning, Deep Learning Theory *ABSTRACT*Deep learning has transformed the field of machine learning. Empirically these methods work brilliantly but it is difficult to understand what exactly they have learned. If this were possible, we would have a perfect understanding of our networks. The community has spend quite a lot of effort on visualising what a neural network learns. However while many methods were invented to better understand deep neural networks, we can show that these methods do not produce the theoretically correct explanation for a linear model, which is a simple neural network. Despite this they are used on multi-layer networks with millions of parameters. We present the idea of creating unit tests for explanations. The idea behind the unit test is that while it might be impossible to define what a good explanation is, it is much easier to detect failure cases. Hence, by creating more and more reliable unit tests, the community can refine its methods iteratively and converge to a good solution. Based on an analysis of linear models we propose a generalization that yields two explanation techniques (PatternNet and PatternAttribution) that are theoretically sound for linear models, pass our unit test and produce improved explanations for deep networks. *BIO*Pieter-Jan Kindermans obtained his PhD degree from Ghent Unviersity in 2014. From 2014 to 2017 he was a postdoc, as a Marie-Curie fellow, in the lab of Klaus-Robert Müller in Berlin. Currently he is a Brain resident at Google. Initially he worked on unsupervised learning for Brain-Computer Interfaces (BCI) and demonstrated that unsupervised learning can replace traditional supervised classifiers. A further extension of this work, in collaboration with the universities of Freiburg and Ghent, ensured that the unsupervised decoder is guarantueed to converge to the optimal supervised solution. This project was nominated for the BCI award in 2017.In addition to his work in BCI, he also explored different subfields of machine learning, including deep learning, large scale training and its applications such as molecular chemistry. His current focus is on better understanding the operation of a deep neural network. His ultimate goal is to make deep learning a knowledge extraction tool for science.

1 0

[DIRO Talk] Marc Law (U of T) Tue Feb 20 11:00AM PCM Z240
by Michael Noukhovitch 20 Fév '18

20 Fév '18

We will have three special DIRO talks next week and kicking it off is *Marc Law*, from University of Toronto giving a talk on *Tuesday Feb 20* at* 11AM* in room *PCM Z240*. *Note that this is on *Tuesday*, and in a different room* Checks all the boxes for a fantastic talk, but you should come see for yourself! Michael *TITLE *Shallow and Deep Metrics for Machine Learning and Computer Vision *KEYWORDS* computer vision, few/zero-shot learning, metric learning *ABSTRACT* Similarity functions and distance metrics are used in many machine learning and computer vision contexts such as clustering, k-nearest neighbors classification, support vector machine, information/image retrieval, visualization etc. Traditionally, machine learning methods fixed sample representations and the used metric before learning a model optimized for the target task. Metric learning approaches, which learn the employed metric in a supervised way, have been proposed to increase performance on tasks such as clustering. In particular, they have shown great generalization performance to compare objects from categories that were not seen during training (for instance in face verification or few-shot learning). In this talk, I will talk about different shallow and deep metric learning approaches optimized for clustering and reducing model complexity. In the clustering task, I will present efficient approaches to learn a metric in a supervised or weakly supervised way. In the model complexity context, I will present approaches to limit the rank of shallow approaches, or reduce the dimensionality of a pretrained deep neural network to perform visualization or increase zero-shot learning performance. *BIO* Marc Law received a PhD in Computer Science from Université Pierre et Marie Curie (Paris, France) in 2015. He was a visiting research scholar in the team of Professor Eric Xing at the School of Computer Science, Carnegie Mellon University in 2015~2016. Since June 2016, he is a postdoctoral fellow in the Department of Computer Science (Machine Learning group) at the University of Toronto under the supervision of Professor Raquel Urtasun and Professor Richard Zemel. During his PhD, he worked mostly on distance metric learning applied to different contexts of computer vision and web archiving. He is currently working on deep learning. His main focus is to propose scalable machine learning methods. He received an award for best PhD from the French Association for Artificial Intelligence in 2016.

2 1

[Tea Talk] Jacob Steinhardt (Stanford) Fri Feb 16 10:30AM AA1360
by Michael Noukhovitch 16 Fév '18

16 Fév '18

This week we have *Jacob Steinhardt* from Stanford giving a special DIRO talk on *Fri Feb 16* at* 10:30AM* in room *AA1360*. Don't be fooled, this talk should be the real deal! Michael *TITLE *Provably Secure Machine Learning *KEYWORDS *Security, Optimization, AI Safety, Adversarial *ABSTRACT* The widespread use of machine learning systems creates a new class of computer security vulnerabilities where, rather than attacking the integrity of the software itself, malicious actors exploit the statistical nature of the learning algorithms. For instance, attackers can add fake data (e.g. by creating fake user accounts), or strategically manipulate inputs to the system once it is deployed. So far, attempts to defend against these attacks have focused on empirical performance against known sets of attacks. I will argue that this is a fundamentally inadequate paradigm for achieving meaningful security guarantees. Instead, we need algorithms that are provably secure by design, in line with best practices for traditional computer security. To achieve this goal, we take inspiration from robust statistics and robust optimization, but with an eye towards the security requirements of modern machine learning systems. Motivated by the trend towards models with thousands or millions of features, we investigate the robustness of learning algorithms in high dimensions. We show that most algorithms are brittle to even small fractions of adversarial data, and then develop new algorithms that are provably robust. Additionally, to accommodate the increasing use of deep learning, we develop an algorithm for certifiably robust optimization of non-convex models such as neural networks. *BIO*Jacob Steinhardt is a graduate student in artificial intelligence at Stanford University working with Percy Liang. His main research interest is in designing machine learning algorithms with the reliability properties of good software. So far this has led to the study of provably secure machine learning systems, as well as the design of learning algorithms that can detect their own failures and generalize predictably in new situations. Outside of research, Jacob is a technical advisor to the Open Philanthropy Project, and mentors gifted high school students through the USACO and SPARC summer programs.

1 1

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

Lisa_seminaires Février 2018