- Lisa_teatalk - mailman.iro.umontreal.ca

[Tea Talk] Maximilian Puelma Touzel (ENS Paris) Thur Mar 1 10:30AM AA1360
by Michael Noukhovitch 01 Mar '18

01 Mar '18

This week we have *Maximilian Puelma Touzel *from* Ecole Normale Supérieure in Paris *giving a talk on *Thursday March 1* at *10:30AM *in room *AA1360*. **New* *If you'd like to meet with the presenter, choose which time slots work for you here (https://doodle.com/poll/2mxtkei7bnznht82) and we'll try to make it work! Guaranteed to be more fun than unboxing, come to the talk and be illuminated! Michael *TITLE *Don’t paint the box black: Using dynamical systems to understand the high-dimensional activity of cortex-inspired neural networks *KEYWORDS *biological NNs, learning dynamics, systems analysis, statistical physics *ABSTRACT*The dynamics of cortical circuits are thought to be responsible for much of the learning in higher vertebrates. Over the last decades, a data-driven theoretical effort in neuroscience has sought to understand the collective dynamics of 'task-ready' cortical activity. In this talk, I will first recount the work that led up to the first self-consistent mean-field theory of such activity in 2010. Employing a long-standing framework from dynamical systems and statistical physics suited for characterizing information processing in high-dimensional dynamics, subsequent work has focussed on understanding the properties of the underlying full spiking network dynamics. I will describe some of these approaches, and present original work that analytically treats the complex phase space geometry of a multi-stable regime exhibited by these models. The outlook for this general framework looks bright as we more broadly seek to elucidate high-dimensional learning dynamics in a variety of systems, both biological and machine. *BIO* Maximilian Puelma Touzel is a theoretical biophysicist interested in understanding how learning systems build and adapt representations of their environment. He studied math and physics at the University of Toronto and then theoretical neuroscience at the Max Planck Institute for Dynamics and Self-Organization in Göttingen, Germany, receiving a PhD there under the supervision of Fred Wolf. Currently, he is a post-doctoral fellow working on inference of probabilistic models of immune repertoire change with Aleksandra Walczak and Thierry Mora in the Laboratoire de Physique Théorique at the Ecole Normale Supérieure in Paris.

1 1

Attention : Courriel Frauduleux
by Julie Nocq 28 Fév '18

28 Fév '18

Bonjour, Un courriel frauduleux circule actuellement sur les boites du DIRO. Ne cliquez sur aucun lien. Si vous l'avez fait, contactez moi. Message du courriel : Nous sommes en train de mettre à niveau tous les comptes de webmail de iro.umontreal.ca, tous les comptes inutilisés seront supprimés, suivez le lien ci-dessous pour confirmer si votre compte est toujours actif. -- Julie Nocq Analyste - programmeuse Administratrice de système Pavillon André-Aisenstadt - DIRO Bureau 3221 514-343-6111 Poste 3497

1 0

[DIRO Talk] Jacob Andreas (UC Berkeley) Mon Feb 26 11AM Z240
by Michael Noukhovitch 27 Fév '18

27 Fév '18

Kicking off next week's special DIRO talks we have *Jacob Andreas* from *UC Berkeley* giving a talk on *Monday Feb 26* at *11:00AM* in room *Claire-McNicoll Z240*. You should be able to learn quite a bit from this presentation just by listening! Michael *TITLE *Learning from Language *KEYWORDS *NLP, compositional/modular learning, reinforcement learning, interpretable ML *ABSTRACT*Natural language is built from a library of concepts and compositional operators that provide a rich source of information about how humans understand the world. Can this information help us build better machine learning models? In this talk, we'll explore three ways of integrating compositional linguistic structure and learning: using language as a source of modular reasoning operators for question answering, as a scaffold for fast and generalizable reinforcement learning, and as a tool for understanding representations in neural networks. *BIO* Jacob Andreas is a fifth-year PhD student at UC Berkeley working in natural language processing. He holds a B.S. from Columbia and an M.Phil. from Cambridge, where he studied as a Churchill scholar. His papers were recognized at NAACL 2016 and ICML 2017. Jacob has been an NSF graduate fellow, a Huawei--Berkeley AI fellow, and a Facebook fellow.

1 2

Fwd: Autonomous Weapons talk by Jean-Baptiste Jeangène Vilmer (French Ministry of Defense) Fri Mar 2 3PM Z345
by Michael Noukhovitch 26 Fév '18

26 Fév '18

CRE and Martin Gilbert are hosting a talk on the subject of autonomous weapons from a different perspective this Friday for all those interested (note that the talk will be given in english but the QA afterwards is bilingual). The speaker, *Jean-Baptiste Jeangène Vilmer*, is the director of* Institute for Strategic Research* (IRSEM, French Ministry of Defense). It will be this *Friday March 2* at *3:00PM* in Claire-McNicoll *Z-345*. ABSTRACT Autonomous weapon systems – known as “killer robots” in popular culture – are weapon systems that can select and attack targets without human involvement. While they do not yet exist, they are already being discussed before the UN, as some NGOs and states have begun demanding a preventive ban against such weapons. This presentation will expose the difficulty of defining them (the notion of autonomy, the typology of more or less autonomous weapons, the notion of “meaningful human control”), the diplomatic debate going on at the UN and the underlying moral debate (responding to deontologist and consequentialist objections to autonomous weapons). After such consideration, this presentation ultimately opposes a preventive ban and proposes, instead, a code of conduct. BIO A Graduate of the Université de Montréal and McGill University, Dr Jean-Baptiste Jeangène Vilmer is the director of the Institute for Strategic Research (IRSEM, French Ministry of Defense). Previously a member of the Policy Planning Staff (Ministry of Foreign Affairs), he was a member of the French delegation at the UN Meeting of Experts on Lethal Autonomous Weapons Systems in 2014, 2015 and 2016. Last article on the topic: “Autonomous Weapon Diplomacy: The Geneva Debates”, Ethics & International Affairs, Online Exclusive, 27 September 2016 ( https://www.ethicsandinternationalaffairs.org/2016/autonomous-weapon-diplom… )

1 0

[DIRO Talk] Guillaume Rabusseau (McGill) Tue Feb 27 11AM Z317
by Michael Noukhovitch 23 Fév '18

23 Fév '18

Our second DIRO talk for next week will be *Guillaume Rabusseau*, currently a postdoc at *McGill*, giving a talk on *Tuesday Feb 27* at *11:00AM* in room *Claire-McNicoll Z317* Structure your schedules around this talk, it should be intense! Michael *TITLE *Machine Learning with Tensors for Structured Data *KEYWORDS* Structured/Graph Learning, Weighted Automata, Multilinear Algebra, ML Theory *ABSTRACT*Over the last few years, the machine learning community has shown a growing interest towards tensors. The most obvious connection between tensors and machine learning appears when the data has a natural tensor structure (e.g. videos, hyperspectral images...) but tensors can also arise as model parameters or as powerful tools to design efficient and consistent learning algorithms. In this talk, I will show how tensor methods can be leveraged for learning with structured data. I will first consider the case of tensor structured data and show how taking the tensor structure into account, rather than vectorizing the data before feeding it to the learning algorithm, can lead to better computational and generalization performances. The second part of the talk will be focused on learning with discrete structured data such as sequences, trees and graphs. I will first argue that classical computational models such as weighted automata are particularly suited to represent functions defined over syntactic objects in the context of machine learning. After outlining the fundamental connections between weighted automata, tensors, and more classical learning models such as recurrent neural networks, I will present recent contributions showcasing the relevance of weighted automata for learning with structured data. To conclude the talk, I will give an overall perspective on how multilinear algebra and tensor methods can be beneficially used in machine learning by presenting future research directions. *BIO*Guillaume Rabusseau is an IVADO postdoctoral research fellow in the Reasoning and Learning Lab at McGill University, where he works with Prakash Panangaden, Joelle Pineau and Doina Precup. His research interests lie at the intersection of theoretical computer science and machine learning. He obtained his PhD in computer science in 2016 at Aix-Marseille University under the supervision of François Denis and Hachem Kadri. His work revolves around exploring inter-connections between tensors and machine learning and developing efficient learning methods for structured data relying on linear and multilinear algebra.

1 0

[DIRO Talk] Ina Fiterau (Stanford) Fri Feb 23 10:30AM AA1360
by Michael Noukhovitch 23 Fév '18

23 Fév '18

For this week's last DIRO talk, we have *Ina Fiterau*, a post-doc from *Stanford* giving a talk on *Friday Feb 23* at *10:30AM* in room *AA1360*. This talk should combine a variety of different modal topics in a seamless way, which sounds bloody exciting! Michael *TITLE *Hybrid Machine Learning Methods for the Interpretation and Integration of Heterogeneous Multimodal Data *KEYWORDS *Biomedical ML, Deep + Shallow Learning, Human-in-the-loop, Mulit-modal *ABSTRACT*The prevalence of smartphones and wearable devices and the widespread use of electronic health records have led to a surge in multimodal health data that is noisy, non-uniform, and collected at an unprecedented scale. This talk focuses on machine learning models that learn expressive representations of multimodal, heterogeneous data for biomedical predictive models designed to interact with domain experts. In the first part of the talk, the focus is on techniques for partitioning data and leveraging low-dimensional structure to enable visualization and annotation by humans. The latter part addresses the construction of hybrid models that combine deep learning with random forests, and the fusing of structured information into temporal representation learning. This array of methods obviates the need for feature engineering while improving on the state of the art for diverse biomedical applications. Use cases include the classification of alerts in a vital sign monitoring system, the prediction of surgical outcomes in children with cerebral palsy, and forecasting the progression of osteoarthritis from subjects' physical activity. *BIO* Madalina Fiterau is a Postdoctoral Fellow in the Computer Science Department at Stanford University, working with Professors Chris Ré and Scott Delp in the Mobilize Center. She has obtained a PhD in Machine Learning from Carnegie Mellon University in September 2015, advised by Professor Artur Dubrawski. The focus of her PhD thesis, entitled “Discovering Compact and Informative Structures through Data Partitioning”, is learning interpretable ensembles, with applicability ranging from image classification to clinical alert prediction. Madalina is currently expanding her research on interpretable models, in part by applying deep learning to obtain salient representations from biomedical “deep” data, including time series, text and images. Madalina is the recipient of the Marr Prize for Best Paper at ICCV 2015 and of Star Research Award at the Annual Congress of the Society of Critical Care Medicine 2016. She has co-organized NIPS workshops on the topic of Machine Learning in Healthcare in 2013, 2014, 2016 and 2017.

1 1

[Tea Talk] Bradly Stadie (UC Berkeley) Wed Feb 21 3:00PM AA1360
by Michael Noukhovitch 21 Fév '18

21 Fév '18

Sorry for the lateness, please note this talk is a normal tea talk and it is *tomorrow!* If you want to meet with the speaker, please email me and I'll set it up! This week we have *Bradly Stadie* from *UC Berkeley* giving a talk on *Wednesday Feb 21* at *3:00PM* in room *AA1360*. See you there! Michael *TITLE* Towards Quickly Learning Agents *KEYWORDS *Meta-learning, Reinforcement Learning, Imitation Learning *ABSTRACT*We consider the problems of meta reinforcement learning, wherein the goal is to train agents that can learn new tasks quickly. We suggest 5 major sub-problems that are crucial for enabling quickly learning agents: Exploration, Imitation, Hierarchy/Planning, Credit Assignment, and Control. The talk will examine the sub-problems of exploration and imitation and their interplay with quickly learning agents in more depth. In particular, we examine: 1) Third Person Imitation: which enables agents to imitate other agents that differ significantly from themselves. 2) One Shot Imitation: which enables a robot to see a single video of a demonstration and copy the demonstrated behavior. 3) E-MAML: a meta-RL algorithm that achieves significantly better exploration than previous meta-RL algorithms. *BIO* Bradly Stadie is a PhD student at UC Berkeley in Pieter Abbeel’s lab. Previously, he spent 2 years working at Open AI under Ilya Sutskever. Bradly’s work has focused on the fields of exploration, imitation, and meta learning with applications to games, robotics, and causal inference. Prior to his interest in AI, Bradly studied mathematics at the University of Chicago, where his work focused on representation theory of locally compact groups and stochastic differential equations.

2 2

[DIRO Talk] Pieter-Jan Kindermans (Google/Ghent) Thur Feb 22 11:00AM AA1355
by Michael Noukhovitch 20 Fév '18

20 Fév '18

Continuing our special DIRO talk week, we have *Pieter-Jan Kindermans* from *Ghent University and Google Brain* giving a talk on *Thursday Feb 22* at *11:00AM* in room *AA1355*. As always, email me if you'd be interested in speaking with him! This talk should be good because I've heard the speaker is great at explaining ML! Michael *TITLE *Towards a reliably understanding and visualization of deep neural networks *KEYWORDS *Explainable ML, Testing Deep Learning, Deep Learning Theory *ABSTRACT*Deep learning has transformed the field of machine learning. Empirically these methods work brilliantly but it is difficult to understand what exactly they have learned. If this were possible, we would have a perfect understanding of our networks. The community has spend quite a lot of effort on visualising what a neural network learns. However while many methods were invented to better understand deep neural networks, we can show that these methods do not produce the theoretically correct explanation for a linear model, which is a simple neural network. Despite this they are used on multi-layer networks with millions of parameters. We present the idea of creating unit tests for explanations. The idea behind the unit test is that while it might be impossible to define what a good explanation is, it is much easier to detect failure cases. Hence, by creating more and more reliable unit tests, the community can refine its methods iteratively and converge to a good solution. Based on an analysis of linear models we propose a generalization that yields two explanation techniques (PatternNet and PatternAttribution) that are theoretically sound for linear models, pass our unit test and produce improved explanations for deep networks. *BIO*Pieter-Jan Kindermans obtained his PhD degree from Ghent Unviersity in 2014. From 2014 to 2017 he was a postdoc, as a Marie-Curie fellow, in the lab of Klaus-Robert Müller in Berlin. Currently he is a Brain resident at Google. Initially he worked on unsupervised learning for Brain-Computer Interfaces (BCI) and demonstrated that unsupervised learning can replace traditional supervised classifiers. A further extension of this work, in collaboration with the universities of Freiburg and Ghent, ensured that the unsupervised decoder is guarantueed to converge to the optimal supervised solution. This project was nominated for the BCI award in 2017.In addition to his work in BCI, he also explored different subfields of machine learning, including deep learning, large scale training and its applications such as molecular chemistry. His current focus is on better understanding the operation of a deep neural network. His ultimate goal is to make deep learning a knowledge extraction tool for science.

1 0

[DIRO Talk] Marc Law (U of T) Tue Feb 20 11:00AM PCM Z240
by Michael Noukhovitch 20 Fév '18

20 Fév '18

We will have three special DIRO talks next week and kicking it off is *Marc Law*, from University of Toronto giving a talk on *Tuesday Feb 20* at* 11AM* in room *PCM Z240*. *Note that this is on *Tuesday*, and in a different room* Checks all the boxes for a fantastic talk, but you should come see for yourself! Michael *TITLE *Shallow and Deep Metrics for Machine Learning and Computer Vision *KEYWORDS* computer vision, few/zero-shot learning, metric learning *ABSTRACT* Similarity functions and distance metrics are used in many machine learning and computer vision contexts such as clustering, k-nearest neighbors classification, support vector machine, information/image retrieval, visualization etc. Traditionally, machine learning methods fixed sample representations and the used metric before learning a model optimized for the target task. Metric learning approaches, which learn the employed metric in a supervised way, have been proposed to increase performance on tasks such as clustering. In particular, they have shown great generalization performance to compare objects from categories that were not seen during training (for instance in face verification or few-shot learning). In this talk, I will talk about different shallow and deep metric learning approaches optimized for clustering and reducing model complexity. In the clustering task, I will present efficient approaches to learn a metric in a supervised or weakly supervised way. In the model complexity context, I will present approaches to limit the rank of shallow approaches, or reduce the dimensionality of a pretrained deep neural network to perform visualization or increase zero-shot learning performance. *BIO* Marc Law received a PhD in Computer Science from Université Pierre et Marie Curie (Paris, France) in 2015. He was a visiting research scholar in the team of Professor Eric Xing at the School of Computer Science, Carnegie Mellon University in 2015~2016. Since June 2016, he is a postdoctoral fellow in the Department of Computer Science (Machine Learning group) at the University of Toronto under the supervision of Professor Raquel Urtasun and Professor Richard Zemel. During his PhD, he worked mostly on distance metric learning applied to different contexts of computer vision and web archiving. He is currently working on deep learning. His main focus is to propose scalable machine learning methods. He received an award for best PhD from the French Association for Artificial Intelligence in 2016.

2 1

[Tea Talk] Jacob Steinhardt (Stanford) Fri Feb 16 10:30AM AA1360
by Michael Noukhovitch 16 Fév '18

16 Fév '18

This week we have *Jacob Steinhardt* from Stanford giving a special DIRO talk on *Fri Feb 16* at* 10:30AM* in room *AA1360*. Don't be fooled, this talk should be the real deal! Michael *TITLE *Provably Secure Machine Learning *KEYWORDS *Security, Optimization, AI Safety, Adversarial *ABSTRACT* The widespread use of machine learning systems creates a new class of computer security vulnerabilities where, rather than attacking the integrity of the software itself, malicious actors exploit the statistical nature of the learning algorithms. For instance, attackers can add fake data (e.g. by creating fake user accounts), or strategically manipulate inputs to the system once it is deployed. So far, attempts to defend against these attacks have focused on empirical performance against known sets of attacks. I will argue that this is a fundamentally inadequate paradigm for achieving meaningful security guarantees. Instead, we need algorithms that are provably secure by design, in line with best practices for traditional computer security. To achieve this goal, we take inspiration from robust statistics and robust optimization, but with an eye towards the security requirements of modern machine learning systems. Motivated by the trend towards models with thousands or millions of features, we investigate the robustness of learning algorithms in high dimensions. We show that most algorithms are brittle to even small fractions of adversarial data, and then develop new algorithms that are provably robust. Additionally, to accommodate the increasing use of deep learning, we develop an algorithm for certifiably robust optimization of non-convex models such as neural networks. *BIO*Jacob Steinhardt is a graduate student in artificial intelligence at Stanford University working with Percy Liang. His main research interest is in designing machine learning algorithms with the reliability properties of good software. So far this has led to the study of provably secure machine learning systems, as well as the design of learning algorithms that can detect their own failures and generalize predictably in new situations. Outside of research, Jacob is a technical advisor to the Open Philanthropy Project, and mentors gifted high school students through the USACO and SPARC summer programs.

1 1

No [Tea Talk] this week
by Michael Noukhovitch 05 Fév '18

05 Fév '18

Good luck on ICML submissions!

1 0

Fwd: Articles sur l'Intelligence artificielle dans le "O Mile-Ex"
by Yoshua Bengio 02 Fév '18

02 Fév '18

---------- Message transféré ---------- De : Morin Danielle <danielle.morin(a)umontreal.ca> Date : 2 février 2018 à 12:43 Objet : RE: Articles sur l'Intelligence artificielle dans le "O Mile-Ex" À : Myriam Côté <myriam.cote(a)rd.mila.quebec> Cc : Bengio Yoshua <yoshua.bengio(a)umontreal.ca>, Myriam CÔTÉ < cotemyri(a)gmail.com> ------------------------------ Good Morning, The piece of news concerning the installation of the hub in artificial intelligence in the building of O Mile-Ex situated on the street St-Urbain was made public this morning by Radio-Canada and in LaPresse +. The information concerning the various tenants of the building, including the MILA, being made public, thus the owner confirmed the "hub" to the journalists. This piece of news(short story) precedes the official announcement and we wanted you are informed about it. Furthermore, the owner of the building will grant an interview this evening to Téléjournal of 6:00 pm of Radio-Canada and to RDI economy. Best regards, *Danielle* *Morin* Directrice générale par intérim MILA – Institut québécois d’intelligence artificielle Téléphone : 514-343-2186 <(514)%20343-2186> Cellulaire : 514-261-0597 <(514)%20261-0597> Danielle.morin(a)umontreal.ca Here are the links towards the Web article of Radio-Canada and that of the exercise book Business(Affairs) of LaPresse +. http://ici.radio-canada.ca/nouvelle/1081709/technologie- montreal-saint-urbain-intelligence-artificielle http://plus.lapresse.ca/screens/47a384ca-5655-4d99- 92d7-ebc482759a00__7C___0.html?utm_medium=Email&utm_ campaign=Microsite+Share&utm_content=Screen

1 0

Fwd: telluride workshop
by Yoshua Bengio 02 Fév '18

02 Fév '18

---------- Forwarded message ---------- From: Tobi Delbruck (INI) <tobi(a)ini.uzh.ch> Date: 2018-02-02 2:28 GMT-05:00 Subject: telluride workshop To: Yoshua Bengio <yoshua.bengio(a)umontreal.ca> Hi Yoshua, if you or someone on your team could share this year's telluride workshop to g+ or other ML community or related forums it would be great. (I don't know these communities very well). The webpage is now up and topic areas have been selected and the workshop is now open for participation. https://sites.google.com/view/telluride2018/home It should be quite fun this year; would be great if people from Montreal could participate. Best, Tobi -- Prof. Tobi Delbruck, Inst. for Neuroinformatics, UZH-ETH Zurich Sensors Group: http://sensors.ini.uzh.ch Personal page: http://www.ini.unizh.ch/~tobi Building 55, Room G84, Winterthurerstr. 190, CH-8057 Zurich, Switzerland <https://maps.google.com/?q=Winterthurerstr.+190,+CH-8057+Zurich,+Switzerlan…> Tel lab +41 44 635 3038, cell +41 76 629 1500

1 0

[Tea Talk] William Hamilton (Stanford) Fri Feb 2, 10:30AM, AA1360
by Michael Noukhovitch 31 Jan '18

31 Jan '18

*Note:* This is the same as the DIRO colloqium email, I'm just making sure that everyone on the mailing lists gets it! Sorry for the late email! This week we have a PhD Candidate from Stanford, *William Hamilton,* giving a talk on *Friday Feb 2* at* 10:30AM* in room *AA1360*. Make sure to chart your way to this tea talk and invite all your friends! Michael *KEYWORDS *ML on Graphs, Massive Structured Datasets, Social Science *TITLE* Machine Learning for Computational Social Science *ABSTRACT* The combination of machine learning and massive social datasets has the potential to revolutionize our ability to predict and understand human behavior. However, machine learning on large social datasets is difficult because these datasets tend to be noisy, dynamic, and involve graph-structured relationships (e.g., social networks between users)—while traditional machine learning tools are largely designed for static datasets comprised of simple Euclidean vectors or grids. In this talk, I will describe new methods that I have developed for machine learning on massive, graph-structured social datasets. The technical focus of the talk will be on techniques for graph embedding, i.e., representation learning on graph-structured data. In the first part, I will describe how I have used graph embedding techniques to enable diverse social applications—from modeling cultural change to predicting conflict between online communities. In the second part, I will describe a new graph embedding framework, called GraphSAGE, that can scale to datasets that are orders of magnitude larger than previous approaches and that has been deployed at the website Pinterest to power a recommender system serving over 200 million users. I will close the talk with a general outlook on social AI technologies, including future technical directions and important ethical considerations. *BIO* William (Will) Hamilton is a PhD Candidate in Computer Science at Stanford University, working jointly in the NLP and SNAP groups. He is co-advised by Dan Jurafsky and Jure Leskovec, and his interests lie at the intersection of machine learning, network science, natural language processing, and computational social science. Will's research is supported by the SAP Stanford Graduate Fellowship and an NSERC PGS-D Grant. Prior to coming to Stanford, Will completed a BSc and MSc at McGill University, where he studied machine learning under the supervision of Joelle Pineau and was the 2014 recipient of the Canadian AI Master's Thesis Award.

1 0

Fwd: Colloque du DIRO, vendredi le 2 février, 10:30, AA1360 Conférencier: William Hamilton
by Yoshua Bengio 31 Jan '18

31 Jan '18

Job talk for ML position. Candidate has a Stanford PhD. Please attend and give Simon L-J and I feedback. More candidates are coming over the next few weeks. ---------- Forwarded message ---------- From: "Pierre Poulin" <poulin(a)iro.umontreal.ca> Date: Jan 31, 2018 11:39 Subject: Colloque du DIRO, vendredi le 2 février, 10:30, AA1360 Conférencier: William Hamilton To: <seminaires(a)iro.umontreal.ca> Cc: Machine Learning for Computational Social Science par William Hamilton PhD candidate, Stanford University Vendredi 2 février, 10:30 à 11:30, salle *** 1360 ***, pavillon André-Aisenstadt Université de Montréal, 2920 chemin de la Tour Résumé: The combination of machine learning and massive social datasets has the potential to revolutionize our ability to predict and understand human behavior. However, machine learning on large social datasets is difficult because these datasets tend to be noisy, dynamic, and involve graph-structured relationships (e.g., social networks between users)—while traditional machine learning tools are largely designed for static datasets comprised of simple Euclidean vectors or grids. In this talk, I will describe new methods that I have developed for machine learning on massive, graph-structured social datasets. The technical focus of the talk will be on techniques for graph embedding, i.e., representation learning on graph-structured data. In the first part, I will describe how I have used graph embedding techniques to enable diverse social applications—from modeling cultural change to predicting conflict between online communities. In the second part, I will describe a new graph embedding framework, called GraphSAGE, that can scale to datasets that are orders of magnitude larger than previous approaches and that has been deployed at the website Pinterest to power a recommender system serving over 200 million users. I will close the talk with a general outlook on social AI technologies, including future technical directions and important ethical considerations. Bio: William (Will) Hamilton is a PhD Candidate in Computer Science at Stanford University, working jointly in the NLP and SNAP groups. He is co-advised by Dan Jurafsky and Jure Leskovec, and his interests lie at the intersection of machine learning, network science, natural language processing, and computational social science. Will's research is supported by the SAP Stanford Graduate Fellowship and an NSERC PGS-D Grant. Prior to coming to Stanford, Will completed a BSc and MSc at McGill University, where he studied machine learning under the supervision of Joelle Pineau and was the 2014 recipient of the Canadian AI Master's Thesis Award. Note: La présentation sera donnée en anglais. Pierre -- Pierre Poulin (poulin(a)iro.umontreal.ca) Directeur Dept. I.R.O., Universite de Montreal (normal mail) (courier) C.P. 6128, Succ. Centre-Ville 2920, chemin de la Tour, room 2163/2194 Montreal (Quebec) Canada, H3C 3J7 Montreal (Quebec) Canada, H3T 1J4 (514) 343-6780, (514) 343-5834 (fax) http://www.iro.umontreal.ca/~poulin

2 1

FW: [Tea Talk] Bart Van Parys (MIT) Fri Jan 26 1:30PM AA1360
by Daniel Doyon 29 Jan '18

29 Jan '18

Hi, I unfortunately missed Bart Van Parys’s Tea Talk last Friday, Jan26. I was curious if his presentation could be forwarded my way to get a flavor of the talk! Thanks, Dan From: lisa_teatalk-bounces(a)iro.umontreal.ca <mailto:lisa_teatalk-bounces@iro.umontreal.ca> [mailto:lisa_teatalk-bounces@iro.umontreal.ca] On Behalf Of Michael Noukhovitch Sent: Friday, January 26, 2018 1:11 PM To: Tea Talks MILA <teatalk-orgs(a)iro.umontreal.ca <mailto:teatalk-orgs@iro.umontreal.ca> >; lisa_teatalk(a)iro.umontreal.ca <mailto:lisa_teatalk@iro.umontreal.ca> ; lisa_seminaires(a)iro.umontreal.ca <mailto:lisa_seminaires@iro.umontreal.ca> ; Lisa Labo <lisa_labo(a)iro.umontreal.ca <mailto:lisa_labo@iro.umontreal.ca> >; lisa_montreal(a)iro.umontreal.ca <mailto:lisa_montreal@iro.umontreal.ca> Subject: Re: [Lisa_teatalk] [Tea Talk] Bart Van Parys (MIT) Fri Jan 26 1:30PM AA1360 Reminder: this is in 20 minutes! On Mon, Jan 22, 2018 at 3:35 PM Michael Noukhovitch <mnoukhov(a)gmail.com <mailto:mnoukhov@gmail.com> > wrote: Hi all! We will have a researcher from MIT, Bart Van Parys, giving a talk on Friday Jan 26th at 1:30PM in room AA1360. Note the later time, 1:30! This is a seminar is organized in collaboration with the Canada Research Chair in Decision Making under Uncertainty at GERAD! So this is ideal for all the optimization fans out there! Based on my experience, this will be a great talk and coming to it should be an easy decision! Michael KEYWORDS Optimization, Sparse Regression, Generalization TITLE Modern Optimization for Sparse Learning and Robust Analytics ABSTRACT We discuss the tremendous potential of integer optimization methods for learning predictive models from high-dimensional data via exact sparse regression. We show that novel integer formulations can solve exact sparse regression problems of sizes counting p=100,000s covariates for n=10,000s of samples. That is, two orders of magnitude more than current state of the art methods. We also indicate that robust optimization methods can help practitioners make data-driven decisions which are safeguarded against over-calibration to one particular data set. We claim that robust optimization methods have an enormous untapped potential when making subsequent decisions based on data. BIO Bart Van Parys is currently a postdoctoral researcher working with Prof. Dimitris Bertsimas at the MIT Sloan School of Management. His research interests are situated on the interface between optimization and machine learning. In 2015 he obtained his Ph.D. in control theory at the Swiss Federal Institute of Technology (ETH) in Zurich under the supervision of Prof. Manfred Morari. He received his M.E. from the University of Leuven in 2011.

1 0

[Tea Talk] Bart Van Parys (MIT) Fri Jan 26 1:30PM AA1360
by Michael Noukhovitch 26 Jan '18

26 Jan '18

Hi all! We will have a researcher from MIT, *Bart Van Parys*, giving a talk on *Friday Jan 26th at 1:30PM* in room *AA1360*. Note the later time, *1:30!* This is a seminar is organized in collaboration with the Canada Research Chair in Decision Making under Uncertainty at GERAD! So this is ideal for all the optimization fans out there! Based on my experience, this will be a great talk and coming to it should be an easy decision! Michael *KEYWORDS *Optimization, Sparse Regression, Generalization *TITLE *Modern Optimization for Sparse Learning and Robust Analytics *ABSTRACT* We discuss the tremendous potential of integer optimization methods for learning predictive models from high-dimensional data via exact sparse regression. We show that novel integer formulations can solve exact sparse regression problems of sizes counting p=100,000s covariates for n=10,000s of samples. That is, two orders of magnitude more than current state of the art methods. We also indicate that robust optimization methods can help practitioners make data-driven decisions which are safeguarded against over-calibration to one particular data set. We claim that robust optimization methods have an enormous untapped potential when making subsequent decisions based on data. *BIO* Bart Van Parys is currently a postdoctoral researcher working with Prof. Dimitris Bertsimas at the MIT Sloan School of Management. His research interests are situated on the interface between optimization and machine learning. In 2015 he obtained his Ph.D. in control theory at the Swiss Federal Institute of Technology (ETH) in Zurich under the supervision of Prof. Manfred Morari. He received his M.E. from the University of Leuven in 2011.

1 1

[Tea Talk] Timothy O'Donnell (McGill) Fri Jan 19 10:30AM AA1360
by Michael Noukhovitch 19 Jan '18

19 Jan '18

*Note: *For the new year, we're moving to a different room, *AA1360* on the first floor! Exciting! For the first tea talk of the year we are going to have *Timothy O'Donnell*, professor at *McGill,* giving a talk on *Friday Jan 19th* at* 10:30AM* in room *AA1360*. This should be a real gronk of talk, and I'd encourage everyone to attend to it! Michael *KEYWORDS *Computational Linguistics, Language Learning, Linguistic Structure *TITLE* Computation, Storage, and Generalization in Language *ABSTRACT* A celebrated property of natural language is creativity, the ability to combine stored units to derive new expressions. This feature is found across multiple levels of linguistic structure: Sounds can be combined to form novel roots, suffixes, and affixes, which can be combined to form novel words, which can be combined to form novel sentences. However, each human language is characterized by its own inventory of units at each level and its own constraints on their combination. A syllable like "derp" sounds like a better morpheme of English than a syllable like "denp." The existence of the words "warmth" and "truth" do not imply the possibility of "coolth," but "warmness," "trueness," and "coolness" are all grammatical. And speakers of English will naturally drop the phrase "on the counter" from "John made dinner on the counter", but not from "John put dinner on the counter". How do learners acquire the inventory of units and constraints at each level of linguistic structure that are particular to their own language? I will discuss a theoretical framework designed to address this question and a number of specific applications. The approach is based on the idea that this problem can be approached by formulating a variety of linguistic learning problems as problems of program induction and then optimizing a tradeoff between a pressure to store fewer, more reusable primitive units and a pressure to account for each linguistic expression in as few computational steps as possible. Although the idea behind this tradeoff is an old one, it has surprisingly deep and far-reaching consequences when applied to domain-specific models of linguistic computation. I will show how this approach can shed light on a number of problems across the linguistic system. *BIO* Tim O'Donnell is an assistant professor in the department of linguistics at McGill University. Previously he was a research scientist at MIT in the department of Brain and Cognitive Sciences. His research focuses on developing mathematical and computational models of language learning and processing. His work draws on techniques from computational linguistics, and artificial intelligence integrating ideas from theoretical linguistics and methods from experimental psychology.

1 1

No [Tea Talk] this week
by Michael Noukhovitch 09 Jan '18

09 Jan '18

Happy new year and welcome back! This week's tea talk has been delayed til March so *no tea talk this week*. Instead you can come talk to me about tea for an hour, no scheduling needed! Cheers! Michael

1 0

Thesis proposal by Tristan Sylvain, Monday 9:30 @ 3195
by Yoshua Bengio 18 Déc '17

18 Déc '17

Hello, Tristan Sylvain will present his thesis proposal Monday 9:30-11:30 at PAA 3195. Everyone is welcome. Jury: President: Simon Lacoste-Julien Supervisor: Yoshua Bengio Member: Ioannis Mitliagkas Deep Neural Networks applied to medical data and Zero-Shot learning This presentation details two ongoing projects, applying deep learning to the problem of survival analysis for medical data, and using an adversarial approach to zero-shot learning. These very different problems have in common the fact that deep learning allows the learning of efficient representations of data that permit generalization when dealing with both highly noisy (in the medical case) and weakly supervised data (the zero-shot learning case). For the survival data, we used a multi-objective training procedure that allowed us to attain state of the art on many different public datasets, when compared to classical methods such as the Cox model. This presentation will provide both insights on the architecture used, and the medical context. Overall this problem is an interesting instance of applying deep learning to medical problems. Zero-shot learning is an active field of machine learning characterized by the fact that the classes we perform test time classification on are all unseen during the training procedure. To compensate for the lack of information on these classes, we are usually given a set of semantic attributes to allow some form of transfer from a model learned on seen train classes. This part of the presentation will present a current project aiming to use conditional generative adversarial networks to generate fake prototypes for unseen classes in a zero-shot learning context. I will present current results, and some of the challenges that lie ahead.

1 1

[Tea Talk] Romain Laroche (MS Maluuba) Fri Dec 1 10:30AM, AA6214
by Michael Noukhovitch 01 Déc '17

01 Déc '17

This week we have a researcher from MS Maluuba, *Romain Laroche,* giving a talk on *Friday Dec 1* at* 10:30AM* in room *AA6214*. See you there, this RL talk is sure to be rewarding! Michael *KEYWORDS* policy-based RL, bootstrapping, data/computational efficiency *TITLE *Safe Policy Improvement with Baseline Bootstrapping *ABSTRACT* A common goal in Reinforcement Learning is to derive a good strategy given a limited batch of data. In this paper, we propose a new strategy to compute a safe policy, guaranteed to perform at least as well as a given baseline strategy. We advocate that the assumptions made in previous work are too strong for real world applications and propose new algorithms allowing those assumptions to be satisfied only in a subset of the state-action pairs. While significantly relaxing the assumptions, our algorithms achieve the same accuracy guarantees than the previous work, and are also much more computationally efficient. We also show that the algorithms can be adapted to model-free Reinforcement Learning. *BIO* Graduated from Ecole Polytechnique in 2001 and then from Telecom PariTech in 2003, Romain Laroche joined the dialogue team at Orange in Paris, where he defended in 2010 a corporate PhD on Reinforcement Learning for industrial dialogue systems at Université Pierre et Marie Curie (Paris VI). Romain joined Maluuba in 2016, and is now a researcher at Microsoft Research Maluuba. During the past 6 years he supervised 5 PhDs, 3 postdocs, and a dozen of undergrad interns. Counting more than 40 papers at international conferences, his interest focuses now on Reinforcement Learning. His preferred application domains are dialogue systems (still his primary real-world motivation), Atari games (for benchmarking against other algorithms), and navigation toy problems (for empirical analysis and algorithm design).

1 1

[Tea Talk] Sergey Plis (MRN) Fri Nov 24, 10:30AM, AA6214
by Michael Noukhovitch 24 Nov '17

24 Nov '17

This week we have TWO tea talks (what a treat!) Apart from Martin Arjovsky's talk on Thursday, we also have the director of machine learning at the Mind Research Network, *Sergey Plis,* giving a talk on *Friday Nov 24* at* 10:30AM* in room *AA6214*. Hope this gives you cause to come, there's lots to learn! Michael *KEYWORDS *neuroscience, brain imaging, bayesian networks, structure learning *TITLE *Causal learning in complex time series datasets *ABSTRACT* Cortical neurons form coherent functional networks that are surprisingly stable across subjects and conditions. Together these networks comprise a set of functional units of the brain. Understanding their interactions can lead to better understanding of brain’s function and dysfunction due to disruption of the interaction structure. The most common way of assessing this structure are cross-correlation matrices (usually referred to as functional connectivity) but their drawbacks make us turn to modelling interactions via the directed graph of a Bayesian network (effective connectivity). Various brain imaging modalities contain different and arguably complementary information about interactions of functional network. Our goal is to bring together multimodal information to improve effective connectivity estimates. In particular, I focus on the problem of finding a common denominator for causal structures learned from time series at different time scales. I will demonstrate 1) a general theory which explains the effects of undersampling on apparent causal structure in terms of the true structure at the causal time scale; 2) a forward algorithm that computes a graph structure at any given undersampling rate; and 3) some inverse algorithms to compute all of the candidate graphs that could have generated the given undersampled structure. Although the problem is inspired by the questions from brain imaging, all time series models that are assessing structural relations are affected by the phenomena that I will describe. This includes Dynamic Bayesian Networks on the causal learning side and such models as vector autoregressive (VAR), structured VAR, Granger causality. *BASED ON* Cook, John W., David Danks, and Sergey M. Plis. (2017) "Learning Dynamic Structure from Undersampled Data <https://www.cs.purdue.edu/homes/eb/causal-uai17/papers/7.pdf>." UAI Causality Workshop A constraint optimization approach to causal discovery from subsampled time series data <https://scholar.google.ca/scholar?oi=bibs&cluster=5464303113063724661&btnI=…> A Hyttinen, S Plis, M Järvisalo, F Eberhardt, D Danks - International Journal of Approximate Reasoning, 2017 Danks, D., & Plis, S. M. (2017). Amalgamating evidence of dynamics. Synthese. [PDF <http://www.andrew.cmu.edu/user/ddanks/papers/Amalgamation-Synthese-Prepub.p…> ] Plis, S., Danks, D., Freeman, C., & Calhoun, V. (2015). Rate-agnostic (causal) structure learning. In Advances in neural information processing systems 28 (pp. 3303-3311). La Jolla, CA: The NIPS Foundation. (First two authors contributed equally.) [PDF <http://www.andrew.cmu.edu/user/ddanks/papers/RASL-NIPS-final.pdf>] Plis, S., Danks, D., & Yang, J. (2015). Mesochronal structure learning. In M. Meila & T. Heskes (Eds.), Uncertainty in artificial intelligence 31 (UAI-2015) (pp. 702-711). Corvallis, OR: AUAI Press. (First two authors contributed equally.) [PDF <http://www.andrew.cmu.edu/user/ddanks/papers/MSL-UAI15-final.pdf>] Danks, D., & Plis, S. (2014). Learning causal structure from undersampled time series. In JMLR: Workshop and Conference Proceedings. (First two authors contributed equally.) [PDF <http://www.andrew.cmu.edu/user/ddanks/papers/DanksPlis-Final.pdf>] *BIO*Dr. Sergey Plis is a Director of Machine Learning at the Mind Research Network. His research interests lie in developing novel and applying existing techniques and approaches to analyzing large scale datasets in multimodal brain imaging and other domains. He develops tools that fall within the fields of machine learning and data science. One key goal is to take advantage of the strengths of imaging modalities and infer structure and patterns that are hard to obtain non-invasively and/or that are unavailable for direct observation. In the long term this amounts to developing methods capable of revealing mechanisms used by the brain for forming task specific transient interaction networks and their cognition-inducing interactions via multimodal fusion at features and interaction levels. Ongoing work is focused on inferring multimodal probabilistic and causal descriptions of these function-induced networks based on fusion of fast and slow imaging modalities. This includes feature estimation via deep learning-based pattern recognition and learning causal graphical models. Dr. Plis is also the R&D director at a young machine learning startup reason8.ai

1 1

[Tea Talk] Martin Arjovsky (NYU) Thr Nov 23, 10:30AM, AA6214
by Michael Noukhovitch 23 Nov '17

23 Nov '17

This week we have a former MILA intern, *Martin Arjovsky,* now at NYU giving a talk on *Thursday Nov 23* at* 10:30AM* in room *AA6214*. Be there or be probabilistically sqaure! Michael *KEYWORDS* deep learning theory, probability theory, geometry *TITLE* A few geometrical insights into unsupervised learning *ABSTRACT* This short talk will address the connections and properties of the energy (including Cramer), and Wasserstein distances, and how they connect with Maximum Mean Discrepancy. We will dive into the similarities and differences among them, and how this impacts their behaviour in different ways. Some of the results are a contextualization of old studies, and some will be presented in this talk for the first time. A recurrent theme in the talk will be the study of the geometry of the space of probability distributions, and how this might shed clarity on the behaviour of different algorithms in practice. *BIO* I'm Martin Arjovsky, I'm currently doing my PhD at New York University, and being advised by Léon Bottou. I did my undergraduate and master's in the University of Buenos Aires, Argentina (my home country). In the middle I took a year off to do internships in different places (Google, Facebook, Microsoft, and the Université de Montréal). My master's thesis advisor was Yoshua Bengio, who also advised me during my stay at UdeM. In general I'm interested in the intersection between learning and mathematics, how we can ground the different learning processes that are involved in different problems, and leverage this knowledge to develop better algorithms.

1 1

[Tea Talk] Zac Kenton (MILA) Fri Nov 17, 10:30AM, AA6214
by Michael Noukhovitch 17 Nov '17

17 Nov '17

This week we have our very own *Zac Kenton*, a visiting researcher at MILA giving a talk on *Friday Nov 17* at* 10:30AM* in room *AA6214*. I've suggested that the full title should be "Three factors influencing minima in SGD: You'll never believe #1 and #3!" See you there! Michael *KEYWORDS:* SGD, Deep Learning Theory, Generalization *TITLE* Three factors influencing minima in SGD *ABSTRACT* We focus on the importance of noise in stochastic gradient descent (SGD) based training of deep neural networks (DNNs). We develop theory that studies SGD training as a stochastic differential equation and show that its stationary distribution is related to the loss surface. Our analysis suggests that the combination of batch size, learning rate, and the variance of the true loss gradients acts as a hyper- parameter steering the behavior of SGD and determines the trade-offs between the depth and width of the minima that SGD converges to. In a nutshell, a higher ratio of learning rate to batch size leads to wider minima. We validate our theory by examining the correlation between these three factors and the final performance and sharpness of the minimum found. As a verification of our theory, we empirically demonstrate that the learning dynamics is similar between experiments with different learning rates and batch sizes in SGD if the ratio of learning rate to batch size is the same. *BIO* Zac studied Mathematics at the University of Cambridge for a bachelors and masters 2009-2013. He then completed a PhD in theoretical physics at the Centre for Research in String Theory, Queen Mary University of London, 2013- August 2017. His thesis was on string theory and early universe inflationary cosmology. In the final stages of his PhD he also worked as a data scientist at ASI Data Science, a London-based data science startup. At MILA he's been working with Stanislaw Jastrzebski, Devansh Arpit and Prof Bengio on topics around generalization in SGD.

1 1

Re: [Lisa_teatalk] [Lisa_labo] please sign letter to ask government to work against lethal autonomous weapons at the UN
by Yoshua Bengio 15 Nov '17

15 Nov '17

Please do! On Nov 10, 2017 09:28, "Joseph Paul Cohen" <joseph(a)josephpcohen.com> wrote: Can I forward this email to researchers and ask them to sign the petition? -Joseph On Nov 10, 2017 06:57, "Yoshua Bengio" <yoshua.umontreal(a)gmail.com> wrote: Hello, Especially if you are an AI researcher in Canada, please consider and sign this letter asking the Canadian government to urgently address the challenge of lethal autonomous weapons (often called “killer robots”) and to take a leading position against Autonomous Weapon Systems on the international stage at the upcoming UN meetings in Geneva. The leading signatories of this letter are Ian Kerr <https://www.facebook.com/ian.kerr?fref=mentions>, Geoff Hinton, Richard S. Sutton <https://www.facebook.com/richard.s.sutton.7?fref=mentions>, Doina Precup <https://www.facebook.com/doina.precup?fref=mentions> and myself. Let's work together towards a socially responsible and ethically grounded use of AI. https://techlaw.uottawa.ca/bankillerai -- Yoshua _______________________________________________ Lisa_labo mailing list Lisa_labo(a)iro.umontreal.ca https://webmail.iro.umontreal.ca/mailman/listinfo/lisa_labo

2 1