- Lisa_teatalk - mailman.iro.umontreal.ca

Talk Tomorrow!!!
by Aaron Courville 30 Mai '17

30 Mai '17

Everyone, We will have a talk tomorrow at 2pm (note unusual day/time). Our speaker is Michael Chang from MIT, working with Josh Tenenbaum. Date / Time: Wednesday May 31st, 2pm-3pm Location: AA3195 Title: *Learning Visual and Physical Models of the Environment* Abstract: An intelligent agent can leverage a model of the environment as a prior to accelerate future learning and generalize beyond its own experience. I view building such models as learning simulator programs that infer latent properties and generate predictions from observation. How can the agent learn the primitives for these programs from observation? What are the means of combination that operate on these primitives that allow zero-shot generalization to come naturally? In this talk, I discuss two of my recent work that propose potential directions for tackling these questions in the context of learning visual concepts and intuitive physics. First, I present an algorithm for learning factorized symbolic representations from raw visual data. I show that these representations capture latent factors of variation that can be manipulated like a graphics code. Next, I present a framework for learning predictive models of intuitive physics. With two-dimensional worlds of bouncing balls, I demonstrate that this framework generalizes to variable object count and variable scene configurations with only spatially and temporally local computation. I conclude by describing open research questions motivated by the results from these approaches. Bio: Michael Chang is a senior in Computer Science at MIT, researching in Professor Joshua Tenenbaum's Computational Cognitive Science Group. He is interested in building algorithms for learning compositional programs and bridging the strengths of symbolic and neural representations. At MIT, he has worked on unsupervised learning of symbolic visual concepts and of physical dynamics. Michael has spent time at the University of Michigan researching with Professor Honglak Lee. Michael will be pursuing a Ph.D. in Computer Science at U.C. Berkeley beginning fall 2017. Links to papers and code are here: http://mbchang.github.io. -- Aaron C. Courville Département d’Informatique et de recherche opérationnelle Université de Montréal email: Aaron.Courville(a)gmail.com

1 0

[Tea Talk] Marco Pedersoli, May 26, AA5340, 13:45
by Dzmitry Bahdanau 26 Mai '17

26 Mai '17

Hi all, Our next speaker is Marco Pedersoli who is a professor at the École de technologie supérieure of Montreal (ETS). Hope to see many of you there! *Title:* Areas of Attention for Image Captioning *Abstract: *We propose “Areas of Attention”, a novel attention-based model for automatic image captioning. Our approachmodels the dependencies between image regions, caption words, and the state of an RNN language model, using three pairwise interactions. In contrast to previous attention-based approaches that associate image regions only to the RNN state, our method allows a direct association between caption words and image regions. During training these associations are inferred from image-level captions, akin to weakly-supervised object detector training. These associ- ations help to improve captioning by localizing the corresponding regions during testing. We also propose and com- pare different ways of generating attention areas: CNN activation grids, object proposals, and spatial transformers nets applied in a convolutional fashion. Spatial transformers give the best results. They allow for image specific at- tention areas, and can be trained jointly with the rest of the network. Our attention mechanism and spatial transformer attention areas together yield state-of-the-art results on the MSCOCO dataset. *Bio:* Marco Pedersoli obtained his Ph.D. from the Autonomous University Of Barcelona (june 2012, with distinction and best thesis award) on Hierarchical Multi-resolution Detection of objects in images. He completed a postdoctoral fellowship at the KU Leuven, where he has developed several innovative approaches for object detection, action classification and pose estimation based on weakly-supervised methods. In September 2015 he moved to INRIA-Grenoble, where he has developed new techniques for the automatic description and understanding of images. From February 2017 he is assistant professor at the École de technologie supérieure of Montreal. He has published more than 20 articles in peer-reviewed internationals journals and top peer-reviewed conferences in computer vision. Dima

1 1

No tea-talk this week
by Junyoung Chung 09 Mai '17

09 Mai '17

Hi all, we do not have a talk this week due to NIPS submission deadline. See you after the deadline and wish you a good luck! Best, --Junyoung

1 0

Timnit Gebru (Stanford CV Lab, PhD candidate), May 5, 13:45, AA5340
by Junyoung Chung 05 Mai '17

05 Mai '17

Hi all, the next tea-talk will be given by Timnit Gebru from Stanford university. The room will be AA5340, and the talk will start at 13:45. Here is the detail of the talk. *Title: *Using Deep Learning and Google Street View to Estimate the Demographic Makeup of the US *Abstract: *Targeted socio-economic policies require an accurate understanding of a country’s demographic makeup. To that end, the United States spends more than 1 billion dollars a year gathering census data such as race, gender, education, occupation and unemployment rates. Compared to the traditional method of collecting surveys across many years which is costly and labor intensive, data-driven, machine learning driven approaches are cheaper and faster—with the potential ability to detect trends in close to real time. In this work, we leverage the ubiquity of Google Street View images and develop a computer vision pipeline to predict income, per capita carbon emission, crime rates and other city attributes from a single source of publicly available visual data. We first detect cars in 50 million images across 200 of the largest US cities and train a model to determine demographic attributes using the detected cars. To facilitate our work, we used a graph-based algorithm to collect the largest and most challenging fine-grained dataset reported to date consisting of over 2600 classes of cars comprised of images from Google Street View and other web sources. Our prediction results correlate well with ground truth income (r=0.82), race, education, voting, sources investigating crime rates, income segregation, per capita carbon emission, and other market research. Finally, we learn interesting relationships between cars and neighborhoods allowing us to perform the first large-scale sociological analysis of cities using computer vision techniques. *Bio: *I am a PhD student in the Stanford Artificial Intelligence Laboratory, studying computer vision under Fei-Fei Li. My main research interest lies in data mining large-scale publicly available images to gain sociological insight, and working on computer vision problems that arise as a result. Some of these include fine-grained image recognition, scalable annotation of images, and domain adaptation. Prior to joining Fei-Fei's lab I worked at Apple designing circuits and signal processing algorithms for various Apple products including the first iPad. I also spent an obligatory year as an entrepreneur (as all Stanford undergrads seem to do). My research is supported by the NSF foundation GRFP fellowship and currently the Stanford DARE fellowship. Best, --Junyoung

1 3

No tea-talk today
by Dzmitry Bahdanau 28 Avr '17

28 Avr '17

Hi all, There is no tea-talk today - blame ICLR for that :) Dima

1 0

[TeaTalk] Michael James, April 21, 13:45, AA5340
by Dzmitry Bahdanau 21 Avr '17

21 Avr '17

Hi all, Our next speaker is Michael James from Cerebras Systems. The talk will take place at *AA5340, April 21, 13:45*. Hope to see many of you there! *Title:* Continuous Propagation and Computer Architectures Specialized for Deep Learning *Abstract: *I will be discussing a novel computing platform designed for training deep neural models, and highlight Continuous Propagation, which is one of the new learning algorithms that it makes possible. Continuous Propagation is a training technique that allows layer-parallelism: training for deep models is parallelized along the depth of the network. The technique is motivated by the continuous differential formulation of learning and has relations to biologically plausible learning. Cerebras Systems is building a next generation system, purpose designed for sparse linear algebra and large stochastic optimization problems. *Bio: *I am a founder and Chief Software Architect at Cerebras Systems, a venture-backed stealth mode startup located in Los Altos, California. Prior to founding Cerebras Systems, I was lead SW Architect at SeaMicro (acquired by AMD for $357 million). I was responsible for the distributed system software that among other things, did placement and work allocation for the more than 16,000 cores in a SeaMicro solution. Post acquisition, I was an AMD Fellow and worked as a hardware and software architect. At AMD, I designed a novel distributed high-availability compute architecture base on cellular automata. Career highlights include: Platform Architect, Kno Inc, where I designed gesture recognition algorithms. Chief Architect, MultiLing Corp, where I designed machine translation software using fuzzy logic. I have designed operating systems, compilers, real-time feedback control systems, signal processing, and recognition systems. I hold patents in computer architecture and distributed algorithms. My alma mater is University of California at Berkeley where I studied Molecular Neurobiology, Mathematics, and Computer Science. Dima

3 3

Fwd: [Lisa_labo] Fwd: [Lisa_montreal] [TeaTalk] Michael James, April 21, 13:45, AA5340
by Devansh Arpit 21 Avr '17

21 Avr '17

Hello Everyone, The tea talk is about to start. Room AA5340. Thank you, Devansh Arpit ---------- Forwarded message ---------- From: Joseph Paul Cohen <joseph(a)josephpcohen.com> Date: Fri, Apr 21, 2017 at 11:58 AM Subject: Re: [Lisa_labo] Fwd: [Lisa_montreal] [TeaTalk] Michael James, April 21, 13:45, AA5340 To: Devansh Arpit <devansharpit(a)gmail.com> Cc: Lisa montreal <lisa_montreal(a)iro.umontreal.ca>, Lisa Labo < lisa_labo(a)iro.umontreal.ca>, "lisa_teatalk(a)iro.umontreal.ca" < lisa_teatalk(a)iro.umontreal.ca>, "lisa_seminaires(a)iro.umontreal.ca" < lisa_seminaires(a)iro.umontreal.ca>, "teatalk-orgs(a)lisa.iro.umontreal.ca" < teatalk-orgs(a)iro.umontreal.ca> There will be food! On Fri, Apr 21, 2017 at 11:12 AM, Devansh Arpit <devansharpit(a)gmail.com> wrote: > Hello Everyone, > > This is a reminder that we have a talk today by Michael James from Cerebras > Systems based in Los Altos, California. > The room will be AA5340 and starts at 13:45. > > > > Thank you, > Devansh Arpit > > ---------- Forwarded message ---------- > From: Dzmitry Bahdanau <dimabgv(a)gmail.com> > Date: Wed, Apr 19, 2017 at 10:39 AM > Subject: [Lisa_montreal] [TeaTalk] Michael James, April 21, 13:45, AA5340 > To: Lisa montreal <lisa_montreal(a)iro.umontreal.ca>, Lisa Labo > <lisa_labo(a)iro.umontreal.ca>, "lisa_teatalk(a)iro.umontreal.ca" > <lisa_teatalk(a)iro.umontreal.ca>, "lisa_seminaires(a)iro.umontreal.ca" > <lisa_seminaires(a)iro.umontreal.ca>, "teatalk-orgs(a)lisa.iro.umontreal.ca" > <teatalk-orgs(a)iro.umontreal.ca> > > > Hi all, > > Our next speaker is Michael James from Cerebras Systems. The talk will take > place at AA5340, April 21, 13:45. Hope to see many of you there! > > Title: Continuous Propagation and Computer Architectures Specialized for > Deep Learning > > Abstract: I will be discussing a novel computing platform designed for > training deep neural models, and highlight Continuous Propagation, which is > one of the new learning algorithms that it makes possible. Continuous > Propagation is a training technique that allows layer-parallelism: training > for deep models is parallelized along the depth of the network. The > technique is motivated by the continuous differential formulation of > learning and has relations to biologically plausible learning. Cerebras > Systems is building a next generation system, purpose designed for sparse > linear algebra and large stochastic optimization problems. > > Bio: I am a founder and Chief Software Architect at Cerebras Systems, a > venture-backed stealth mode startup located in Los Altos, California. Prior > to founding Cerebras Systems, I was lead SW Architect at SeaMicro (acquired > by AMD for $357 million). I was responsible for the distributed system > software that among other things, did placement and work allocation for the > more than 16,000 cores in a SeaMicro solution. Post acquisition, I was an > AMD Fellow and worked as a hardware and software architect. At AMD, I > designed a novel distributed high-availability compute architecture base on > cellular automata. Career highlights include: Platform Architect, Kno Inc, > where I designed gesture recognition algorithms. Chief Architect, MultiLing > Corp, where I designed machine translation software using fuzzy logic. I > have designed operating systems, compilers, real-time feedback control > systems, signal processing, and recognition systems. I hold patents in > computer architecture and distributed algorithms. My alma mater is > University of California at Berkeley where I studied Molecular Neurobiology, > Mathematics, and Computer Science. > > Dima > > _______________________________________________ > Lisa_montreal mailing list > Lisa_montreal(a)iro.umontreal.ca > https://webmail.iro.umontreal.ca/mailman/listinfo/lisa_montreal > > > > _______________________________________________ > Lisa_labo mailing list > Lisa_labo(a)iro.umontreal.ca > https://webmail.iro.umontreal.ca/mailman/listinfo/lisa_labo >

1 0

[Tea-Talk] No tea-talk on April 14
by Dzmitry Bahdanau 11 Avr '17

11 Avr '17

Hi all, This is a reminder that there will be no tea-talk on April 14, since the university will be closed for the Easter break. Enjoy the holidays! Dima

1 0

Amir Moravej, April 7, 13:45, Z209
by Junyoung Chung 07 Avr '17

07 Avr '17

Hi all, the next tea-talk will be given by Amir Moravej from BotlerAI. The room will be Z209, and the talk starts at 13:45. Title: AI-Powered Immigration Chatbot About: Immigration Law refers to government policies which regulate the right of entry, and residency right in each country. Immigration Law absorbs features from other closely related areas of law, including Administrative Law, Constitutional Law, Criminal Law, Tax Law etc. The Immigration Process refers to a set oa steps an individual should take in order to obtain a legal status under Immigration Law. Botler is a Chatbot which automates Immigration Process. From a system perspective, the immigration process can be modeled as a two steps process: Eligibility Assessment, and Application Packaging. Botler models Eligibility Assessment as a rule-based system initiative dialogue using a finite-state dialogue manager which is modeled as a graph of action rules. Application Packaging is a three step process: document authenticity, document relevance, and packaging. Document authenticity is modeled using a CNN for image object recognition. Document relevance is modeled using an OCR for content extraction, and a text processing module to determine content relevancy. Packaging is a two step process, filling out required forms, and creating a map of required documents to the actual documents provided by the user. Botlers’ dataset contains the largest Canadian immigration forum, tens of thousands of conversations in the form of email threads between human legal agents and users, more than five millions of Canadian immigration case eligibility assessment results, and thousands of Immigration Law related documents. Bio: Amir graduated as a software engineer in 2009, and came to Montreal to continue his graduate studies at Concordia. He finished his Master's degree in Quality Systems Engineering in 2012 and started working in startups ever since. He contributed to the development of several projects, including a content personalization platform for Relevant AI, a QA system for Toti VT, a relevancy engine for nTerop, a data visualization platform for Retinad VR, and the full stack of Squado app. He founded Botler AI in 2016 after he faced several challenges with him immigration process. Best, --Junyoung

1 2

Brian Ziebart on "Supervised Machine Learning as an Adversarial Game", March 31, 13:45, AA6214
by Junyoung Chung 31 Mar '17

31 Mar '17

Hi all, the next tea-talk will be given by Prof. Brian Ziebart from the University of Illinois at Chicago. The talk will be held at 13:45 & AA6214. If you would like to chat with Brian, please fill your names and available time-slots on the below spreadsheet: https://docs.google.com/spreadsheets/d/14PpRI2WYl_zPlLPGC5SPQu4fFbJDFHIIXt4… Title: Supervised Machine Learning as an Adversarial Game Abstract: A standard approach to supervised machine learning is to choose the form of a predictor and to then optimize its parameters based on training data. Approximations of the predictor's performance measure are often required to make this optimization problem tractable. Instead of approximating the performance measure and using the exact training data, this talk explores adversarial approximations of the training data while optimizing the exact performance measure. This formulation ensures stronger theoretical guarantees (e.g., Fisher consistency) than other surrogate loss methods, better aligns predictor construction with univariate and multivariate performance measures found in computer vision, information retrieval, and rank learning tasks, and provides flexibility for addressing sample selection bias for improved active learning. Bio: Brian Ziebart is an Assistant Professor in the Department of Computer Science at the University of Illinois at Chicago. He earned his PhD in Machine Learning from Carnegie Mellon University where he was also a postdoctoral fellow. His interests lie in the intersections between machine learning, game theory, and decision theory. He has published over 25 articles in leading machine learning and artificial intelligence venues, including one that received a Best Paper award at the International Conference on Machine Learning. Best, --Junyoung

3 6

Talk CHANGE of LOCATION Today 2:30pm - 3:30pm
by Aaron Courville 28 Mar '17

28 Mar '17

This is a reminder that this afternoon we will have a talk by Lluís Castrejón, a visiting student from Toronto currently working with Raquel Urtasun and Sanja Fidler. He will be presenting his recently accepted CVPR (oral) paper. Time: 2:30pm - 3:30pm. Location: AA5340 (NOTE NEW LOCATION). Title: Annotating Object Instances with a Polygon-RNN Abstract: We propose an approach for semi-automatic annotation of object instances. While most current methods treat object segmentation as a pixel-labeling problem, we here cast it as a polygon prediction task, mimicking how most current datasets have been annotated. In particular, our approach takes as input an image crop and produces a vertex of the polygon, one at a time, allowing a human annotator to interfere at any time and correct the point. Our model easily integrates any correction,producing as accurate segmentations as desired by the annotator. We show that our annotation method speeds up the annotation process by factor of 4.7 across all classes in Cityscapes, while achieving 78.4% agreement in IoU with original ground-truth, matching the typical agreement between human annotators. For cars, our speed-up factor is even higher, at 7.3 for an agreement of 82.2%. We further show generalization capabilities of our approach on unseen datasets. Cheers, Aaron -- Aaron C. Courville Département d’Informatique et de recherche opérationnelle Université de Montréal email:Aaron.Courville@gmail.com

1 0

Talk (Special Time) Tuesday 2:30pm - 3:30pm
by Aaron Courville 27 Mar '17

27 Mar '17

On Tuesday (tomorrow) we will have a talk by Lluís Castrejón, a visiting student from Toronto currently working with Raquel Urtasun and Sanja Fidler. He will be presenting his recently accepted CVPR (oral) paper. If you would like to meet with him please let me know. Time: 2:30pm - 3:30pm. Location: AA6214. Title: Annotating Object Instances with a Polygon-RNN Abstract: We propose an approach for semi-automatic annotation of object instances. While most current methods treat object segmentation as a pixel-labeling problem, we here cast it as a polygon prediction task, mimicking how most current datasets have been annotated. In particular, our approach takes as input an image crop and produces a vertex of the polygon, one at a time, allowing a human annotator to interfere at any time and correct the point. Our model easily integrates any correction,producing as accurate segmentations as desired by the annotator. We show that our annotation method speeds up the annotation process by factor of 4.7 across all classes in Cityscapes, while achieving 78.4% agreement in IoU with original ground-truth, matching the typical agreement between human annotators. For cars, our speed-up factor is even higher, at 7.3 for an agreement of 82.2%. We further show generalization capabilities of our approach on unseen datasets. Cheers, Aaron -- Aaron C. Courville Département d’Informatique et de recherche opérationnelle Université de Montréal email:Aaron.Courville@gmail.com

1 0

[Tea-Talk] Andreas Moshovos, March 24, 13:45, AA6214
by Dzmitry Bahdanau 24 Mar '17

24 Mar '17

Hi all, Our next tea-talk will be given by Dr. Andreas Moshovos *on March 24, at 13:45 (new time!), room AA6214*, who is a professor at University of Toronto. Hope you see many of you there! *Please let me know if you would like to speak to Andreas after the talk. *You can directly put your name on the spreadsheet below: https://docs.google.com/spreadsheets/d/1vVn0eOAlTmkmd-DpEwuMD2NEQH_H8NLlY5Y… *Title: *Exploiting Value Content to Accelerate Inference with Convolutional Neural Networks *Abstract: *Sufficiently capable computing hardware is essential for practical applications of Deep Learning. Until very recently computing hardware capabilities have been increasing at an exponential rate. As a result, around 2010 computing hardware capabilities reached the level necessary to demonstrate Deep Learning’s true potential. Unfortunately, semiconductor technology scaling, the keep enabler of this past exponential growth in capabilities, has slowed down dramatically. Fortunately, specialized computing hardware design has the potential to deliver another 2 to 3 orders of improvements in computing capabilities. Our goal is to develop the techniques necessary for boosting computing hardware capabilities thus enabling further innovation in Deep Learning. We are developing specialized computing hardware for Deep Learning Networks whose key feature is that they are value-based. We have been developing value-based accelerators that take advantage of expected properties in the runtime calculated value stream of Deep Learning Networks such as the value distribution of activations, or even their bit content. Using image classification convolutional neural networks, we have demonstrated 2 to 3 orders of magnitude execution time improvements over conventional graphics processor hardware and up to 4.5x improvements over a state-of-the-art accelerator. In this talk we will review the need for specialized computing hardware for Deep Learning and summarize our efforts. We will also briefly touch upon the recently approved NSERC COHESA Strategic Partnership Network on Hardware Acceleration for Machine Learning. NSERC COHESA brings together 19 Researchers across multiple Canadian Universities and 8 Industrial Partners. *Bio: *Andreas Moshovos teaches how to design and optimize computing hardware engines at the University of Toronto where he has the privilege of collaborating with several talented students on techniques to improve execution time, energy efficiency and cost for computing hardware. He has also taught at Northwestern University, USA, the University of Athens, Greece, the Hellenic Open University, Greece, and as an invited professor at the École Polytechnique Fédérale de Lausanne, Switzerland. He has received the ACM SIGARCH Maurice Wilkes award in 2010, an NSF CAREER Award in 2000, two IBM Faculty awards, a Semiconductor Research Corporation Inventor recognition award, and a MICRO Hall of Fame award. He has served at the Program Chair for the ACM/IEEE International Symposium on Microarchiteture and the IEEE International Symposium on the Performance Analysis of Systems and Software. He studied computer science at the University of Crete, Greece, at New York University, USA, and at the University of Wisconsin-Madison, USA. Best, Dima

1 2

[Tea-Talk] Devon Hjelm & Laurent Dinh, March 17, 14:30, AA6214
by Dzmitry Bahdanau 17 Mar '17

17 Mar '17

Hi all, At the next tea-talk Devon Hjelm (post-doc) and Laurent Dinh (PhD student) from MILA will present their ICML submissions. Please come in great numbers on *March 17 to AA6214 at 13:30*! The details are below. *Speaker:* Devon Hjelm *Title:* Boundary-Seeking Generative Adversarial Networks *Abstract: *We introduce a novel approach to training generative adversarial networks (GANs, Goodfellow et al., 2014), which stems from reinterpreting the generator objective to match a target distribution that converges to the data distribution at the limit of a perfect discriminator. This objective can be interpreted as training the generator to produce samples that lie on the decision boundary of the current discriminator in training, and we call this method boundary-seeking GANs (BS-GAN). This approach can be used to train a generator with discrete output in the case that the generator is parametrized by a conditional distribution, and we demonstrate this with discrete image data. We also observe that the Gumbel-softmax trick does not work for training GANs with discrete data. Finally, our approach suggests a new objective function even for continuously valued data, and we demonstrate this with common image datasets. *Speaker:* Laurent Dinh *Title: *Sharp Minima Can Generative For Deep Nets *Abstract: *Despite their overwhelming capacity to overfit, deep learning architectures tend to generalize relatively well to unseen data, allowing them to be deployed in practice. However, explaining why this is the case is still an open area of research. One standing hypothesis that is gaining popularity, e.g. Hochreiter & Schmidhuber (1997); Keskar et al. (2017), is that the flatness of minima of the loss function found by stochastic gradient based methods results in good generalization. This paper argues that most notions of flatness are problematic for deep models and can not be directly applied to explain generalization. Specifically, when focusing on deep networks with rectifier units, we can exploit the particular geometry of parameter space induced by the inherent symmetries that these architectures exhibit to build equivalent models corresponding to arbitrarily sharper minima. Or, depending on the definition of flatness, it is the same for any given minimum. Furthermore, if we allow to reparametrize a function, the geometry of its parameters can change drastically without affecting its generalization properties. Best, Dima

2 3

Jörn Diedrichsen on "The brain’s GPU? In search of the cerebellum’s universal computation", March 10, 13:30, AA6214
by Junyoung Chung 10 Mar '17

10 Mar '17

Hi all, the next tea talk will be given by a guest speaker, Jörn Diedrichsen from UWO. Same place, same time: March 10, at 13:30-14:30, room AA6214. Hope to see many of you there! Title: The brain’s GPU? In search of the cerebellum’s universal computation Abstract: The cerebellum exhibits a highly specialized and uniform neuronal circuitry, which likely evolved to solve very specific problems in sensory-motor control. In the human brain, the cerebellar circuitry has dramatically expanded, and it contributes here to virtually every possible cognitive function. But what is this elusive computation that the cerebellum contributes to cortical processing? I will show some insights from our research on motor control and learning, which shows that the cerebellum is critically involved in prediction and fast error-based learning. I will then talk about the first steps that we have recently taken to test computational theories about the function of the cerebellum across different cognitive domains using functional imaging. Bio: Jörn Diedrichsen is a Wester Research Chair in the Brain and Mind Institute at Western University, and Professor of Statistics and Computer Science. His laboratory studies the cerebellar function and motor control and motor learning in the human brain. He has also developed a number of novel multivariate analysis techniques for functional imaging data. Best, --Junyoung

3 4

[Extra Tea-Talk] Martin Arjovsky, March 7, 14:30, AA6214
by Dzmitry Bahdanau 07 Mar '17

07 Mar '17

Hi all, This Tuesday (March 7) we will have an extra tea-talk by *Martin Arjovsky*. Please note that the time is different (14:30), but the place is the same (AA6214). Hope to see many of you there! *Title: *On Different Distances Between Distributions and Generative Adversarial Networks *Abstract: *Generative adversarial networks (GANs) are notoriously difficult to train. At the core of it, we show that these problems arise naturally when trying to learn distributions whose support lie in low dimensional manifolds. We show how these problems are consequences of trying to optimize the classical divergences (KL, JSD, etc) between our real and data distribution, and that these are symptoms of a more general phenomenon, pointing towards the inefficacy of the usual divergences in certain settings. After that, we bring into play the Wasserstein distance, which we prove doesn't suffer from the same behaviour, and provide a first step on an algorithm that tries to approximately optimize this distance. Dima

2 2

[TeaTalk] Zhouhan Lin, March 3, 13:30, AA6214
by Dzmitry Bahdanau 03 Mar '17

03 Mar '17

Hi all, Our next speaker is Zhouhan Lin, a PhD student from MILA. The time and place are usual: March 3, at 1:30pm, room AA6214. Home to see many of you there! *Title: *A Structured Self-Attentive Sentence Embedding *Abstract: *This paper proposes a new model for extracting an interpretable sentence embedding by introducing self-attention. Instead of using a vector, we use a 2-D matrix to represent the embedding, with each row of the matrix attending on a different part of the sentence. We also propose a self-attention mechanism and a special regularization term for the model. As a side effect, the embedding comes with an easy way of visualizing what specific parts of the sentence are encoded into the embedding. We evaluate our model on 3 different tasks: author profiling, sentiment classification and textual entailment. Results show that our model yields a significant performance gain compared to other sentence embedding methods in all of the 3 tasks Dima

1 3

[TeaTalk] Jean-Marc Rousseau, Feb 17, 13:30, AA6214
by Dzmitry Bahdanau 17 Fév '17

17 Fév '17

Hi all, The next tea-talk will be given by Jean-Marc Rousseau on February 17, at 1:30pm, room AA6214. Jean-Marc is the Technology Transfer Director of IVADO and will tell us about entrepreneurship and IVADO. *Title:* Starting a start up; what is involved Best, Dima

2 2

Fwd: Colloque DIRO, jeudi 16 février, Ted Meeds (Vrije U, Amsterdam)
by Yoshua Bengio 14 Fév '17

14 Fév '17

Another ML professor job talk on Thursday. There might be some time for students or postdocs interested in meeting our visitor, so let Linda know if you are interested. ---------- Forwarded message ---------- From: Pierre McKenzie <mckenzie(a)iro.umontreal.ca> Date: 2017-02-14 14:23 GMT-05:00 Subject: Colloque DIRO, jeudi 16 février, Ted Meeds (Vrije U, Amsterdam) To: seminaires(a)iro.umontreal.ca Cc: veronique.sage(a)umontreal.ca *Modeling science with machine learning: from inference in simulation-based models to understanding cancer.* par * Ted Meeds * Vrije Universiteit Amsterdam *Jeudi 16 février, 15:30-16:30*, *Salle 6214*, Pavillon André-Aisenstadt Université de Montréal, 2920 Chemin de la Tour Café avant 15:00-15:30 * Cette conférence sera donnée en anglais.* *Résumé:* In this talk I will explore the role that machine learning models, algorithms, and inference procedures play in several areas of science. Likelihood-free inference, or approximate Bayesian computation (ABC) is a framework for performing Bayesian inference in a wide variety of simulation-based sciences, from population genetics to computational psychology. I will first describe recent algorithmic improvements in ABC inference which are due in part from viewing inference from a machine learning perspective. I will then describe ongoing work using deep neural networks to model heterogeneous pan-cancer data and challenging tumour segmentation tasks whose long term goal is targeted immunotherapy treatment. Finally I will touch upon recent work on compressing neural networks using simple machine learning techniques, resulting in sparse and quantized parameters which, when applied to scientific domains, provides parsimonious models that are simple enough to be understood by experts.

1 0

Alex Shestopaloff, Feb 10, 13:30, AA6214
by Junyoung Chung 10 Fév '17

10 Fév '17

Hi all, we will have a talk by Alex Shestopaoff on this Friday. Hope to see many of you there! :) *Title:* Sampling latent states for high-dimensional non-linear state space models with the embedded HMM method *Abstract:* We propose a new scheme for selecting pool states for the embedded Hidden Markov Model (HMM) Markov Chain Monte Carlo (MCMC) method. This new scheme allows the embedded HMM method to be used for efficient sampling of state sequences in state-space models where the state can be high-dimensional. Previously, embedded HMM methods were only applied to models with a one-dimensional state space. We demonstrate that using our proposed pool state selection scheme, an embedded HMM sampler can have similar performance to a well-tuned sampler that uses a combination of Particle Gibbs with Backward Sampling (PGBS) and Metropolis updates. The scaling to higher dimensions is made possible by selecting pool states locally near the current value of the state sequence. The proposed pool state selection scheme also allows each iteration of the embedded HMM sampler to take time linear in the number of the pool states, as opposed to quadratic as in the original embedded HMM sampler. We also consider a model with a multimodal posterior, and show how a technique we term ``mirroring'' can be used to efficiently move between the modes. (Joint work with Radford M. Neal). *Bio:* Alex has graduated from U. of T (Prof. Radford Neal's group). Best, --Junyoung

1 1

[TeaTalk] Prof. Hervé Lombaert, Feb 3, 13:30, AA6214
by Dzmitry Bahdanau 03 Fév '17

03 Fév '17

Hi all, You are invited to attend the talk by Prof. Hervé Lombaert this Friday, Feb 3, at 13:30, room AA6214. *Title:* Spectral Matching & Learning of Surface Data - Example on Brain Surfaces *Abstract:* How to analyze complex shapes, such as of the highly folded surface of the brain? In this talk, I will show how spectral representations of shapes can benefit learning problems where data lives on surfaces. Key operations, such as segmentation and registration, typically need a common mapping of surfaces, often obtained via slow and complex mesh deformations in a Euclidean space. Here, we exploit spectral coordinates derived from the Laplacian eigenfunctions of shapes and also address the inherent instability of spectral shape decompositions. Spectral coordinates have the advantage over Euclidean coordinates, to be geometry aware and to parameterize surfaces explicitly. This change of paradigm, from Euclidean to spectral representations, enables a classifier to be applied *directly* on surface data, via spectral coordinates. The talk will focus, first, on spectral representations of shapes, with an example on brain surface matching, and second, on the learning of surface data, with an example on automatic brain surface parcellation. *Speaker's bio: *Hervé Lombaert is a Starting Research Scientist at Inria Sophia-Antipolis, France, and Associate Professor at ETS, Montreal - with research interests in Statistics on Shapes, Data & Medical Images. He had the chance to work in multiple centers, including Microsoft Research (Cambridge, UK), Siemens Corporate Research (Princeton, NJ), Inria Sophia-Antipolis (France), McGill University (Canada), and Polytechnique Montreal (Canada). He is also a recipient of the François Erbsmann Prize, a top prize in Medical Image Analysis, earned the select NSERC Postdoctoral Fellowship and the FQRNT Étudiant-Chercheur Étoile - more at [ http://cim.mcgill.ca/~lombaert] Best, Dima

1 1

[TeaTalk] Talks by Çağlar Gülçehre and Vincent Dumoulin
by Dzmitry Bahdanau 26 Jan '17

26 Jan '17

Hi all, Next Friday (27.01) at 1:30pm, room 6214, we will have not just one, but two tea-talk presentations by Çağlar and Vincent, both MILA students. Please come in great numbers! Speaker 1: Çağlar Gülçehre Title: TARDIS: an RNN with Wormhole Connections Abstract: In this talk, I will discuss a new memory augmented model which we have been working on with Sarath and Yoshua since last summer. Basically we revisit the idea of memory augmented models and investigate some of their aspects theoretically and propose a new model. The presentation will be about an ongoing work(although the arXiv paper will be out soon). Basically, we revisit the idea of memory augmented models and investigate some of their aspects theoretically and propose a new model. In this paper, we propose a novel memory augmented neural network model called TARDIS (Temporal Automatic Relation Discovery in Sequences). Our model (TARDIS) has an RNN controller which modifies an external memory component where it can store a selective set of embeddings of its own previous hidden states and revisit them as and when needed. In terms of the memory structure, our proposed model is similar to Dynamic Neural Turing Machines ~(D-NTM), but both read and write operations are simpler and more efficient. TARDIS uses discrete addressing for read/write operations which helps to substantially reduce the vanishing gradient problem with very long sequences. We evaluate our models on different long-term dependency tasks and report competitive results in all of them. Speaker 2: Vincent Dumoulin Title: A Learned Representation for Artistic Style Abstract: In this work (done while interning at Google Brain and submitted to ICLR2017) we investigate the construction of a single, scalable deep network that can parsimoniously capture the artistic style of a diversity of paintings. We demonstrate that such a network generalizes across a diversity of artistic styles by reducing a painting to a point in an embedding space. Importantly, this model permits a user to explore new painting styles by arbitrarily combining the styles learned from individual paintings. Best, Dima

1 1

Fwd: Colloque DIRO, jeudi 26 janvier, Pooyan Fazli (Cleveland State U.)
by Yoshua Bengio 21 Jan '17

21 Jan '17

More robotics + machine learning next Thursday, another job talk. ---------- Forwarded message ---------- From: Pierre McKenzie <mckenzie(a)iro.umontreal.ca> Date: 2017-01-20 11:55 GMT-05:00 Subject: Colloque DIRO, jeudi 26 janvier, Pooyan Fazli (Cleveland State U.) To: seminaires(a)iro.umontreal.ca *Distributed Robotics: Algorithms and Applications * par * Pooyan Fazli * Cleveland State University *Jeudi 26 janvier, 15:30-16:30*, *Salle 6214*, Pavillon André-Aisenstadt Université de Montréal, 2920 Chemin de la Tour Café avant 15:00-15:30 *Cette présentation sera donnée en anglais.* *Résumé:* Future robots will have to work in dynamic, uncertain, adversarial, and multi-goal environments. Thus, the integration of the advanced physical and cognitive systems required by the next generation of robots is a challenging task. It is not feasible to design a ‘universal’ robot capable of working across a wide range of applications. The robots of today are examples of the challenges of cost, long training period, and limited functionality, which will apply to robots of the near future as well. Given these challenges, multi-robot systems may be more suitable than single-robot systems in many real world applications. In this talk, I shall discuss two challenging problems in distributed robotics: 1) Service robots frequently face similar tasks. However, they are still not able to share their knowledge efficiently on how to accomplish those tasks. We introduce a new framework, which allows remote and heterogeneous robots to share plans and instructions on the tasks assigned to them. We demonstrate the feasibility of the framework with experiments between two geographically distributed robots and analyze the performance of the framework quantitatively. 2) In distributed coverage, a team of robots is deployed to move around a target area to perform sensing, monitoring, data collection, search, or distributed servicing tasks. We focus on the problem of multi-robot coverage and patrolling of the boundaries of a target area and the structures inside it. Events may occur at any position on the boundaries, and the goal of the robot team is to maximize the total detection reward of the events. To this end, we introduce an online, distributed algorithm, in which each robot autonomously learns the event distribution on the boundaries and then plans accordingly to visit the most promising parts of the target area. We also investigate how robots’ visual range and communication among the robots affect the performance of the robot team in the coverage problem. *Biographie:* Pooyan Fazli is an assistant professor and the founding director of the People and Robots Laboratory (PeRL) in the Electrical Engineering and Computer Science Department at Cleveland State University. He received his Ph.D. in Computer Science from the University of British Columbia and his M.Sc. and B.Sc. in Computer Science from Amirkabir University of Technology. Prior to joining CSU, he was a postdoctoral fellow in the CORAL Research Group at Carnegie Mellon University and in the Laboratory for Computational Intelligence at the University of British Columbia. His research focuses on artificial intelligence, autonomous robots, multi-robot systems, human-robot teams/interaction, machine learning, and robot vision and perception.

1 0

[TeaTalk] Talk by Professor Brendan Frey on 'Using Machine Learning to Detect and Treat Genetic Disease'
by Junyoung Chung 20 Jan '17

20 Jan '17

Hi everyone, we will have our first tea talk on this Friday. Please NOTE that this tea talk is scheduled at **13:30-14:30** in **AA6214**. We are in a transition process of moving the tea talk time slot from 11:00-1200 to 13:30-14:30. Professor Brendan Frey from U of T will tell us about his work on applying machine learning to detect and treat genetic disease. If anyone is interested in meeting with Brendan, please tell Dima or me, we will host a short meeting after his talk. Who : Professor Brendan Frey When : 20/1/2017 13:30-14:30 Where: AA6214 Title: Using Machine Learning to Detect and Treat Genetic Disease Abstract: Diseases and corresponding treatments often have to do with genetic modifications, such as changing a T to a G at a particular location within DNA. In 2015, my University of Toronto group spun out a startup, Deep Genomics, which is using machine learning to model how these changes impact cellular processes involved in disease, such as cancer and neurological disorders. For example, to train our splicing model, we are using 9661 different datasets, each with a number of training cases that varies from 8 million to 220 million. In this talk, I'll describe some of the machine learning challenges that, if solved, will completely change the field of medicine, save hundreds of thousands of lives, and improve the quality of life for hundreds of millions of people. Biography: Brendan Frey is the CEO and Co-Founder of Deep Genomics and a Professor in Engineering and Computer Science at the University of Toronto. He has made fundamental contributions in the areas of machine learning and genome biology, and is widely known for his work on using machine learning to understand how genetic variation leads to disease. Dr. Frey has received numerous distinctions and is a Fellow of the Royal Society of Canada, a Senior Fellow of the Canadian Institute for Advanced Research, and a John C. Polanyi Fellow. He recently co-founded a University of Toronto spinoff, Deep Genomics, which is using machine learning to bring about a new approach to genomic medicine. Dr. Frey has consulted for several industrial research and development laboratories in Canada, the United States and England, and has served on the Technical Advisory Board of Microsoft Research. His former students and postdoctoral fellows include professors, industrial researchers and developers at universities and industrial laboratories from across Canada, the United States and Europe. Best, --Junyoung

1 1

New student orientation this Friday
by Junyoung Chung 10 Jan '17

10 Jan '17

Hi everyone, we will not have a tea talk on this Friday, but instead there will be an orientation for new students at 11AM, AA3195. Best, --Junyoung

1 0