- Lisa_teatalk - mailman.iro.umontreal.ca

[Tea Talk] Rosemary Ke (MILA) Nov 10 10:30AM AA6214
by Michael Noukhovitch 10 Nov '17

10 Nov '17

This week we have our very own *Rosemary Ke* giving a talk on *Friday Nov 10* at* 10:30AM* in room *AA6214*. See you there! Michael *KEYWORDS* RNN, BPTT, Attention Mechanism *TITLE *Sparse Attentive Backtracking: Long-Range Credit Assignment in Recurrent Networks *ABSTRACT* A major drawback of backpropagation through time (BPTT) is the difficulty of learning long-term dependencies, coming from having to propagate credit information backwards through every single step of the forward computation. This makes BPTT both computationally impractical and biologically implausible. For this reason, full backpropagation through time is rarely used on long sequences, and truncated backpropagation through time is used as a heuristic. However, this usually leads to biased estimates of the gradient in which longer term dependencies are ignored. Addressing this issue, we propose an alternative algorithm, Sparse Attentive Backtracking, which might also be related to principles used by brains to learn long-term dependencies. Sparse Attentive Backtracking learns an attention mechanism over the hidden states of the past and selectively backpropagates through paths with high attention weights. This allows the model to learn long term dependencies while only backtracking for a small number of time steps, not just from the recent past but also from attended relevant past states.

1 1

[Tea Talk] Pierre-Yves Lablanche (AIMS-South Africa) Nov 3 10:30AM AA6214
by Michael Noukhovitch 03 Nov '17

03 Nov '17

This week we have a Post-Doc from AIMS-South Africa, *Pierre-Yves Lablanche,* giving a talk on *Nov 3* at* 10:30AM* in room *AA6214* If anyone is interested in talking with Pierre-Yves about collaborating, hearing about different datasets/challenges, or is just interested in having a 1:1 chat, email me and I'll set it up! Sorry for the late email! Michael *KEYWORDS *Astronomy/Astrophysics, Statistical ML, Big Data *TITLE *Machine Learning for Astronomy *ABSTRACT* Astronomy and astrophysics like many other scientific field is entering the big data era. Observational instruments such as the Large Synoptic Survey Telescope (LSST) and the Square Kilometre Array (SKA) will produce hundreds of petabytes of data per year supporting many scientific projects and existing analysis and processing tools will not be sufficient to tackle the upcoming “big data” challenge. As a consequence a lot of focus has been given to machine learning techniques both supervised and unsupervised over the past years. In this talk I will present a case study of galaxies identification using unsupervised learning and discuss the results evaluation as well as the method application to future astronomical data. In addition I will give insights about some of the latest machine learning technique being used and developed within the astronomical community. *BIO* Dr. Pierre-Yves Lablanche, born in Saint-Etienne in France, completed a BSc. in Physics and Chemistry at the Université Jean Monnet in Saint- Etienne in 2006 and a MSc. in Physics at the Univsersité Lyon 1 in 2008. He did his Ph.D. in astrophysics between Lyon and the European Southern Observatory (Münich, Germany) investigating the dynamics of barred galaxies via N-body simulations until 2012. The Ph.D. years were followed by two years of work as a scientific educator in a local association in Saint-Chamond (France) before volunteering to become a tutor at the African Institute for Mathematical Sciences (AIMS) in Arusha (Tanzania) in September 2014. A year later he joined the AIMS research centre in Cape Town (South Africa) where he is still working as a postdoctoral research fellow. Although his Ph.D. focused on astrophysics, Pierre-Yves nowadays develop and applies machine learning techniques in various field such as astronomy, anthropology, text analysis and infrastructure development. Aside from his research he is involved in many science outreach activities in France, Tanzania and South Africa and collaborates with the International Astronomical Union Office of Astronomy for Development on various international projects. Dr. Lablanche more specifically focuses on activities toward minorities and disabled people humbly trying to make a little difference everywhere he goes, strongly believing that “Education is the most powerful weapon which you can use to change the world” (Nelson Mandela).

1 1

[Tea Talk] Cancelled for ICLR
by Michael Noukhovitch 23 Oct '17

23 Oct '17

Tea talk this Friday is *cancelled* due to the ICLR deadline, good luck to everyone rushing! Michael

1 0

[Tea Tutorial] Guillaume Alain (MILA) Oct 20th 10:30AM AA6214
by Michael Noukhovitch 17 Oct '17

17 Oct '17

This week we won't really have a tea talk, but instead will have our very own *Guillaume Alain* giving a *tea tutorial* on *Friday Oct 20th* at* 10:30AM* in room *AA6214*. Guillaume will be going through the *ML Research Workflow*! <https://xkcd.com/1172/> Should be really useful especially for those new to the lab. See you there! Michael

2 1

[Tea Talk] Hui Jiang (York University), Fri Oct 13, 10:30AM, AA6214
by Michael Noukhovitch 11 Oct '17

11 Oct '17

This week we have a professor from York University, *Hui Jiang,* giving a talk on *Friday Oct 13* at* 10:30AM* in room *AA6214*. Don't be scared just because it's Friday the 13th in December! Come to the tea talk! Michael *KEYWORDS* NLP, Word Embedding, Language Modelling *TITLE* A New General Deep Learning Approach for Natural Language Processing *ABSTRACT* Word embedding techniques, representing each discrete word as a dense vector in continuous high-dimension space, have achieved successes in many natural language processing (NLP) tasks. However, more NLP tasks rely on modelling variable-length sequences of words, not just isolated words. The conventional approach is to formulate these NLP tasks as sequence labelling problems and apply conditional random fields (CRF), convolutional neural networks (CNN) and recurrent neural networks (RNN). In this talk, I will introduce a new general deep learning approach applicable to almost all NLP tasks, not limited to sequence labelling problems. The proposed method is built upon a simple but theoretically-guaranteed lossless encoding method, namely fixed-size ordinally-forgetting encoding (FOFE), which can almost uniquely encode any variable-length word sequence into fixed-size representation [1]. Next, simple feedforward neural networks are used as universal function approximators to map fixed-size FOFE codes to various NLP targets. This framework is appealing since it is elegant and well-founded in theory and meanwhile fairly easy and fast to train in practice. It is totally data-driven without any feature engineering, and equally applicable to a wide range of NLP tasks. In this talk, I will introduce our recent work to apply this approach to several important NLP tasks, such as word embedding [4], language modelling [1], named entity recognition (NER) and mention detection [2], coreference resolution and text categorization. Experiments have shown that the proposed approach yields strong performance in all examined tasks, including Google 1-billion-word language modelling, TAC KBP EDL contests, Pronoun Disambiguation Problem (PDP) in Winograd Schema Challenge [3]. Finally, I will briefly mention some on-going work to apply this approach to more NLP problems, such as factoid Q/A, word sense disambiguation (WSD), parsing and machine translation. References: [1] S. Zhang, H. Jiang, M. Xu, J. Hou, L. Dai, “The Fixed-Size Ordinally-Forgetting Encoding Method for Neural Network Language Models,” Proc. of ACL 2015. [2] M. Xu, H. Jiang and S. Watcharawittayakul, “A Local Detection Approach for Named Entity Recognition and Mention Detection,” Proc. of ACL 2017. [3] Q. Liu, H. Jiang, A. Evdokimov, Z. Ling, X. Zhu, S. Wei and Y. Hu, "Cause-Effect Knowledge Acquisition and Neural Association Model for Solving A Set of Winograd Schema Problems," Proc. of IJCAI 2017. [4] J. Sanu, M. Xu, Hui Jiang and Q. Liu, “Word Embeddings based on Fixed-Size Ordinally Forgetting Encoding,” Proc. of EMNLP 2017. *BIO* Hui Jiang received B.Eng. and M.Eng. degrees from University of Science and Technology of China (USTC), China and the Ph.D. degree from the University of Tokyo, Japan, all in electrical engineering. Since 2002, he has been working at Department of Electrical Engineering and Computer Science, York University, Toronto, Canada, initially as an assistant professor, then an associate professor and currently a full professor. His current research interests include machine learning, especially deep learning or neural networks, with its applications to speech and audio processing, natural language processing and computer vision. He served as an associate editor for IEEE Trans. on Audio, Speech and Language Processing (T-ASLP) between 2009-2013, and some technical committees of international conferences. He has recently received the 2016 IEEE Signal Processing Society (SPS) Best Paper Award.

1 0

Talk by David Grangier thursday 15h30 room 1360
by Pascal Vincent 11 Oct '17

11 Oct '17

Hello MILA friends, A reminder that tomorrow thursday at 15:30 we will have a talk by David Grangier on a novel and much faster seq2seq architecture for translation "Neural Machine Translation: Achieving Fast Training and Fast Inference with Gated Convolutions." This is part of the "colloques du DIRO series", so there is a traditional coffee+biscuits half-an-hour before. Reposting the full announcement below ( url: http://diro.umontreal.ca/ departement/colloques/une-nouvelle/news/david-grangier- neural-machine-translation-achie-42551/ ) Conférencier : David Grangier, Facebook AI Research, Menlo Park, CA Titre : Neural Machine Translation: Achieving Fast Training and Fast Inference with Gated Convolutions Travail conjoint avec Michael Auli, Yann Dauphin, Angela Fan, Jonas Ghering et Sergey Edunov Local : André-Aisenstadt 1360 Date et heure : Jeudi 12 octobre, 15h30, café-biscuits à 15h Résumé : Neural architectures for Machine Translation (MT) and related language modeling tasks is an active research field. The first part of our talk introduces several architectural changes to the original work of Bahdanau et al. 2014. We replace non-linearities with our novel gated linear units, recurrent units with convolutions and introduce multi-hop attention to allow more complex attention patterns. These changes improve generalization performance, training efficiency and decoding speed. The second part of our talk analyzes the properties of the distribution predicted by the model, examine how predictions differ from their empirical counterpart and we discuss how this influences beam search. Biographie du conférencier : David Grangier is a research scientist at Facebook AI Research, Menlo Park, CA. David earned his PhD in Machine Learning from Ecole Polytechnique Federale de Lausanne advised by Samy Bengio. He worked at different industrial labs, including NEC Labs America (2008-2011), AT&T Research (2011-2012) and Microsoft Research (2012-2014). Currently, David works on machine learning and its application to natural language processing, he is particularly interested in text generation tasks. http://david.grangier.info/ Pour vous préparer, vous pourriez lire... Convolutional Sequence to Sequence Learning Jonas Gehring, Michael Auli, David Grangier, Denis Yarats, Yann N. Dauphin - International Conference on Machine Learning (ICML). 2017. Language Modeling with Gated Convolutional Networks Yann N. Dauphin, Angela Fan, Michael Auli and David Grangier - International Conference on Machine Learning (ICML). 2017. Efficient softmax approximation for GPUs Edouard Grave, Armand Joulin, Moustapha Cisse and David Grangier and Hervé Jegou - International Conference on Machine Learning (ICML). 2017. A Convolutional Encoder Model for Neural Machine Translation Jonas Gehring, Michael Auli, David Grangier, Yann N. Dauphin - Conference of the Association for Computational Linguistics (ACL). 2017. Neural Generation of Text from Structured Data with Application to the Bibliography Domain Remi Lebret, David Grangier and Michael Auli - Conference on Empirical Methods in Natural Language Processing (EMNLP). 2016. Vocabulary Selection Strategies for Neural Machine Translation Gurvan L'Hostis, David Grangier, Michael Auli - arXiv:1610.00072. 2016. Strategies for Training Large Vocabulary Neural Language Models W. Chen, D. Grangier and M. Auli - Conference of the Association for Computational Linguistics (ACL). 2016.

1 0

[Tea Talk] Clement Farabet (NVIDIA) Mon Oct 9, 2pm, Claire-McNicoll Z-209
by Michael Noukhovitch 09 Oct '17

09 Oct '17

Next week, we have a flurry of tea talks and it all starts with VP of AI infrastructure at NVIDIA, *Clement Farabet* on *Monday October 9th *at *2pm* at *Pavilion Claire McNicoll room Z-209*. There will be a *small social *after the event so you can talk directly with NVIDIA researchers! NVIDIA has also been incredibly kind and will bring *FREE FOOD* as well as hold a *RAFFLE!* I don't know what the raffle is for, but will leave it your imagination ;) This is an awesome event, and hope everyone can make it! If you plan to attend, please sign up on their eventbrite link: https://www.eventbrite.com/e/mila-tea-talk-tickets-38158869205 *KEYWORDS *Infrastructure, Self-driving Cars, Domain Transfer *TITLE* Deep Learning Research at NVIDIA *ABSTRACT* NVIDIA is the leading platform for Deep Learning research and invests in a broad range of research projects within key industries, such as graphics and self-driving cars. In this talk I will discuss a few of our projects in-depth: from leveraging synthetic data to reduce our need on data; ray-tracing for real-time virtual worlds; auto face animation for game designers; and auto image transforms for photo editing. I will also talk about how we apply this research work into a product and operate a fast-moving R&D team to build autonomous vehicles. *BIO* Clement Farabet is VP of AI Infrastructure at NVIDIA. His team is responsible for building NVIDIA's next-generation AI platform, leveraging NVIDIA's hardware to enable a broad range of new applications, ranging from self-driving cars to medical imaging. Clement Farabet received a Master’s Degree in Electrical Engineering with honors from Institut National des Sciences Appliquées (INSA) de Lyon, France in 2008. His Master’s thesis work on reconfigurable hardware for deep neural networks was developed at the Courant Institute of Mathematical Sciences of New York University with Professor Yann LeCun, and led to a patent. He then joined Professor Yann LeCun’s laboratory in 2008, as a research scientist. In 2009, he started collaborating with Yale University’s e-Lab, led by Professor Eugenio Culurciello. This joint work later led to the creation of TeraDeep ( www.teradeep.com) In 2010, he started the PhD program at Université Paris-Est, co-advised by Professors Laurent Najman and Yann LeCun. His thesis focused on real-time image understanding/parsing with deep convolutional networks. The main contributions of his thesis were multi-scale convolutional networks and graph-based techniques for efficient segmentations of class prediction maps. He graduated in 2013, and went on to cofound Madbits, a company that focused on representing, understanding and connecting images. Madbits was acquired by Twitter in 2014. At Twitter, he cofounded Cortex, a team of software engineers, data scientists, and research scientists dedicated to developing state-of-the-art machine learning capabilities to refine and enable new products. He subsequently lead and managed a team called Cortex Core, which focused on building a high-leverage modular machine/deep learning platform to power every aspect of the Twitter product (recommendation systems, search, timeline ranking, etc.). This team focused on (1) developing models of text, images, video, users, and (2) making these models seamlessly importable as components of user-facing ML systems.

1 2

[Tea Talk] Phil Bachman (Maluuba MS), Fri Oct 6, 10:30AM, AA6214
by Michael Noukhovitch 06 Oct '17

06 Oct '17

This week we have a researcher from Maluuba Microsoft, *Phil Bachman,* giving a talk on *Friday Oct 6* at the new, earlier time: *10:30AM* in room *AA6214*. See you there! Michael *KEYWORDS* Active learning, Meta-learning, Bayesian *TITLE* Metalearning and active learning -- like chocolate and peanut butter. *ABSTRACT* This talk expounds on the natural synergy of meta and active learning. The general scheme of metalearning is to swap hard-coded procedures that output models for trainable models (that output models). For example, one can use many related classification problems to train a model which outputs a classifier given an input set of labeled examples. In effect, the trained "set2func" model replaces the role of, e.g., SGD in producing a classifier for each problem. >From a Bayesian perspective, the set2func model can be interpreted as encoding both a prior over classifiers and a procedure for performing inference w.r.t. the learned prior given some labeled data. Existing approaches to meta classification can be interpreted as learning a prior over classifiers, which is co-adapted with some mechanism for performing MAP inference. E.g., the "base" parameters learned by MAML give the mean of a Gaussian prior over parameterizations of the "base" model. Adapting the base parameters to a new problem instance via SGD can be interpreted as approximate posterior inference (see, e.g. "Early Stopping as Non-parametric Variational Inference"). Current metalearning methods perform MAP inference, which can be particularly limiting in the case of tiny sets of labeled data. Consider the Omniglot meta classification task. If we have unlabeled data for all classes in the current problem, but labeled data only for a subset of the classes, a MAP-based model won't explicitly represent the "cluster-based" label assignments on which this task is based. A "Bayesian" set2func model could express this property of the underlying task distribution through the classifiers assigned high likelihood by its cross-task prior and per-task posteriors. The role of active learning, in the setting of Bayesian metalearning, is to collect labeled examples for the current problem in a way that maximally reduces entropy in the posterior distribution over classifiers. Having an explicit representation of the relevant posterior should permit more effective active learning, and adapting the prior to the task distribution should permit tighter, more-accurate per-task posteriors. Conversely, encouraging the set2func model to represent its prior and posteriors in a way that facilitates active learning may improve the quality of the learned prior and posteriors. Developing models which perform well in the few-shot setting (metalearning), and which efficiently collect information to improve task performance (active learning), will be critical to success in settings like "life-long learning". This talk discusses these themes, and presents some related concrete results.

1 1

[Tea Talk] Ioannis Mitliagkas, Fri Sep 29, 10:30AM, AA6214
by Michael Noukhovitch 26 Sep '17

26 Sep '17

This week we have a cool new MILA professor, *Ioannis Mitliagkas,* giving a talk this *Friday Sep 29* at a new, earlier time: *10:30AM* in room *AA6214*. If time permits, he'll also be giving details of his winter course on topics in AI! Hope to see you all there! Don't forget about the earlier time!! - Michael *KEYWORDS* optimization, YellowFin, large scale DL, GAN stabilization *TITLE* Understanding momentum dynamics for faster training, better scaling, and easier tuning *ABSTRACT* This talk revolves around Polyak’s momentum gradient descent method, also known as ‘momentum’. Its stochastic version, momentum stochastic gradient descent (SGD), is one of the most commonly used optimization methods in deep learning. Throughout the talk we will study a number of important properties of this versatile method, and see how this understanding can be used to engineer better deep learning systems. I will first go over the basic formulation of momentum. Then I will summarize a theoretical result on a previously unknown connection between momentum dynamics and asynchronous optimization. Understanding this connection, allows us to improve the efficiency of large-scale deep learning systems. I will go over a recent collaboration with Intel and the National Energy Research Scientific Computing Center (NERSC) on a 15 PetaFLOP system consisting of 9600 nodes. Finally, I will demonstrate how analyzing the behavior of momentum on simple objectives can lead to tuning rules for its learning rate and momentum hyperparameters. Our implementation of these rules is called YellowFin and is a simple adaptive method that can handle different objectives, as well as varying asynchronous dynamics, without hand-tuning. Yellowfin often outperforms state-of-the-art adaptive methods. At the end of the talk, I will discuss some preliminary thoughts on the training dynamics of GANs and some ideas on how momentum dynamics can, again, play a key role in stabilizing adversarial training. Finally, if time permits, I will give an outline my other research interests, as well as a rough plan for my upcoming ’Topics in AI’ class. *BIO* Ioannis Mitliagkas is an assistant professor in the Department of Computer Science and Operations Research (DIRO) at the University of Montreal. Before that, he was a Postdoctoral Scholar with the Department of Statistics and Computer Science at Stanford University. He obtained his Ph.D. from the Department of Electrical and Computer Engineering at the University of Texas at Austin. His research focuses on statistical learning and inference problems, with work in efficient large-scale and distributed algorithms, theoretical and data-dependent guarantees and tuning complex systems. His recent work includes understanding and optimizing the scanning used in Gibbs sampling for inference, as well as understanding the interaction between optimization and the dynamics of large-scale learning systems.

1 0

Anirudh Goyal, September 15, AA6214, 11AM
by Michael Noukhovitch 15 Sep '17

15 Sep '17

Hi All! Dima is going on an internship so I'll be organizing tea talks for the next couple of months! This week, our very own *Anirudh* will present his NIPS paper (congrats btw!) it will be at *AA6214 on Friday September 15th, 11AM *Hope to see everyone there! *Title:* Z-Forcing: Training Stochastic Recurrent Networks *Abstract:* Many efforts have been devoted to training generative latent variable models with autoregressive decoders, such as recurrent neural networks (RNN). Stochastic recurrent models have been successful in capturing the variability observed in natural sequential data such as speech. We propose a novel sequential latent variable model unifying successful ideas from recently proposed architectures: each step in the sequence is associated with a latent variable that is used to condition the recurrent dynamics for future steps. Training is performed with amortised variational inference where the approximate posterior is augmented with a RNN that runs backward through the sequence. In addition to maximizing the variational lower bound, we ease training of the latent variables by adding an auxiliary cost which forces them to reconstruct the state of the backward recurrent network. This provides the latent variables with a task-independent objective that enhances the performance of the overall model. Although being conceptually simple, our model achieves state-of-the-art results on standard speech benchmarks such as TIMIT and BLIZZARD and competitive performance on sequential MNIST. Finally, we apply our model on language modeling in the IMDB dataset. The auxiliary cost is crucial for learning interpretable latent variables. Michael

2 2

[TeaTalk] Karan Grewal, September 8, 13:45, AA6214
by Dzmitry Bahdanau 08 Sep '17

08 Sep '17

Hi all, Our next speaker is Karan Grewal, who is currently an intern at MILA. The talk will take place at *AA6214, at 13:45, September 8*. Hope to see many of you there! *Title: *Variance Regularizing Adversarial Learning *Abstract:* Generative Adversarial Networks have become a breakthrough towards generating synthetic data in recent years, however many problems may arise during training which result in the generator being unable to learn the desired data distribution/manifold. Most notably, the discriminator often overpowers the generator, which leads to a poor training signal and vanishing gradients. We study the effects of the discriminator's unnormalized output distribution on the generator's ability to learn. To combat this problem, we propose Variance Regularizing Adversarial Learning -- regularizing the discriminator's unnormalized output distribution to fit a mixture of gaussians, yielding higher variance and hence a lipschitz-like discriminator function. To do this, we provide two methods: (1) using the KL-divergence as a penalty in the discriminator's loss and (2) playing meta-adversarial games to force the discriminator to fit a MoG. We show that our models are robust in the presence of high training ratios and compare them to other variants of GANs. *Bio:* Karan Grewal is an intern at MILA and a senior undergraduate student at the University of Toronto. He is working on improving and stabilizing generative models with Devon Hjelm and Yoshua Bengio. Previously, he worked on applying machine learning to social contexts. Dima

1 3

predoctoral thesis proposal, Taesup Kim, Tuesday 5 Sept, 15h30
by Yoshua Bengio 04 Sep '17

04 Sep '17

Hello, Taesup Kim, MILA PhD candidate, will present his predoctoral subject proposal Tuesday September 5 at 15h30 (room 3395, PAA): *Title: Dynamic Neural Networks: Learning to Adapt to DataAbstractRecent advances in deep neural networks have shown significant results on several machine learning and pattern recognition problems. While these achievements are impressive that some are already working on real-world products, there still exists a lot of potential innovations inspired by human behaviors and we cover in this proposal some related approaches to imitate human using deep neural networks involving both generative and discriminative models. Especially, we introduce some dynamic neural networks having dynamic modules that operate differently and adaptively over different data to improve both the optimization process and the performance. Related future works are also presented to show how the dynamic neural networks can be applied broadly over different domains and applications.* Please come and learn about his research and support him! *-- Yoshua*

1 0

predoctoral thesis proposal, Chinnadhurai Sankar, Tuesday 5 Sept, 13h30
by Yoshua Bengio 04 Sep '17

04 Sep '17

Hello, Chinnadhurai Sankar, MILA PhD candidate, will present his predoctoral subject proposal Tuesday September 5 at 13h30 (room 3395, PAA): *Title: Problems in Non-Goal Oriented Dialog Systems* *Abstract:* *This predoctoral presentation will mainly focus on data-driven approaches to non-goal oriented dialog systems research. I shall talk about two dialog systems projects that I have been part of. First, I shall propose a reinforcement learning based response selection model for dialogue management. The proposed method was implemented & tested in the Amazon Alexa Competition and shown to demonstrate superior performance when compared to traditional heuristics based approaches and supervised learning based approaches. **Next, I shall introduce a hybrid retrieval based generative dialog system architecture which is designed to provide a sense of specificity in the generated responses.* Please come and learn about his research and support him! *-- Yoshua*

1 0

[TeaTalk] Iulian Serban (Wednesday, Aug 30) and Andrew Jesson (Friday, Sept 1)
by Dzmitry Bahdanau 01 Sep '17

01 Sep '17

Hi all, We are lucky to have two speakers this week! First,* Iulian Serban* will tell us about the progress of MILA team in Alexa challenge (*Z-209 between 14h00 - 15h00, August 30th*). Next, we will learn about deep learning with extreme class imbalance from *Andrew Jesson *from Imagia (*AA6214, 13:45 - 14:45, September 1*). Please find the details below. Dima *Speaker:* Iulian Serban *Title:* A Deep Reinforcement Learning Chatbot *Abstract*: We present MILABOT: a deep reinforcement learning chatbot developed at MILA for the Amazon Alexa Prize competition. MILABOT is capable of conversing with humans on popular small talk topics through both speech and text. The system consists of an ensemble of natural language generation and retrieval models, including sequence-to-sequence and latent variable neural network models. By applying reinforcement learning to crowd-sourced data and real-world user interactions, the system has been trained to select an appropriate response from the models in its ensemble. The system has been evaluated through A/B testing with real-world users, where it performed excellent compared to competing systems. *Bio:* Iulian Serban is a PhD candidate at MILA, where he investigates deep learning and reinforcement learning methods for dialogue and other natural language processing applications. Iulian is supervised by Yoshua Bengio, Aaron Courville and Joelle Pineau. He holds an MSc in machine learning from University College London, and a BSc in theoretical mathematics and statistics from Copenhagen University. Previously, Iulian worked at DigiCorpus ApS, where he developed a physiotherapy computer vision system, now deployed in clinics across Denmark. *Speaker:* Andrew Jesson *Title:* CASED: Curriculum Adaptive Sampling for Extreme Data Imbalance *Abstract: *We introduce CASED, a novel curriculum sampling algorithm that facilitates the optimization of deep learning segmentation or detection models on data sets with extreme class imbalance. We evaluate the CASED learning framework on the task of lung nodule detection in chest CT. In contrast to two-stage solutions wherein nodule candidates are first proposed by a segmentation model and then refined by a second detection stage, CASED improves the training of deep nodule segmentation models (e.g. UNet) to the point where state of the art results are achieved using only a trivial detection stage. CASED improves the optimization of deep segmentation models by allowing them to first learn how to distinguish nodules from their immediate surroundings, while continuously adding a greater proportion of difficult-to-classify global context, until uniformly sampling from the empirical data distribution. Using CASED during training yields a minimalist proposal to the lung nodule detection problem competitive the LUNA16 nodule detection benchmark with an average sensitivity score of 88.7%. Furthermore, we find that models trained using CASED are robust to nodule annotation quality by showing that comparable results can be achieved when only a point and radius for each ground truth nodule are provided during training. Finally, the CASED learning framework makes no assumptions with regard to imaging modality or segmentation target and so should generalize to other medical imaging problems where class imbalance is a persistent problem. *Bio:* Andrew Jesson is the leader of the new A.I. Research team at Imagia. He is currently subitting his Master's thesis in electrical engineering with a focus on medical imaging at McGill University. Andrew has spent the last 12 years in Montreal where he has aslo earned a B.A. in philosophy from Concordia University and a B.Eng from McGill.

2 6

[TeaTalk] Yaniv Romano (Thursday, Aug 24) and Georgy Derevyanko (Friday, Aug 25)
by Dzmitry Bahdanau 25 Aoû '17

25 Aoû '17

Hi all, We have two tea-talks this week! First, at *13:30 on Thursday, August 24, at Z-209*, we will have a presentation by Yaniv Romano, who is a PhD student at Technion, Israel. Second, at *13:45 on Friday, August 25, at AA6214, Georgy Derevyanko*, who is currently a post-doc at Concordia and MILA, will share his research with us. Please find more details below, and hope to see you in great numbers! Dima *Speaker 1:* Yaniv Romano *Title: *A Quest for a Universal Model for Signals: From Sparsity to ConvNets *Abstract: *The celebrated sparse representation model assumes that a signal can be represented as a linear combination of a few columns, also called atoms, taken from a matrix, termed a dictionary. When dealing with high-dimensional signals, addressing the dictionary learning problem becomes computationally infeasible due to the curse of dimensionality. Traditionally, this problem was circumvented by learning a local sparse model on small overlapping patches, extracted from the global signal, and processing (e.g. denoising) these independently. We will start this talk by proposing various approaches to bridge the gap between the efficient independent local processing and the need to model the global signal at hand. In particular, we will describe novel image restoration algorithms, leading not only to state-of-the-art results, but also to a systematic and generic way to boost the performance of many existing algorithms. A different approach to treat high dimensional signals is the convolutional sparse coding (CSC). This global model assumes that a signal can be represented as a superposition of a few local atoms, or small filters, shifted to different positions. A recent work suggested a novel theoretical analysis of this global model, which is based on the observation that while being global, the CSC can be characterized and analyzed locally. We will extend this local-global relation by showing how one can efficiently solve the pursuit problem and train the filters involved, while operating locally on image patches. Armed with these new insights, we proceed by proposing a multi-layer extension of this model, ML-CSC, in which signals are assumed to emerge from a cascade of CSC layers. This, in turn, is shown to be tightly connected to Convolutional Neural Networks (CNN), so much so that the forward-pass of the CNN is in fact the Thresholding pursuit serving the ML-CSC model. This connection brings a fresh view to CNN, as we are able to attribute to this architecture theoretical claims such as uniqueness of the representations throughout the network, and their stable estimation, all guaranteed under simple local sparsity conditions. Lastly, identifying the weaknesses in the above scheme, we propose an alternative to the forward-pass algorithm, which is both tightly connected to deconvolutional and recurrent neural networks, and has better theoretical guarantees. *Bio: *Yaniv Romano received his B.Sc. degree from the Department of Electrical Engineering, Technion – Israel Institute of Technology, in 2012, where he is currently pursuing his Ph.D.. He received the 2015 Zeff fellowship, the 2017 Andrew and Erna Finci Viterbi fellowship, and the 2017 Irwin and Joan Jacobs fellowship. In parallel to his studies, he has been working in the industry since 2011 as an Image Processing Algorithm Developer. The super-resolution technology he invented as an intern in Google Research was launched in 2017, leading to significant bandwidth savings of billions of images. *Speaker 2: *Georgy Derevyanko *Title:* Protein folding project *Abstract:* Protein folding and structure prediction is a 50 years old problem. The solution to it will immediately change the whole industry of drug discovery and biology itself. This talk is a call for collaboration to advance the field of protein structure prediction using deep learning techniques. I will give an overview of the state-of-art algorithms in the field and their assessment procedures. Also, I will present some of our publication-ready and preliminary results. Moreover, I will present the library of differentiable transformations of a protein chemical structure and pre-processed datasets in order to lower the problem entry barrier*.* *Bio:* -2014 PhD in "Physics for life sciences" from Universite Joseph Fourier (Grenoble, France) -2016 PostDoc in experimental structural biology at Forshungszentrum Juelich (Juelich, Germany) -present PostDoc at Concordia CERMM & MILA

2 4

Yacine Jernite, August 4, AA6214, 13:45
by Junyoung Chung 04 Aoû '17

04 Aoû '17

Hi all, our next speaker is Yacine Jernite from NYU. The talk will be at AA6214 on August 4, 13:45. Title: Learning Representations of Language from Text Abstract: Learning good general-purpose representations of written language can be useful for a wide range of applications, but there are still a number of open questions in the field, from model architecture to learning strategies. In this talk, we start by briefly going over these questions, then review some recent work related to two of them: the choice of the reading level for text data and of a learning objective for unsupervised training. First, we describe two recent architectures which look at sub-word information to build word and text representations. Then, considering the prevalence of language modeling as an unsupervised objective, we propose an adaptive hierarchical approach which speeds up learning while avoiding some of the pitfalls of comparable methods. Finally, we propose an alternative discriminative learning objective for sentence representation systems based on discourse modeling. Bio: Yacine is currently pursuing a PhD at NYU in David Sontag's Clinical Machine Learning group, where his research interests include graphical models, applying machine learning to medical data, and language understanding. His current focus is on using unsupervised methods to take advantage of the vast amounts of freely available text data when learning language representation systems. Yacine holds a Diplome d'ingenieur from Ecole Polytechnique (2011) and an M.Sc. in Machine Learning from ENS Cachan (2012). Best, --Junyoung

2 3

[TeaTalk] James Wright, July 28, AA6214, 13:45
by Dzmitry Bahdanau 28 Jul '17

28 Jul '17

Hi all, Our next speaker is James Wright from Microsoft Research NYC. The talk will take place on July 28, at AA6214, 13:45. Hope to see many of there! Title: Deep Learning for Predicting Human Strategic Behavior Abstract: Predicting the behavior of human participants in strategic settings is an important problem in many domains. Most existing work either assumes that participants are perfectly rational, or attempts to directly model each participant's cognitive processes based on insights from cognitive psychology and experimental economics. In this work, we present an alternative, a deep learning approach that automatically performs cognitive modeling without relying on such expert knowledge. We introduce a novel architecture that allows a single network to generalize across different input and output dimensions by using matrix units rather than scalar units, and show that its performance significantly outperforms that of the previous state of the art, which relies on expert-constructed features. Bio: James Wright's research focuses on problems at the intersection of economics, behavioral modeling, and machine learning, with a focus on modeling and predicting human behavior in strategic settings. Prior to joining Microsoft Research NYC, he completed a Ph.D. in Computer Science at the University of British Columbia, where he was advised by Kevin Leyton-Brown. Dima

1 2

Eugene Vorontsov, July 21, AA6214, 13:45
by Junyoung Chung 21 Jul '17

21 Jul '17

Hi all, our next speaker is Eugene Vorontsov. Hope to see you many of you there. When: 13:45, July 21 Where: AA6214 Title: On orthogonality and learning recurrent networks with long term dependencies Abstract: It is well known that it is challenging to train deep neural networks and recurrent neural networks for tasks that exhibit long term dependencies. The vanishing or exploding gradient problem is a well known issue associated with these challenges. One approach to addressing vanishing and exploding gradients is to use either soft or hard constraints on weight matrices so as to encourage or enforce orthogonality. Orthogonal matrices preserve gradient norm during backpropagation and may therefore be a desirable property. This paper explores issues with optimization convergence, speed and gradient stability when encouraging or enforcing orthogonality. To perform this analysis, we propose a weight matrix factorization and parameterization strategy through which we can bound matrix norms and therein control the degree of expansivity induced during backpropagation. We find that hard constraints on orthogonality can negatively affect the speed of convergence and model performance. Bio: I am a PhD student with professors Chris Pal and Samuel Kadoury at École Polytechnique de Montréal and MILA. I am working on medical image segmentation models and have recently begun developing an interest in optimization and regularization of deep neural networks. Prior to MILA, I studied Engineering Science at the University of Toronto, specializing in biomedical engineering. An aspect of computer science that appealed to me is the fast production of experimental results -- unfortunately, I also like big models. --Junyoung

3 6

[TeaTalk] Ethan Perez, July 14, AA6214, 13:45
by Dzmitry Bahdanau 14 Jul '17

14 Jul '17

Hi all, Our next speaker is Ethan Perez, who is currently interning at MILA with Aaron Courville. Please come to AA6214 on July 14 at 13:45! *Title:* Learning Visual Reasoning Without Strong Priors *Abstract:* Achieving artificial visual reasoning - the ability to answer image-related questions which require a multi-step, high-level process - is an important step towards artificial general intelligence. This multi-modal task requires learning a question-dependent, structured reasoning process over images from language. Standard deep learning approaches tend to exploit biases in the data rather than learn this underlying structure, while leading methods learn to visually reason successfully but are hand-crafted for reasoning. We show that a general-purpose, Conditional Batch Normalization method achieves state-of-the-art results on the Compositional Language and Elementary Visual Reasoning (CLEVR) task with a 2.4% error rate. We outperform the next best end-to-end method (4.5%) which uses data augmentation and even methods that use extra supervision (3.1%). We probe our model to shed light on how it reasons, showing it has learned a question-dependent, multi-step process. Previous work has operated under the assumption that visual reasoning calls for a specialized architecture, but we show that a general architecture with proper conditioning can learn to visually reason effectively. *Bio:* Ethan Perez is a rising 4th year computer science undergrad at Rice University. He is currently interning at MILA working with Aaron Courville on Visual Reasoning. Previously, he has researched on deep semi-supervised learning methods and built machine learning models for location detection at Google Maps and fraud detection at Uber. Dima

1 1

Fwd: [Lisa_seminaires] Seminar in statistics and Monte Carlo Thursday July 13, 10h45
by Yoshua Bengio 12 Jul '17

12 Jul '17

---------- Message transféré ---------- De : Pierre L'Ecuyer <lecuyer(a)iro.umontreal.ca> Date : 13 juillet 2017 à 07:21 Objet : [Lisa_seminaires] Seminar in statistics and Monte Carlo Thursday July 13, 10h45 À : "lisa_seminaires(a)iro.umontreal.ca" <lisa_seminaires(a)iro.umontreal.ca> A seminar of interest for people in machine learning and in statistics: https://www.gerad.ca/fr/events/1479 -- Pierre L'Ecuyer, Professeur Titulaire Chaire du Canada en Simulation et Optimisation Stochastique CIRRELT, GERAD, and DIRO, Université de Montréal, Canada http://www.iro.umontreal.ca/~lecuyer _______________________________________________ Lisa_seminaires mailing list Lisa_seminaires(a)iro.umontreal.ca https://webmail.iro.umontreal.ca/mailman/listinfo/lisa_seminaires

1 0

[TeaTalk] Rémi Leblond & Jean-Baptiste Alayrac, July 7, AA6214, 13:45
by Dzmitry Bahdanau 07 Jul '17

07 Jul '17

Hi all, For those of wondering if there will be a tea-talk this Friday I have good news - the answer is yes! Our next speakers will be Rémi Leblond and Jean-Baptiste Alayrac, both PhD students at INRIA, working with Simon Lacoste-Julien. Please come to AA6214 on July 7 at 13:45! Please find more information below: 1) Searnn, Rémi Leblond *Title:* SeaRNN: Training RNNs with Global-Local Losses *Abstract:* We propose SEARNN, a novel training algorithm for recurrent neural networks (RNNs) inspired by the “learning to search” (L2S) approach to structured prediction. RNNs have been widely successful in structured prediction applications such as machine translation or parsing, and are commonly trained using maximum likelihood estimation (MLE). Unfortunately, this training loss is not always an appropriate surrogate for the test error: by only maximizing the ground truth probability, it fails to exploit the wealth of information offered by structured losses. Further, it introduces discrepancies between training and predicting (such as exposure bias) that may hurt test performance. Instead, SEARNN leverages test-alike search space exploration to introduce global-local losses that are closer to the test error. We demonstrate improved performance over MLE on three different tasks: OCR, spelling correction and text chunking. Finally, we propose a subsampling strategy to enable SEARNN to scale to large vocabulary sizes. *Bio:* Rémi is a second year Ph.D. student at INRIA in the SIERRA team under the supervision of Simon Lacoste-Julien <http://www.di.ens.fr/~slacoste/>. He's interested in very large scale optimization for new complex machine learning models. His current focus is on parallel and distributed algorithms and structured prediction using neural networks. After graduating from Ecole Polytechnique <https://www.polytechnique.edu/> and Corps des Mines (French MPA), he worked in the Big Data field for three years, as a Data Scientist at SpotRight <http://spotright.com/> in the US (2011-2012) and as a Data Engineer at the Ministry of Defense (2013-2015). 2) Joint discovery of objects and manipulation actions, JB Alayrac *Title:* Joint discovery of objects and manipulation actions *Abstract:* Many human activities involve object manipulations aiming to modify the object state. Examples of common state changes include full/empty bottle, open/closed door, and attached/detached car wheel. In this work, we seek to automatically discover the states of objects and the associated manipulation actions. Given a set of videos for a particular task, we propose a joint model that learns to identify object states and to localize state-modifying actions. Our model is formulated as a discriminative clustering cost with constraints. We assume a consistent temporal order for the changes in object states and manipulation actions, and introduce new optimization techniques to learn model parameters without additional supervision. We demonstrate successful discovery of seven manipulation actions and corresponding object states on a new dataset of videos depicting real-life object manipulations. We show that our joint formulation results in an improvement of object state discovery by action recognition and vice versa. *Bio:* Jean-Baptiste Alayrac received the MS degree in computer science in Ecole Normale Superieure (ENS), in Paris in 2014. He is currently working toward the PhD degree in the research teams WILLOW and SIERRA at INRIA Paris under the supervision of Josef Sivic, Ivan Laptev and Simon Lacoste-Julien. His research focuses on structured prediction from vision and natural language. Dima

2 3

Alexandros Dimakis, June 15th 11AM, AA6214
by Junyoung Chung 15 Jui '17

15 Jui '17

Hi all, we have another tea-talk next week. This talk will be given by Professor Alexandros Dimakis from the University of Texas at Austin. The talk will take place at *AA6214* at *11AM *on June 15th (*Thursday*). Please NOTE that there is a tea-talk on 13th (*Tuesday*) by Jackie C. K. Cheung (McGill University) at *Z109* (Pavillon Claire-McNicoll) at *2PM*. Check Dima's email for more detailed information about the talk. *Alexandros' talk.* *Title: *Generative Models and Compressed Sensing *Abstract: *The goal of compressed sensing is to estimate a vector from an underdetermined system of noisy linear measurements, by making use of prior knowledge on the structure of vectors in the relevant domain. For almost all results in this literature, the structure is represented by sparsity in a well-chosen basis. We show how to achieve guarantees similar to standard compressed sensing but without employing sparsity at all. Instead, we suppose that vectors lie near the range of a generative model, e.g. a GAN or a VAE. We show how the problems of image inpainting and super-resolution are special cases of our general framework. We show how to generalize the RIP condition for generative models and that random gaussian measurement matrices have this property with high probability. A Lipschitz condition for the generative neural network is a key technical condition. We will also discuss on-going work for adding causality and distributed training to these models. (based on joint work with Ashish Bora, Ajil Jalal and Eric Price) Code: https://github.com/AshishBora/csgm Homepage: users.ece.utexas.edu/~dimakis *Bio:* Alex Dimakis is an Associate Professor at the ECE department, University of Texas at Austin. He received his Ph.D. in 2008 from UC Berkeley and the Diploma degree from the National Technical University of Athens in 2003. During 2009 he was a CMI postdoctoral scholar at Caltech. He received an NSF Career award, a Google faculty research award and the Eli Jury dissertation award. He is the co-recipient of several best paper awards including the joint Information Theory and Communications Society Best Paper Award in 2012. He is currently serving as an associate editor for IEEE Transactions on Information Theory. His research interests include information theory, coding theory and machine learning. --Junyoung

1 3

[TeaTalk] Jackie Cheung, June 13, Z-209, 14:00
by Dzmitry Bahdanau 13 Jui '17

13 Jui '17

Hi all, Our next tea-talk will be given by Jackie C. K. Cheung, who is an assistant professor at McGill university. It will take place at Z-209 (Pavillon Claire-McNicoll), on June 13, at 14:00. Hope you to see you in great numbers! *Please note the unusual day and time of the talk!* *Title: *Leveraging External Knowledge for Machine Comprehension of Rare Entities *Abstract: *Thanks to the "long tail" of natural language, entities that only appear a handful of times in a training corpus account for the vast majority of entities overall. Standard techniques for training word embeddings such as word2vec and GloVe do not scale well to modelling these entities, because they depend on having many entity or word occurrences for reliable word co-occurrence statistics. We propose to use external knowledge in the form of short entity definitions as a source of information to train entity representations. We describe experiments in two settings. In the first, we present a simple scheme for initializing entity representations for knowledge base completion. In the second, we introduce a new machine comprehension dataset, the Wikilinks Rare Entities data set, and an accompanying entity cloze task which requires predicting rare named entities in context. We describe a hierarchical double encoder model which reads a text passage and an entity definition in order to predict which entity fits in context. Our model substantially outperforms a number of strong baselines which do not utilize entity definitions, demonstrating the impact of external knowledge in making inferences about rare entities. *Bio:* Jackie CK Cheung is an assistant professor in the School of Computer Science at McGill University. He received his Ph.D. at the University of Toronto, and was awarded a Facebook Fellowship for his doctoral research. He conducts research in computational semantics and natural language generation, with a focus on topics such as inducing event structure from distributional semantics, and automatic summarization. His work is funded by the Natural Sciences and Engineering Research Council of Canada (NSERC), and the Fonds de recherche du Québec - Nature et technologies (FRQNT). Dima

1 2

[TeaTalk] Danny Tarlow, June 9, 13:45, AA5340
by Dzmitry Bahdanau 09 Jui '17

09 Jui '17

Hi all, You have just received an email about the tea-talk on June 13, but I have good news for you: we will have one more before that! Our next (really next) speaker is *Danny Tarlow*, who is a Research Scientist at Google Brain Montreal. He will present on *June 9, 13:45, at AA5340* (our usual tea-talk slot). Please find detailed information below. For those who are confused, the coming tea-talks will be given on June 9, 13 and 15 (yet to be announced). *Title:* Learning to Code: Machine Learning for Program Induction *Abstract:* I'll present two of our recent works on using machine learning ideas to induce computer programs from input-output examples. The first system is TerpreT, which casts program synthesis as a continuous optimization problem on which we perform gradient descent. It enables comparison of gradient-based program synthesis techniques to discrete search techniques that are popular in the programming languages community. Based on our learnings from TerpreT, we develop the second system, DeepCoder, which induces programs from input-output examples using a neural network to guide discrete search techniques. DeepCoder achieves an order of magnitude speedup over optimized search techniques, and it can solve problems of difficulty comparable to the very simplest problems on programming competition websites. *Bio:* Danny Tarlow is a Research Scientist at Google Brain Montreal. His main research interests are in the application of machine learning to problems involving structured data, with a specific interest in the intersection of machine learning and programming languages. He is a co-editor of the recent MIT Press book on Perturbations, Optimization, and Statistics (2017). His work has won awards at UAI (Best Student Paper, Runner Up), the ICML Workshop on Constructive Machine Learning (Best Paper), the NIPS Workshop on Neural Abstract Machines and Program Induction (Best Paper), and NIPS (Best Paper). He holds a Ph.D. from the Machine Learning group at the University of Toronto (2013) and previously was a Researcher at Microsoft Research Cambridge UK (2013 - 2017) with a Research Fellowship at Darwin College, University of Cambridge (2013 - 2016). Dima

2 4

Christian Gagné and Jean-François Lalonde from U. of Laval.
by Junyoung Chung 02 Jui '17

02 Jui '17

Hi all, we have two tea-talks (each is a 30-min talk) this week! Both speakers are from Univ. of Laval. You will find more information below. Talk by Christian Gagné Time: 13:45~14:15 Title: Alternating Direction Method of Multipliers for Sparse Convolutional Neural Networks Abstract: The storage and computation requirements of Convolutional Neural Networks (CNNs) can be prohibitive for exploiting these models over low-power or embedded devices. We are proposing to reduce the computational complexity of the CNNs by minimizing an objective function, including the recognition loss that is augmented with a sparsity-promoting penalty term. The sparsity structure of the network is identified using the Alternating Direction Method of Multipliers (ADMM), which is widely used in large optimization problems. This method alternates between promoting the sparsity of the network and optimizing the recognition performance, which allows us to exploit the two-part structure of the corresponding objective functions. In particular, we take advantage of the separability of the sparsity-inducing penalty functions to decompose the minimization problem into sub-problems that can be solved sequentially. Applying our method to a variety of state-of-the-art CNN models, our proposed method is able to simplify the original model, generating models with less computation and fewer parameters, while maintaining and often improving generalization performance. Accomplishments on a variety of models strongly verify that our proposed ADMM-based method can be a very useful tool for simplifying and improving deep CNNs. Short bio: Christian Gagné is professor at the Electrical Engineering and Computer Engineering Department of Université Laval since 2008. He is member of the Computer Vision and Systems Laboratory, the REPARTI FRQNT Strategic Cluster, and the Big Data Research Centre. He completed a Ph.D. in Electrical Engineering (Université Laval) in 2005 before being postdoctoral fellow at the INRIA Saclay - Île-de-France (France) and the University of Lausanne (Switzerland) in 2005-2006. He was research associate for Informatique WGZ Inc. (2006-2007) and MacDonald, Dettwiler, and Associates Ltd. (2007-2008). His research interests are on the engineering of intelligent systems, especially systems based on machine learning, evolutionary computing, and their application to pattern recognition, smart sensors, smart grids, and simulation of public health processes. Talk by Jean-François Lalonde Time: 14:15~14:45 Title: Learning to Predict Illumination from a Single Image Abstract: We propose an automatic method to infer high dynamic range illumination from a single, limited field-of-view, low dynamic range photograph of a scene. In contrast to previous work that relies on specialized image capture, user input, and/or simple scene models, we train an end-to-end deep neural network that directly regresses a limited field-of-view photo to HDR illumination, without strong assumptions on scene geometry, material properties, or lighting. This allows us to automatically recover high-quality HDR illumination estimates that significantly outperform previous state-of-the-art methods. Consequently, using our illumination estimates for applications like 3D object insertion, we can achieve results that are photo-realistic. Bio: Jean-François Lalonde is an Assistant Professor in Electrical and Computer Engineering at Laval University, Quebec City, since 2013. Previously, he was a Post-Doctoral Associate at Disney Research, Pittsburgh. He received a B.Eng. degree in Computer Engineering with honors from Laval University, Canada, in 2004. He earned his M.S at the Robotics Institute at Carnegie Mellon University in 2006 under Prof. Martial Hebert and received his Ph.D., also from Carnegie Mellon, in 2011 under the supervision of Profs. Alexei A. Efros and Srinivasa G. Narasimhan. His thesis, titled "Understanding and Recreating Appearance under Natural Illumination," won the 2010-11 CMU School of Computer Science Distinguished Dissertation Award. After graduation, he became a Computer Vision Scientist at Tandent, Inc., where he helped develop LightBrush™, the first commercial intrinsic imaging application. He also introduced intrinsic videos at SIGGRAPH 2012 while at Tandent. His research focuses on lighting-aware image understanding and synthesis by leveraging large amounts of data. More at www.jflalonde.ca. Best, --Junyoung

1 3