June 2013 - Lisa_seminaires - mailman.iro.umontreal.ca

Fwd: Rappel : Séminaire RALI-OLST du 26 juin 2013
by Yoshua Bengio 25 Jun '13

25 Jun '13

Hi all, For those who can follow a talk in French with slides in English, and with some interest in NLP, this should be an interesting talk: tomorrow at 11:30 at 3195. -- Yoshua Begin forwarded message: > From: Philippe Langlais <felipe(a)iro.umontreal.ca> > Subject: Fwd: Rappel : Séminaire RALI-OLST du 26 juin 2013 > Date: 25 juin 2013 21:58:25 HAE > To: Yoshua Bengio <bengioy(a)iro.UMontreal.CA> > > Yoshua, > > un séminaire qui pourrait intéresser certains membres de ta gang. > > Méthodes Statistiques en Traitement des Langues - Etat des lieux et perspectives. > François Yvon > Université Paris Sud et LIMSI/CNRS > Dans cet exposé, je vais présenter une rétrospective très générale de vingt années de travaux conduits en Traitement Automatique des Langue, en me focalisant exclusivement sur les méthodes fondées sur l'exploitation statistique de gros corpus annotés. En m'appuyant sur cette synthèse, je discuterai ensuite quelques perspectives pour le domaine du TAL, en discutant en particulier de ses interactions avec le domaine de la linguistique. > > (exposé en français, slides in English) > > f. > > > Début du message réexpédié : > >> De : Webmestre RALI <critter(a)iro.umontreal.ca> >> Date : 25 juin 2013 21:43:27 HAE >> À : rali-olst(a)iro.umontreal.ca >> Objet : Rappel : Séminaire RALI-OLST du 26 juin 2013 >> >> Annonce de séminaire RALI-OLST >> >> Méthodes Statistiques en Traitement des Langues - Etat des lieux et perspectives. >> François Yvon >> Université Paris Sud et LIMSI/CNRS >> >> Le 26 juin 2013 à 11h30 >> Salle 3195, Pavillon André-Aisenstadt >> >> ---- >> >> >> Consulter l'annonce du séminaire sur le Web : >> http://rali.iro.umontreal.ca/rali/?q=fr/node/1222/show/3567 > > RALI / DIRO phone: +1 (514) 343 61 11 ext: 47494 > Université de Montréal perso: +1 (514) 507 5314 > C.P. 6128 fax: +1 (514) 343 5834 > Succursale Centre-ville email: felipe(a)iro.umontreal.ca > Montréal (Québec), Canada, H3C 3J7 >

1 0

Fwd: Rappel : Séminaire RALI-OLST du 26 juin 2013
by Yoshua Bengio 25 Jun '13

25 Jun '13

Hi all, For those who can follow a talk in French with slides in English, and with some interest in NLP, this should be an interesting talk: tomorrow at 11:30 at 3195. -- Yoshua Begin forwarded message: > From: Philippe Langlais <felipe(a)iro.umontreal.ca> > Subject: Fwd: Rappel : Séminaire RALI-OLST du 26 juin 2013 > Date: 25 juin 2013 21:58:25 HAE > To: Yoshua Bengio <bengioy(a)iro.UMontreal.CA> > > Yoshua, > > un séminaire qui pourrait intéresser certains membres de ta gang. > > Méthodes Statistiques en Traitement des Langues - Etat des lieux et perspectives. > François Yvon > Université Paris Sud et LIMSI/CNRS > Dans cet exposé, je vais présenter une rétrospective très générale de vingt années de travaux conduits en Traitement Automatique des Langue, en me focalisant exclusivement sur les méthodes fondées sur l'exploitation statistique de gros corpus annotés. En m'appuyant sur cette synthèse, je discuterai ensuite quelques perspectives pour le domaine du TAL, en discutant en particulier de ses interactions avec le domaine de la linguistique. > > (exposé en français, slides in English) > > f. > > > Début du message réexpédié : > >> De : Webmestre RALI <critter(a)iro.umontreal.ca> >> Date : 25 juin 2013 21:43:27 HAE >> À : rali-olst(a)iro.umontreal.ca >> Objet : Rappel : Séminaire RALI-OLST du 26 juin 2013 >> >> Annonce de séminaire RALI-OLST >> >> Méthodes Statistiques en Traitement des Langues - Etat des lieux et perspectives. >> François Yvon >> Université Paris Sud et LIMSI/CNRS >> >> Le 26 juin 2013 à 11h30 >> Salle 3195, Pavillon André-Aisenstadt >> >> ---- >> >> >> Consulter l'annonce du séminaire sur le Web : >> http://rali.iro.umontreal.ca/rali/?q=fr/node/1222/show/3567 > > RALI / DIRO phone: +1 (514) 343 61 11 ext: 47494 > Université de Montréal perso: +1 (514) 507 5314 > C.P. 6128 fax: +1 (514) 343 5834 > Succursale Centre-ville email: felipe(a)iro.umontreal.ca > Montréal (Québec), Canada, H3C 3J7 >

1 0

Re: [Lisa_seminaires] [Lisa_labo] ICML-practice talks: Thursday, June 13th, AA3195, @ 2:00PM
by Yoshua Bengio 13 Jun '13

13 Jun '13

ICML practice talks starting in 2 minutes at 3195! --Yoshua On 2013-06-12, at 14:27, Yoshua Bengio wrote: > > Actually for Razvan it is a full ICML oral. > > -- Yoshua > > On 2013-06-12, at 13:57, Guillaume Desjardins wrote: > >> Please join us tomorrow afternoon (2:00-3:30PM) for a special round of >> ICML practice talks. Speakers will be Razvan Pascanu (remotely via >> Google Hangouts) and Yoshua Bengio (for both a workshop talk, and a 5 >> min ICML spotlight). Titles and abstract below. >> >> The talk will be held in AA3195 as usual. >> >> >> Speaker: Razvan Pascanu >> Title: On the difficulty of training Recurrent Neural Networks >> >> There are two widely known issues with properly training Recurrent >> Neural Networks, the vanishing and the exploding gradient problems >> detailed in Bengio et al. (1994). In this paper we attempt to improve >> the understanding of the underlying issues by exploring these problems >> from an analytical, a geometric and a dynamical systems perspective. >> Our analysis is used to justify a simple yet effective solution. We >> propose a gradient norm clipping strategy to deal with exploding >> gradients and a soft constraint for the vanishing gradients problem. >> We validate empirically our hypothesis and proposed solutions in the >> experimental section. >> >> >> Speaker: Yoshua Bengio (workshop talk) >> Title: From Latent Anonymous Variables to Deep Stochastic Networks >> >> Graphical models with anonymous latent variables (such as Markov >> Random Fields and variants of Boltzmann machines) are very powerful >> models which have had a great impact in machine learning. We propose >> here to consider a type of models over the joint distribution of >> observed variables that shares many of the properties of these >> anonymous latent variable models, but without the need for potentially >> hurtful approximate inference or approximation of the partition >> function during training (or both, like in Deep Boltzmann Machines). >> The only approximation is function approximation. The proposed deep >> stochastic networks can provably estimate the underlying joint >> distribution of the data (or a conditional, if discriminant learning >> is preferred), in the sense that they are consistent estimators of >> this joint or conditional distribution. However, unlike the usual >> probabilistic models, they only represent the learned distribution >> implicitly, through the convergent distribution of a Markov chain. >> Like them, though, they can handle missing values and structured >> output. Unlike them, they can be trained by straightforward gradient >> descent in a deep (possibly recurrent) network in which noise is >> injected. Most of the impressive experimental progress in deep >> learning in recent years (in particular for speech and object >> recognition) has been with with deep supervised networks, sometimes >> with noise injected (dropout). Interestingly, the same kinds of >> architectures can be used to train the proposed unsupervised (or >> structured output) models. >> >> >> Speaker: Yoshua Bengio (ICML spotlight). >> Better Mixing via Deep Representations >> >> It has been hypothesized, and supported with experimental evidence, >> that deeper representations, when well trained, tend to do a better >> job at disentangling the underlying factors of variation. We study the >> following related conjecture: better representations, in the sense of >> better disentangling, can be exploited to produce Markov chains that >> mix faster between modes. Consequently, mixing between modes would be >> more efficient at higher levels of representation. To better >> understand this, we propose a secondary conjecture: the higher-level >> samples fill more uniformly the space they occupy and the high-density >> manifolds tend to unfold when represented at higher levels. The paper >> discusses these hypotheses and tests them experimentally through >> visualization and measurements of mixing between modes and >> interpolating between samples. >> _______________________________________________ >> Lisa_labo mailing list >> Lisa_labo(a)iro.umontreal.ca >> https://webmail.iro.umontreal.ca/mailman/listinfo/lisa_labo > > _______________________________________________ > Lisa_labo mailing list > Lisa_labo(a)iro.umontreal.ca > https://webmail.iro.umontreal.ca/mailman/listinfo/lisa_labo

2 1

Fwd: Machine Learning/Scientific Programmer Postions at NIH - Announcement
by Yoshua Bengio 10 Jun '13

10 Jun '13

Begin forwarded message: > From: "Briggman, Kevin (NIH/NINDS) [E]" <kevin.briggman(a)nih.gov> > Subject: Machine Learning/Scientific Programmer Postions at NIH - Announcement > Date: 10 juin 2013 02:35:02 HAE > To: "bengioy(a)iro.umontreal.ca" <bengioy(a)iro.umontreal.ca> > > Dear Dr. Bengio > > We are hiring for a couple open ML/programming positions. Would you please include the attached job announcement onhttp://deeplearning.net/ and/or forward to anyone that may be interested? > > Thanks > Kevin Briggman > > _________________________________________________________ > Kevin Briggman, PhD > Circuit Dynamics and Connectivity Unit > NINDS / National Institutes of Health > Bldg 37 / Rm 1016 > 37 Convent Dr > Bethesda, MD 20892-3702 > > phone: 301-402-7155 > email: briggmankl(a)mail.nih.gov >

1 0

Fwd: [Lisa_labo] John Langford presentation
by Yoshua Bengio 06 Jun '13

06 Jun '13

Hi, John Langford's presentation has been taped by Frederic Bastien and is ready for your enjoyment (the video and the slides). > https://www.youtube.com/user/nouiz -- Yoshua Begin forwarded message: > From: Frédéric Bastien <nouiz(a)nouiz.org> > Subject: [Lisa_labo] John Langford presentation > Date: 6 juin 2013 15:28:33 HAE > To: Lisa Labo <lisa_labo(a)iro.umontreal.ca> > Cc: Chris Pal <chris.j.pal(a)gmail.com>, John Langford <jl(a)hunch.net> > > Hi, > > There was problem on Mac with the previous video. Quicktime support just a subset of H264 codec. So I uploaded them on youtube. You can see them here (with links to the slides): > > https://www.youtube.com/user/nouiz > > The text on the projector isn't easy to read. It is like that in the original. You can check the slide to read them correctly. > > Frédéric > _______________________________________________ > Lisa_labo mailing list > Lisa_labo(a)iro.umontreal.ca > https://webmail.iro.umontreal.ca/mailman/listinfo/lisa_labo

1 0

Talk by Anoop Korattikara: Thursday, June 6th at 2:00pm (AA3195)
by Guillaume Desjardins 06 Jun '13

06 Jun '13

Please join us this upcoming Thursday (June 6th) for the following talk given by Anoop Korattikara (student of Max Welling). The talk will be held at 2:00pm in AA3195 as usual. Title: Markov Chain Monte Carlo and the Bias-Variance Tradeoff Bayesian posterior sampling can be painfully slow on very large datasets, since traditional MCMC methods such as Hybrid Monte Carlo are designed to be asymptotically unbiased and require processing the entire dataset to generate each sample. Thus, given a small amount of sampling time, the variance of estimates computed using such methods could be prohibitive. We argue that lower risk estimates can often be obtained using ™approximate™ MCMC methods that mix very fast (and thus lower the variance quickly) at the expense of a small bias in the stationary distribution. I will first talk about two such biased algorithms: Stochastic Gradient Langevin Dynamics and its successor Stochastic Gradient Fisher Scoring, both of which use stochastic gradients estimated from mini-batches of data, allowing them to mix very fast. Then I will present our current work on a new (biased) MCMC algorithm that uses a sequential hypothesis test to approximate the Metropolis-Hastings test, allowing us to accept/reject samples with high confidence using only a fraction of the data required for the exact test.

2 1

Fwd: Some documentation...
by Yoshua Bengio 01 Jun '13

01 Jun '13

Begin forwarded message: > From: Patrice Marcotte <marcotte(a)iro.umontreal.ca> > Subject: Fwd: TR: Some documentation... > Date: 31 mai 2013 15:30:13 HAE > To: employes(a)iro.umontreal.ca > > Veuillez noter un séminaire en apprentissage automatique (machine learning). > > > COLLOQUE DU DIRO > DATE: mercredi 5 juin 2013, 10h00 > > LIEU: salle 3195, pavillon André-Aisenstadt > > CONFÉRENCIER: John LANGFORD, Senior Research Scientist, Microsoft Research > > John Langford studied Physics and Computer Science at the California Institute of Technology, earning a double bachelor's degree in 1997, and received his Ph.D. from Carnegie Mellon University in 2002. Since then, he has worked at Yahoo!, Toyota Technological Institute, and IBM's Watson Research Center. He is also the primary author of the popular Machine Learning weblog, hunch.net and the principle developer of Vowpal Wabbit. Previous research projects include Isomap, Captcha, Learning Reductions, Cover Trees, and Contextual Bandit learning. > TITRE: LEARNING FROM LOTS OF DATA > > ABSTRACT: > In the last decade, sources of data for machine learning have become extremely plentiful, leading to new classes of problems. I will discuss two such problems, whose joint solution is critical to successful learning here: > > (1) How do you create learning algorithms capable of coherently dealing with large quantities of data? > > (2) How do you learn given the partial feedback nature available in most large scale data sources? > > In addition, I'll discuss some open problems which I'd like to solve in these areas. >

1 0