Lisa_seminaires Mars 2010

lisa_seminaires@iro.umontreal.ca

2 participants
3 discussions

Fwd: CIS Montreal Seminar Announcement
by Yoshua Bengio 28 Mar '10

28 Mar '10

Begin forwarded message: > From: "Boukadoum, A. Mounir" <boukadoum.mounir(a)uqam.ca> > Date: March 28, 2010 1:25:46 AM EDT (CA) > To: <Mohamed.Cheriet(a)etsmtl.ca>, <skooi(a)videotron.ca>, > <kamwa(a)ireq.ca>, <JEAN.ROUAT(a)IEEE.ORG>, <cysu(a)me.concordia.ca>, > "Proulx, Robert" <proulx.robert(a)uqam.ca>, <grogono(a)cs.concordia.ca>, > <kash(a)ece.concordia.ca>, "Godin, Robert" <godin.robert(a)uqam.ca>, <suen(a)cenparmi.concordia.ca > >, <lowther(a)ece.mcgill.ca>, <gerard.lachiver(a)USherbrooke.ca>, <rejean.plamondon(a)polymtl.ca > >, <yves.gagnon(a)ltrim.com>, <aderd(a)attglobal.net>, <simon.arpin(a)groupeteknika.com > >, <Tony.Wong(a)etsmtl.ca>, <dygao888(a)gmail.com>, <ybelala1(a)yahoo.ca>, > <aderd(a)attglobal.net>, <c.faust(a)sympatico.ca>, <chartier.sylvain(a)courrier.uqam.ca > >, <Denis.Gingras(a)imsi.usherbrooke.ca>, <karimt(a)cim.mcgill.ca>, <farida.cheriet(a)polymtl.ca > >, <meunier(a)iro.umontreal.ca>, <livia(a)livia.etsmtl.ca>, <cenparmi-all(a)cenparmi.concordia.ca > >, <liv4d(a)polymtl.ca>, <bengioy(a)iro.umontreal.ca>, <synchromedia-ets-all(a)synchromedia.ca > >, <javad.sadri(a)mail.mcgill.ca> > Cc: <lamarre.laurent(a)ireq.ca>, <nazih(a)IEEE.ORG>, > <smainm(a)imalog.com>, <weiping(a)ece.concordia.ca>, > <shauheen(a)IEEE.ORG>, "Nkambou, Roger" <nkambou.roger(a)uqam.ca> > Subject: CIS Montreal Seminar Announcement > > IEEE CIS Montreal Chapter > Seminar Announcement > > Title: WordSpace as a technique for text data-mining > By: Dr. Lorne Bouchard > > Date and time: April 1, 2010 at 10:30 > Where: UQAM, President-Kennedy building, # PK-4610 > 201, President-Kennedy Ave (Metro Place-des-Arts) > > This seminar is organized in conjunction with DIC (Doctorat en > Informatique Cognitive) at UQAM. The presentation will be in French > > > Mounir Boukadoum, Ph.D., SMIEEE > CIS Montreal Chapter Chair > > > Abstract > > WordSpace is a data mining technique that can be used to elucidate > meaning in textual information. As with earlier work on information > retrieval, WordSpace is based upon a geometric model of high > dimensionality, the dimensionality of the space being subsequently > reduced in order to extract the sought results. Random indexing, > based upon the work of Kanerva and Plate on the representation of > symbolic data in neuronal nets, avoids the dimension reduction step > by directly constructing a reduced dimensional space. The talk will > illustrate the technique with examples taken from the literature and > will show how to easily deploy it using the many open source > programs that are available. > > Presenter’s Short Biography > > Dr Lorne Bouchard received the B.A. (Mathematics-Physics) magna cum > laude frorm College Sainte-Marie, Montreal, in 1960, the B.Sc.A. > (Engineering Physics) magna cum laude from Ecole Polytechnique, > Montreal, in 1964, the M.Sc. (Computer Science) from University of > Manchester, Manchester (UK), in 1966, and the Ph.D. (Mathematics) > from Boston University, Boston (USA), in 1979. He joined UQAM in > 1970 and is of the pioneers behind its current Computer Science > programs. After holding various academic positions as teacher and > program manager, both at the national and international level, he > “retired” in 2006 only to come back as adjunct professor and > continue his activities with undiminished energy. His current > research interests are in the areas of knowledge engineering and > natural language processing, with application to web information > spotting. > > >

1 0

2 AISTATS 2010 practice talks
by Yoshua Bengio 20 Mar '10

20 Mar '10

Hi, The following two AISTATS 2010 papers will be presented on March 24th, starting at 10h30, in room Z-205 of Pavillon at U. Montreal, by their first authors, respectively Guillaume Desjardins and Xavier Glorot: ------------------------------------------------------------------------------------------------------------------------------------------------- Guillaume Desjardins (with Aaron Courville, Yoshua Bengio, Pascal Vincent, Olivier Delalleau) Parallel Tempering for Training of Restricted Boltzmann Machines Alternating Gibbs sampling between visible and latent units is the most common scheme used for sampling from Restricted Boltzmann Machines (RBM), a crucial component in deep architectures such as Deep Belief Networks. However, we find that it often does a very poor job of rendering the diversity of modes captured by the trained model. We suspect that this property hinders RBM training methods such as the Persistent Contrastive Divergence algorithm that rely on Gibbs sampling to approximate the likelihood gradient. To alleviate this problem, we explore the use of tempered Markov Chain Monte-Carlo for sampling in RBMs. We find both through visualization of samples and measures of likelihood on a toy dataset that it helps both sampling and learning. ------------------------------------------------------------------------------------------------------------------------------------------------- Xavier Glorot (with Yoshua Bengio) Understanding the difficulty of training deep feedforward neural networks Whereas before 2006 it appears that deep multi-layer neural networks were not successfully trained, since then several algorithms have been shown to successfully train them, with experimental results showing the superiority of deeper vs less deep architectures. All these experimental results were obtained with new initialization or training mechanisms. Our objective here is to understand better why standard gradient descent from random initialization is doing so poorly with deep neural networks, to better understand these recent relative successes and help design better algorithms in the future. We first observe the influence of the non-linear activations functions. We find that the logistic sigmoid activation is unsuited for deep networks with random initialization because of its mean value, which can drive especially the top hidden layer into saturation. Surprisingly, we find that saturated units can move out of saturation by themselves, albeit slowly, and explaining the plateaus sometimes seen when training neural networks. We find that a new non-linearity that saturates less can often be beneficial. Finally, we study how activations and gradients vary across layers and during training, with the idea that training may be more difficult when the singular values of the Jacobian associated with each layer are far from 1. Based on these considerations, we propose a new initialization scheme that brings substantially faster convergence. ------------------------------------------------------------------------------------------------------------------------------------------------- -- Yoshua

1 0

UdeM-McGill-MITACS machine learning seminar Thu Mar. 4@14h30, AA-3195
by Dumitru Erhan 03 Mar '10

03 Mar '10

Next week's seminar (see http://www.iro.umontreal.ca/article.php3?id_article=107&lang=en): *Evaluating and Learning Invariant Features* by Ian Goodfellow Stanford University Location: *Pavillon André-Aisenstadt (UdeM), room AA-3195* Time: Thursday, March 4, 14:30. *Abstract*: One approach to unsupervised learning is to attempt to discover a function that decomposes the input into variables representing distinct factors of variation. A weaker version of this is to learn a representation where each variable is invariant to a small number of confounding aspects of the input. Experimental results show that the current degree of invariance achieved by some deep learning algorithms is small compared to that achieved by systems hand-designed to be invariant to specific transformations. I discuss some ideas for future invariance evaluation experiments and for methods of automatically learning to separate factors of variation. -- http://dumitru.ca, +1-514-432-8435

1 1

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

Lisa_seminaires Mars 2010