October 2011 - Lisa_seminaires - mailman.iro.umontreal.ca

Re: [Lisa_seminaires] UdeM-McGill-mPrime machine learning seminar Wed. Oct. 26th @ 14h00, AA3195
by Guillaume Desjardins 26 Oct '11

26 Oct '11

A reminder for today's talk by Mikolov Tomáš. I had mistakenly advertised this as a mPrime talk, but it is in fact a tea-talk. It is still open to everyone however. By request, the talk will again be broadcasted using Google+ Hangouts. Make sure to add the user lisa.umontreal(a)gmail.com to your circles to get access to the talk. This will again be a trial run, so expect a few technical hiccups. See you there ! On Fri, Oct 21, 2011 at 4:25 PM, Guillaume Desjardins <guillaume.desjardins(a)gmail.com> wrote: > In a back-to-back special, a second UdeM-McGill-mPrime machine learning > seminar will also be held on Wednesday, Oct. 26th. The talk given by Mikolov > Tomáš, will > take place from 14h00-15h00. The talk will be held at the Université > de Montréal, room number to be confirmed shortly. Hope to see you there ! > > Title: Language modeling with recurrent neural networks > Abstract: > Statistical language models are important part of almost any speech > recognition system today. The most basic but also the most successful models > so far are based on n-gram statistics. Comparison of performance of > different language modeling techniques on different tasks will be presented. > Among all, neural network based language models perform the best. Next, > useful extensions of the basic neural network model as well as different > architectures will be discussed, such as recurrent neural network > architecture, classes in the output layer and joint training with a maximum > entropy model. > Next, I will present results achieved with a novel RNNME model (recurrent > neural network trained together with a maximum entropy model), on a > state-of-the-art setup from IBM for Broadcast News speech recognition (NIST > RT04). Word error rate reductions over large 4-gram model are over 10%. > Previously the best language model on this setup, a so-called "model M" from > IBM (regularized class-based maximum entropy model), provides about 5% > reduction of WER over 4-gram model. > Finally, I will talk about character-level and subword-level language > modeling experiments with different models, including a recently proposed > RNN model trained with a new Hessian-Free optimizer. These models can assign > meaningful probability to any words, and can be considered as a solution to > well known problems with infinite vocabularies (OOV problems). Moreover, > their size is significantly lower than of standard models. > This talk presents joint work with Anoop Deoras, Ilya Sutskever, Stefan > Kombrink and Hai Son Le. >

1 0

Re: [Lisa_seminaires] UdeM-McGill-mPrime machine learning seminar Tues. Oct. 25th @ 14h00, AA3195
by Guillaume Desjardins 25 Oct '11

25 Oct '11

A reminder for today's mPrime talk. The talk will be held in AA3195 as usual. We will also be testing an experimental feature today: live broadcast of the talk via Google Hangouts ! If you are unable to attend, you can thus simply join our hangout by adding the user "lisa.umontreal(a)gmail.com" to your circles. Hangout should start around 13h50. See you there ! On Fri, Oct 21, 2011 at 4:18 PM, Guillaume Desjardins <guillaume.desjardins(a)gmail.com> wrote: > A UdeM-McGill-mPrime machine learning seminar will be held this > Tuesday, Oct. 25th. The talk given by Richard Socher, will > take place from 14h00-15h00. The talk will be held at the Université > de Montréal, room number to be confirmed shortly. Hope to see you there ! > > Title: Recursive Deep Learning in Natural Language Processing and Computer > Vision > Abstract: > Hierarchical and recursive structure is commonly found in different > modalities, including natural language sentences and scene images. I > will present some of our recent work on three recursive neural network > architectures that learn meaning representations for such hierarchical > structure. These models obtain state-of-the-art performance on several > language and vision tasks. > The meaning of phrases and sentences is determined by the meanings of > its words and the rules of compositionality. We introduce a recursive > neural network (RNN) for syntactic parsing which can learn vector > representations that capture both syntactic and semantic information > of phrases and sentences. For instance, the phrases "declined to > comment" and "would not disclose" have similar representations. > Since our RNN does not depend on specific assumptions for language, it > can also be used to find hierarchical structure in complex scene > images. This algorithm obtains state-of-the-art performance for > semantic scene segmentation on the Stanford Background and the MSRC > datasets and outperforms Gist descriptors for scene classification by 4%. > The ability to identify sentiments about personal experiences, > products, movies etc. is crucial to understand user generated content > in social networks, blogs or product reviews. The second architecture > I will talk about is based on recursive autoencoders (RAE). RAEs learn > vector representations for phrases sufficiently well as to outperform > other traditional supervised sentiment classification methods on > several standard datasets. > We also show that without supervision RAEs can learn features which > outperform previous approaches for paraphrase detection on the > Microsoft Research Paraphrase corpus. > This talk presents joint work with Andrew Ng and Chris Manning.

1 0

UdeM-McGill-mPrime machine learning seminar Wed. Oct. 26th @ 14h00, location TBD
by Guillaume Desjardins 21 Oct '11

21 Oct '11

In a back-to-back special, a second UdeM-McGill-mPrime machine learning seminar will also be held on Wednesday, Oct. 26th. The talk given by Mikolov Tomáš, will take place from 14h00-15h00. The talk will be held at the Université de Montréal, room number to be confirmed shortly. Hope to see you there ! Title: Language modeling with recurrent neural networks Abstract: Statistical language models are important part of almost any speech recognition system today. The most basic but also the most successful models so far are based on n-gram statistics. Comparison of performance of different language modeling techniques on different tasks will be presented. Among all, neural network based language models perform the best. Next, useful extensions of the basic neural network model as well as different architectures will be discussed, such as recurrent neural network architecture, classes in the output layer and joint training with a maximum entropy model. Next, I will present results achieved with a novel RNNME model (recurrent neural network trained together with a maximum entropy model), on a state-of-the-art setup from IBM for Broadcast News speech recognition (NIST RT04). Word error rate reductions over large 4-gram model are over 10%. Previously the best language model on this setup, a so-called "model M" from IBM (regularized class-based maximum entropy model), provides about 5% reduction of WER over 4-gram model. Finally, I will talk about character-level and subword-level language modeling experiments with different models, including a recently proposed RNN model trained with a new Hessian-Free optimizer. These models can assign meaningful probability to any words, and can be considered as a solution to well known problems with infinite vocabularies (OOV problems). Moreover, their size is significantly lower than of standard models. This talk presents joint work with Anoop Deoras, Ilya Sutskever, Stefan Kombrink and Hai Son Le.

1 0

UdeM-McGill-mPrime machine learning seminar Tues. Oct. 25th @ 14h00, location TBD
by Guillaume Desjardins 21 Oct '11

21 Oct '11

A UdeM-McGill-mPrime machine learning seminar will be held this Tuesday, Oct. 25th. The talk given by Richard Socher, will take place from 14h00-15h00. The talk will be held at the Université de Montréal, room number to be confirmed shortly. Hope to see you there ! Title: Recursive Deep Learning in Natural Language Processing and Computer Vision Abstract: Hierarchical and recursive structure is commonly found in different modalities, including natural language sentences and scene images. I will present some of our recent work on three recursive neural network architectures that learn meaning representations for such hierarchical structure. These models obtain state-of-the-art performance on several language and vision tasks. The meaning of phrases and sentences is determined by the meanings of its words and the rules of compositionality. We introduce a recursive neural network (RNN) for syntactic parsing which can learn vector representations that capture both syntactic and semantic information of phrases and sentences. For instance, the phrases "declined to comment" and "would not disclose" have similar representations. Since our RNN does not depend on specific assumptions for language, it can also be used to find hierarchical structure in complex scene images. This algorithm obtains state-of-the-art performance for semantic scene segmentation on the Stanford Background and the MSRC datasets and outperforms Gist descriptors for scene classification by 4%. The ability to identify sentiments about personal experiences, products, movies etc. is crucial to understand user generated content in social networks, blogs or product reviews. The second architecture I will talk about is based on recursive autoencoders (RAE). RAEs learn vector representations for phrases sufficiently well as to outperform other traditional supervised sentiment classification methods on several standard datasets. We also show that without supervision RAEs can learn features which outperform previous approaches for paraphrase detection on the Microsoft Research Paraphrase corpus. This talk presents joint work with Andrew Ng and Chris Manning.

1 0

Reminder: UdeM-McGill-mPrime machine learning seminar tomorrow, Wed. Oct. 12th @ 14h00, room AA3195
by Guillaume Desjardins 11 Oct '11

11 Oct '11

A reminder for tomorrow's mPrime talk by Jean-Francois Paiement. The talk will be held in AA3195 as usual. See you there ! ---------- Forwarded message ---------- From: Guillaume Desjardins <guillaume.desjardins(a)gmail.com> Date: Mon, Oct 10, 2011 at 10:39 PM Subject: UdeM-McGill-mPrime machine learning seminar Wed. Oct. 12th @ 14h00, location AA3195 To: lisa_seminaires(a)iro.umontreal.ca A UdeM-McGill-mPrime machine learning seminar will be held this Wednesday, Oct. 12th. The talk given by Jean-Francois Paiement, will take place from 14h00-15h00. The talk will be held at the Université de Montréal, but the precise room location will be determined shortly and communicated via email. Hope to see you there ! Title: Learning from Heterogeneous Sources via Gradient Boosting Consensus Abstract: Multiple data sources containing different types of features may be available for a given task. For instance, users' profiles can be used to build recommendation systems. In addition, a model can also use users' historical behaviors and social networks to infer users' interests on related products. We argue that it is desirable to collectively use any available multiple heterogeneous data sources in order to build efficient learning models. We call this framework "heterogeneous learning". In our proposed setting, data sources can include (i) non-overlapping features, (ii) non-overlapping instances, and (iii) multiple networks (i.e. graphs) connecting instances. In this paper, we propose a general optimization framework for heterogeneous learning, and devise a corresponding learning model from gradient boosting. The idea is to minimize the empirical loss with two constraints: (1) There should be consensus among the predictions of overlapping instances (if any) from different data sources; (2) Connected instances in graph datasets may have similar predictions. The objective function is solved by stochastic gradient boosting trees. Furthermore, a weighting strategy is designed to emphasize informative data sources, and deemphasize the noisy ones. We formally prove that the proposed strategy leads to a tighter error bound. This approach consistently outperforms a standard concatenation of data sources on movie rating prediction, number recognition and terrorist attack detection tasks. We observe that the proposed model can improve out-of-sample error rate by as much as 80%. This is joint work with Xiaoxiao Shi (University of Illinois at Chicago), David Grangier (AT&T Labs), and Philip S. Yu (University of Illinois at Chicago).

1 0

UdeM-McGill-mPrime machine learning seminar Wed. Oct. 12th @ 14h00, location TBD
by Guillaume Desjardins 10 Oct '11

10 Oct '11

A UdeM-McGill-mPrime machine learning seminar will be held this Wednesday, Oct. 12th. The talk given by Jean-Francois Paiement, will take place from 14h00-15h00. The talk will be held at the Université de Montréal, but the precise room location will be determined shortly and communicated via email. Hope to see you there ! Title: Learning from Heterogeneous Sources via Gradient Boosting Consensus Abstract: Multiple data sources containing different types of features may be available for a given task. For instance, users' profiles can be used to build recommendation systems. In addition, a model can also use users' historical behaviors and social networks to infer users' interests on related products. We argue that it is desirable to collectively use any available multiple heterogeneous data sources in order to build efficient learning models. We call this framework "heterogeneous learning". In our proposed setting, data sources can include (i) non-overlapping features, (ii) non-overlapping instances, and (iii) multiple networks (i.e. graphs) connecting instances. In this paper, we propose a general optimization framework for heterogeneous learning, and devise a corresponding learning model from gradient boosting. The idea is to minimize the empirical loss with two constraints: (1) There should be consensus among the predictions of overlapping instances (if any) from different data sources; (2) Connected instances in graph datasets may have similar predictions. The objective function is solved by stochastic gradient boosting trees. Furthermore, a weighting strategy is designed to emphasize informative data sources, and deemphasize the noisy ones. We formally prove that the proposed strategy leads to a tighter error bound. This approach consistently outperforms a standard concatenation of data sources on movie rating prediction, number recognition and terrorist attack detection tasks. We observe that the proposed model can improve out-of-sample error rate by as much as 80%. This is joint work with Xiaoxiao Shi (University of Illinois at Chicago), David Grangier (AT&T Labs), and Philip S. Yu (University of Illinois at Chicago).

1 0