Hi everyone,
Next week, we're lucky to have two invited speakers giving a talk at UdeM, the first on Monday, the second on Friday, both at 15:00.
Descriptions of both talks follow.
----
Title: Cheap and Fast But is it Good? Evaluating Nonexpert Annotations for Machine Learning Tasks Speaker: Rion Snow Date: Monday, November 17th, 15:00 Place: Room 3195 (pav. André-Aisenstadt, UdeM)
Abstract: Human annotation is crucial for many machine learning tasks but can be expensive and time-consuming. We explore the use of Amazon's Mechanical Turk web service, a significantly cheaper and faster method for collecting annotations from a broad base of paid non-expert contributors over the Web. We investigate five task in the field of natural language processing: affect recognition, word similarity, recognizing textual entailment, event temporal ordering, and word sense disambiguation. For all five, we show high agreement between Mechanical Turk non-expert annotations and existing gold standard labels provided by expert labelers. For the task of affect recognition, we also show that using non-expert labels for training machine learning algorithms can be as effective as using gold standard annotations from experts. We propose a technique for bias correction that significantly improves annotation quality on two tasks. We conclude that many large labeling tasks can be effectively designed and carried out in this method at a fraction of the usual expense.
A summary of this work may be found online at: http://blog.doloreslabs.com/2008/09/amt-fast-cheap-good-machine-learning/.
Bio: Rion Snow is a PhD Candidate in Computer Science at Stanford University, advised by Professors Andrew Ng and Dan Jurafsky. Rion works in the intersection of machine learning and natural language processing, with a focus in computational semantics. He leads the Stanford Wordnet Project, which aims at learning large-scale semantic networks automatically from natural text. His work on automatically inferring semantic taxonomies received the Best Paper Award at the 2006 conference for the Association of Computational Linguistics. His publications and contact information may be found at his home page: http://ai.stanford.edu/~rion/.
----
Title: Analysis of EEG data by means of ordinal pattern distributions Speaker: Mathieu Sinn Date: Friday, November 21st, 15:00 Place: Room 3195 (pav. André-Aisenstadt, UdeM)
Abstract: Ordinal time series analysis is a new fast and robust approach to the investigation of long and complex time series, such as electroencephalographic (EEG) data. The basic idea is to consider locally the order relations between the values instead of the values themselves.
An ordinal pattern represents the order relations between a finite number of equidistant values of a time series. Assuming that these values are pairwise different, their unique rank order can be simply described by a permutation. By looking at the distributions of ordinal patterns (or measures derived from the distributions) in different parts of a time series, changes in the dynamic of the underlying system can be detected, such as increasing or decreasing complexity. One derived measure is the permutation entropy, which is the Shannon entropy of ordinal pattern distributions. It has been shown that in certain cases of time series obtained by dynamical systems, the permutation entropy is related to the Kolmogorov-Sinai entropy which is a well-known measure for the complexity of time series and the underlying systems, respectively.
A main application of the permutation entropy so far has been the detection of epileptic activity in EEG data, as it is widely accepted that the complexity of the brain dynamics changes (namely, decreases) during epileptic seizures. However, since clearly different ordinal pattern distributions may yield very similar permutation entropy values, we propose to take the whole distributions into account. First attempts show that a cluster analysis of ordinal pattern distributions may be a good approach to an automatic separation and classification of EEG data. Beneath the detection and localization of epileptic activity, another interesting application of this method is the automatic classification of sleep stages according to the criterions of Rechtschaffen and Kales.
This is joint work with Karsten Keller.
----
Hoping to see you there,