We have a special post-ICML/UAI seminar on Monday!
On unsupervised deep learning, sparsity, and hierarchical representations
by Andrew Y. Ng
Location: Pavillon André-Aisenstadt (UdeM), room 3195
Time: June 22nd, 13h30
The ability to learn rich feature representations from unlabeled data
holds the promise of getting learning methods to usefully learn from a
vastly increased amount of data, and therefore perhaps also achieving
vastly improved performance. In this talk, I'll describe our recent
work on learning sparse representations from unlabeled data,
particularly when learning deep representations. In detail, I'll
describe a sparse DBN algorithm that learns (from natural images) to
detect edges at the lowest layer, and combine edges to form "corners"
at the next layer. I'll also describe a comparison of this method to
biological data from visual cortical area V2. Further, I show that a
larger scale version of this method, developed using the idea of
convolutional networks, automatically identifies "object parts", and
learns to group such object parts together to form more complete
models of objects. For example, when trained on images of faces, the
algorithm identifies edges at the lowest level; assembles edges to
form parts of faces such as "eyes" and "nose" features at the next
level; and finally learns more complete models of faces at the highest
level. These methods also give features that give good classification
performance on data of many different types of modalities, including
images, audio, handwriting, text, and 3d laser scanner data. Finally,
I believe that good evaluation metrics are needed to develop good
unsupervised deep learning algorithms, and will describe some recent
results on developing new metrics.
Joint work with Honglak Lee and others.
Bio:
Andrew Ng is an Assistant Professor of Computer Science at Stanford
University. His research interests include machine learning,
reinforcement learning/control, and broad-competence AI. His group has
won best paper/best student paper awards at ACL, CEAS, 3DRR and ICML. He
is also a recipient of the Alfred P. Sloan Fellowship, and the IJCAI 2009
Computers and Thought award.