We have a special post-ICML/UAI seminar on Monday!
On unsupervised deep learning, sparsity, and hierarchical representations
by Andrew Y. Ng
Location: Pavillon André-Aisenstadt (UdeM), room 3195 Time: June 22nd, 13h30
The ability to learn rich feature representations from unlabeled data holds the promise of getting learning methods to usefully learn from a vastly increased amount of data, and therefore perhaps also achieving vastly improved performance. In this talk, I'll describe our recent work on learning sparse representations from unlabeled data, particularly when learning deep representations. In detail, I'll describe a sparse DBN algorithm that learns (from natural images) to detect edges at the lowest layer, and combine edges to form "corners" at the next layer. I'll also describe a comparison of this method to biological data from visual cortical area V2. Further, I show that a larger scale version of this method, developed using the idea of convolutional networks, automatically identifies "object parts", and learns to group such object parts together to form more complete models of objects. For example, when trained on images of faces, the algorithm identifies edges at the lowest level; assembles edges to form parts of faces such as "eyes" and "nose" features at the next level; and finally learns more complete models of faces at the highest level. These methods also give features that give good classification performance on data of many different types of modalities, including images, audio, handwriting, text, and 3d laser scanner data. Finally, I believe that good evaluation metrics are needed to develop good unsupervised deep learning algorithms, and will describe some recent results on developing new metrics.
Joint work with Honglak Lee and others.
Bio:
Andrew Ng is an Assistant Professor of Computer Science at Stanford University. His research interests include machine learning, reinforcement learning/control, and broad-competence AI. His group has won best paper/best student paper awards at ACL, CEAS, 3DRR and ICML. He is also a recipient of the Alfred P. Sloan Fellowship, and the IJCAI 2009 Computers and Thought award.