• Sorted by Date • Classified by Publication Type • Default Ordering •
Jason Weston, Ronan Collobert, Frederic Ratle, Hossein Mobahi, Pavel Kuksa, and Koray Kavukcuoglu. Deep Learning via Semi-Supervised Embedding. In ICML 2009 Workshop on Learning Feature Hierarchies, 2009.
We show how nonlinear embedding algorithms popular for use with shallow semi-supervised learning techniques such as kernel methods can be applied to deep multi-layer architectures, either as a regularizer at the output layer, or on each layer of the architecture. This provides a simple alternative to existing approaches to deep learning whilst yielding competitive error rates compared to those methods, and existing shallow semi-supervised techniques. We then go on to generalize this approach to take advantage of sequential data: for images, and text. For images, we take advantage of the temporal coherence that naturally exists in unlabeled video recordings. That is, two successive frames are likely to contain the same object or objects. We demonstrate the effectiveness of this method in a semi-supervised setting on some pose invariant object and face recognition tasks. For text, we describe a unified approach to tagging: a single convolutional neural network architecture that, given a sentence, outputs a host of language processing predictions: part-of-speech tags, chunks, named entity tags, and semantic roles. State-of-the-art performance is attained by learning word embeddings using a text specific semi-supervised task called a language model.
@inproceedings{icml2009workshopjason, Abstract = {We show how nonlinear embedding algorithms popular for use with shallow semi-supervised learning techniques such as kernel methods can be applied to deep multi-layer architectures, either as a regularizer at the output layer, or on each layer of the architecture. This provides a simple alternative to existing approaches to deep learning whilst yielding competitive error rates compared to those methods, and existing shallow semi-supervised techniques. We then go on to generalize this approach to take advantage of sequential data: for images, and text. For images, we take advantage of the temporal coherence that naturally exists in unlabeled video recordings. That is, two successive frames are likely to contain the same object or objects. We demonstrate the effectiveness of this method in a semi-supervised setting on some pose invariant object and face recognition tasks. For text, we describe a unified approach to tagging: a single convolutional neural network architecture that, given a sentence, outputs a host of language processing predictions: part-of-speech tags, chunks, named entity tags, and semantic roles. State-of-the-art performance is attained by learning word embeddings using a text specific semi-supervised task called a language model.}, Author = {Jason Weston and Ronan Collobert and Frederic Ratle and Hossein Mobahi and Pavel Kuksa and Koray Kavukcuoglu}, Bib2Html_Pubtype = {Workshop}, Booktitle = {ICML 2009 Workshop on Learning Feature Hierarchies}, Owner = {pkuksa}, Timestamp = {2009.12.30}, Title = {Deep Learning via Semi-Supervised Embedding}, Url = {http://www.cs.toronto.edu/~rsalakhu/deeplearning/jason_icml2009.pdf}, Year = {2009}, Bdsk-Url-1 = {http://www.cs.toronto.edu/~rsalakhu/deeplearning/jason_icml2009.pdf}}
Generated by bib2html.pl (written by Patrick Riley ) on Tue Mar 17, 2020 17:09:56