Pavel P. Kuksa's Publications

Sorted by DateClassified by Publication TypeDefault Ordering

Deep Learning via Semi-Supervised Embedding

Jason Weston, Ronan Collobert, Frederic Ratle, Hossein Mobahi, Pavel Kuksa, and Koray Kavukcuoglu. Deep Learning via Semi-Supervised Embedding. In ICML 2009 Workshop on Learning Feature Hierarchies, 2009.

Download

[URL] 

Abstract

We show how nonlinear embedding algorithms popular for use with shallow semi-supervised learning techniques such as kernel methods can be applied to deep multi-layer architectures, either as a regularizer at the output layer, or on each layer of the architecture. This provides a simple alternative to existing approaches to deep learning whilst yielding competitive error rates compared to those methods, and existing shallow semi-supervised techniques. We then go on to generalize this approach to take advantage of sequential data: for images, and text. For images, we take advantage of the temporal coherence that naturally exists in unlabeled video recordings. That is, two successive frames are likely to contain the same object or objects. We demonstrate the effectiveness of this method in a semi-supervised setting on some pose invariant object and face recognition tasks. For text, we describe a unified approach to tagging: a single convolutional neural network architecture that, given a sentence, outputs a host of language processing predictions: part-of-speech tags, chunks, named entity tags, and semantic roles. State-of-the-art performance is attained by learning word embeddings using a text specific semi-supervised task called a language model.

BibTeX

@inproceedings{icml2009workshopjason,
	Abstract = {We show how nonlinear embedding algorithms popular for use with shallow
	semi-supervised learning techniques such as kernel methods can be
	applied to deep multi-layer architectures, either as a regularizer
	at the output layer, or on each layer of the architecture. This provides
	a simple alternative to existing approaches to deep learning whilst
	yielding competitive error rates compared to those methods, and existing
	shallow semi-supervised techniques.
	We then go on to generalize this approach to take advantage of sequential
	data: for images, and text.
	For images, we take advantage of the temporal coherence that naturally
	exists in unlabeled video recordings. That is, two successive frames
	are likely to contain the same object or objects. We demonstrate
	the effectiveness of this method in a semi-supervised setting on
	some pose invariant object and face recognition tasks.
	For text, we describe a unified approach to tagging: a single convolutional
	neural network architecture that, given a sentence, outputs a host
	of language processing predictions: part-of-speech tags, chunks,
	named entity tags, and semantic roles. State-of-the-art performance
	is attained by learning word embeddings using a text specific semi-supervised
	task called a language model.},
	Author = {Jason Weston and Ronan Collobert and Frederic Ratle and Hossein Mobahi and Pavel Kuksa and Koray Kavukcuoglu},
	Bib2Html_Pubtype = {Workshop},
	Booktitle = {ICML 2009 Workshop on Learning Feature Hierarchies},
	Owner = {pkuksa},
	Timestamp = {2009.12.30},
	Title = {Deep Learning via Semi-Supervised Embedding},
	Url = {http://www.cs.toronto.edu/~rsalakhu/deeplearning/jason_icml2009.pdf},
	Year = {2009},
	Bdsk-Url-1 = {http://www.cs.toronto.edu/~rsalakhu/deeplearning/jason_icml2009.pdf}}

Generated by bib2html.pl (written by Patrick Riley ) on Tue Mar 17, 2020 17:09:56