Work

Effective auto encoder for unsupervised sparse representation

Year: 2015
Type: dissertation
Abstract: High dimensionality and the sheer size of unlabeled data available today demand new development in unsupervised learning of sparse representation. Despite of recent advances in representation learning, most of the current methods are limited when dealing with large scale unlabeled data. In this study, we propose a new unsupervised method that is able to learn sparse representation from unlabeled data efficiently. We derive a closed-form solution based on the sequential minimal optimization (SMO) for training an auto encoder-decoder module, which efficiently extracts sparse and compact features from any data set with various size. The inference process in the proposed learning algorithm does not require any expensive Hessian computation for solving the underlying optimization problems. Decomposition of the non-convex optimization problem in our model enables us to solve each sub-problems analytically. Using several image datasets including CIFAR-10, CALTECH-101 and AR face database, we demonstrate the effectiveness in terms of computation time and classification accuracy. Proposed method discovers dictionaries that are able to capture low level features in larger dimensional patches in quite lower executional time than the other alternatives. Then by detailed experimental results, we present that our module outperforms various similar single layer state-of-the-art methods including Sparse Filtering and K-Means clustering method.
Author: Faria Mahnaz
Language: English
Open Access status: green
OpenAlex ID: W1904421428
Cites: 43
Cited by:
Related to: 10