Work

Cross-Validation Optimization for Large Scale Structured Classification Kernel Methods

Year: 2008
Type: article
Abstract: We propose a highly efficient framework for penalized likelihood kernel methods applied\nto multi-class models with a large, structured set of classes. As opposed to many previous\napproaches which try to decompose the fitting problem into many smaller ones, we focus\non a Newton optimization of the complete model, making use of model structure and\nlinear conjugate gradients in order to approximate Newton search directions. Crucially,\nour learning method is based entirely on matrix-vector multiplication primitives with the\nkernel matrices and their derivatives, allowing straightforward specialization to new kernels,\nand focusing code optimization efforts to these primitives only.\nKernel parameters are learned automatically, by maximizing the cross-validation log\nlikelihood in a gradient-based way, and predictive probabilities are estimated. We demonstrate\nour approach on large scale text classification tasks with hierarchical structure on\nthousands of classes, achieving state-of-the-art results in an order of magnitude less time\nthan previous work.
Language: English
Open Access status: green
OpenAlex ID: W2153838241
FWCI: 1.686
Cites: 37
Cited by: 37
Related to: 8