Work

Penalty hyperparameter optimization with diversity measure for nonnegative low-rank approximation

Year: 2024

Type: article

Abstract: Learning tasks are often based on penalized optimization problems in which a sparse solution is desired. This can lead to more interpretative results by identifying a smaller subset of important features or components and reducing the dimensionality of the data representation, as well. In this study, we propose a new method to solve a constrained Frobenius norm-based nonnegative low-rank approximation, and the tuning of the associated penalty hyperparameter, simultaneously. The penalty term added is a particular diversity measure that is more effective for sparseness purposes than other classical norm-based penalties (i.e., ℓ 1 or ℓ 2 , 1 norms). As it is well known, setting the hyperparameters of an algorithm is not an easy task. Our work drew on developing an optimization method and the corresponding algorithm that simultaneously solves the sparsity-constrained nonnegative approximation problem and optimizes its associated penalty hyperparameters. We test the proposed method by numerical experiments and show its promising results on several synthetic and real datasets.

Source: Applied Numerical Mathematics

DOI: 10.1016/j.apnum.2024.10.002

Authors: Nicoletta Del Buono, Flavia Esposito, Laura Selicato, Rafał Zdunek

Institutions: University of Bari Aldo Moro, Water Research Institute, National Research Council, Wrocław University of Science and Technology

Language: English

Topic: Sparse and Compressive Sensing Techniques

Subfield: Computational Mechanics

Field: Engineering

Domain: Physical Sciences

Open Access status: hybrid

Estimated APC paid: $1,560

Funders: Ministero dell’Istruzione, dell’Università e della Ricerca, Universita degli Studi di Bari Aldo Moro, Gruppo Nazionale per il Calcolo Scientifico