Work

Learning Kolmogorov-Arnold Neural Activation Functions by Infinite-Dimensional Optimization

Year: 2025

Type: conference-paper

Abstract: Inspired by the Kolmogorov-Arnold Representation Theorem, Kolmogorov-Arnold Networks (KAN) have recently reshaped the landscape of functional representation in the context of training univariate activation functions, thus achieving remarkable performance gains over traditional Multi-Layer Perceptron (MLPs). However, the parametrization of the activation functions in KANs hinders their scalability and usage in machine learning applications. In this article, we propose a novel infinite-dimensional optimization framework to learn the activation functions, enabling the achievement of boundedness, interpretability, and, seamless compatibility with training by backpropagation are all preserved for the network, therefore, reducing the number of required trainable parameters. Through functional representation examples, our results reveal superior accuracy in tasks such as functional representation.

Source: VBN Forskningsportal (Aalborg Universitet)

Authors: Khalyavin, Leon, Moreschini, Alessio, Parisini, Thomas; id_orcid 0000-0001-5396-9665

Institution: Imperial College London

Language: English

Topic: Neural Networks and Applications

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: green

Funder: European Commission

Awards: KIOS Research and Innovation Centre of Excellence, G434337789

OpenAlex ID: W7112026893