Work

Embeddings for Numerical Features Using tanh Activation

Year: 2025

Type: conference-paper

Abstract: Recent advances in tabular deep learning have demonstrated the importance of embeddings for numerical features, where scalar values are mapped to high-dimensional spaces before being processed by the main model.Here, we propose an embedding method using the hyperbolic tangent (tanh) activation function that allows neural networks to achieve better accuracy on tabular data via an inductive bias similar to that of decision trees.To make training with the new embedding method reliable and efficient, we additionally propose a principled initialization method.Experiments demonstrate that the new approach improves upon or matches accuracy results from previously proposed embedding methods across multiple tabular datasets and model architectures.

DOI: 10.18653/v1/2025.trl-1.18

Authors: Bingyan Liu, Charles Elkan, Anil N. Hirani

Institutions: University of Illinois Urbana-Champaign, University of California San Diego

Language: English

Topic: Neural Networks and Applications

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: gold

Funders: National Science Foundation, National Centre for Supercomputing Applications, University of Illinois at Urbana-Champaign, Office of Advanced Cyberinfrastructure