Year: 2025
Type: conference-paper
Abstract: Understanding structured table data with language models is crucial for various downstream tasks in information retrieval.However, transformer-based table embedding models struggle to consistently represent headers across varying entity contexts.This inconsistency undermines the generalizability of embeddings across in-domain tables that share universal semantics.To address this gap, we propose a novel pretraining method for in-domain tables, HAETAE, that explicitly separates header embeddings from contextual entity embeddings.Our method introduces a dedicated header encoder and learnable alignment mechanisms, built upon header-aware serialization.Experimental results demonstrate that HAETAE enhances generalization and stability in predicting headers and values of in-domain tables, achieving higher accuracy than baselines while showing the notable potential of knowledge transfer in cross-domain tables.The source code of HAETAE is available at https://github.com/woojoonjung/HAETAE.
Authors: Woojun Jung, Susik Yoon
Institution: Korea University
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
Award: G7542792045
OpenAlex ID: W4412377883