Work

TableLoRA: Low-rank Adaptation on Table Structure Understanding for Large Language Models

Year: 2025

Type: conference-paper

Abstract: Tabular data are crucial in many fields and their understanding by large language models (LLMs) under high parameter efficiency paradigm is important.However, directly applying parameter-efficient fine-tuning (PEFT) techniques to tabular tasks presents significant challenges, particularly in terms of better table serialization and the representation of twodimensional structured information within a one-dimensional sequence.To address this, we propose TableLoRA, a module designed to improve LLMs' understanding of table structure during PEFT.It incorporates special tokens for serializing tables with special token encoder and uses 2D LoRA to encode low-rank information on cell positions.Experiments on four tabular-related datasets demonstrate that TableLoRA consistently outperforms vanilla LoRA and surpasses various table encoding methods tested in control experiments.These findings reveal that TableLoRA, as a tablespecific LoRA, enhances the ability of LLMs to process tabular data effectively, especially in low-parameter settings, demonstrating its potential as a robust solution for handling tablerelated tasks.Vanilla LLama : 15.2 Llama + LoRA: 15.2 Llama + TableLoRA : 15

Source: Proceedings of the conference - Association for Computational Linguistics. Meeting

DOI: 10.18653/v1/2025.acl-long.1090