Year: 2026
Type: conference-paper
Abstract: Tabular data is frequently captured in image form across a wide range of real-world scenarios such as financial reports, handwritten records, and document scans.These visual representations pose unique challenges for machine understanding, as they combine both structural and visual complexities.While recent advances in Multimodal Large Language Models (MLLMs) show promising results in table understanding, they typically assume the relevant table is readily available.However, a more practical scenario involves identifying and reasoning over relevant tables from largescale collections to answer user queries.To address this gap, we propose TabRAG, a framework that enables MLLMs to answer queries over large collections of table images.Our approach first retrieves candidate tables using jointly trained visual-text foundation models, then leverages MLLMs to perform fine-grained reranking of these candidates, and finally employs MLLMs to reason over the selected tables for answer generation.Through extensive experiments on a newly constructed dataset comprising 88,161 training and 9,819 testing samples across 8 benchmarks with 48,504 unique tables, we demonstrate that our framework significantly outperforms existing methods by 7.0% in retrieval recall and 6.1% in answer accuracy, offering a practical solution for realworld table understanding tasks.
Institution: University of Wisconsin–Madison
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W7140112893