Year: 2026
Type: conference-paper
Abstract: Annual reports communicate corporate performance to stakeholders through dense tables and explanatory text, with rich grounding signals making automated reasoning challenging.Existing QA benchmarks focus on retrieval or single-modality reasoning, rarely require justification for answers with both textual and tabular evidence.We introduce ARQA (Annual Report QA), a benchmark of ~2.5KQA pairs spanning ten fiscal years of automotive enterprise annual reports and three reasoning families-LOOKUP, ARITHMETIC, and INSIGHT.Data are produced via a planner-generator pipeline, deterministically verified and recomputed, and fully reviewed by domain experts.We evaluate state-of-the-art instruction-tuned language models on ARQA, showing strong factual retrieval but persistent weaknesses in grounded arithmetic and causal reasoning.We release ARQA and its evaluation toolkit 1 to facilitate research on auditable, evidence-first reasoning over enterprise documents.
Authors: Ruilong Wang, Simone Balloccu
Institution: Technische Universität Darmstadt
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W7140107318