Year: 2026
Type: conference-paper
Abstract: Charts are central to analytical reasoning, yet existing benchmarks for chart understanding focus almost exclusively on single-chart interpretation rather than comparative reasoning across multiple charts.To address this gap, we introduce ChartDiff, the first large-scale benchmark for cross-chart comparative summarization.ChartDiff consists of 8,541 chart pairs spanning diverse data sources, chart types, and visual styles, each annotated with LLM-generated and human-verified summaries describing differences in trends, fluctuations, and anomalies.Using ChartDiff, we evaluate general-purpose, chart-specialized, and pipeline-based models.Our results show that frontier general-purpose models achieve the highest GPT-based quality, while specialized and pipeline-based methods obtain higher ROUGE scores but lower human-aligned evaluation, revealing a clear mismatch between lexical overlap and actual summary quality.We further find that multiseries charts remain challenging across model families, whereas strong end-to-end models are relatively robust to differences in plotting libraries.Overall, our findings demonstrate that comparative chart reasoning remains a significant challenge for current vision-language models and position ChartDiff as a new benchmark for advancing research on multi-chart understanding.Comparison Summary: From 2000 to 2009, Maldives maintained a high and fairly stable tariff around 20-21%, while Venezuela showed a clear downward trend from 13.37% to 9.09% with only minor fluctuations.In 2010-2019, Maldives changed much more dramatically: after staying near 20% in 2010-2011, its tariff collapsed to 6.45% in 2012, then gradually recovered through the decade before rebounding sharply to 18.39% in 2019.Venezuela, by contrast, remained relatively steady in the later period, mostly fluctuating within a narrower 8.4-11.0%range after its earlier decline.Overall, the key shift is that Maldives moved from stability to major volatility and structural change, whereas Venezuela moved from a steady decline to a comparatively stable low-tariff pattern.
Author: Rongtian Ye
Institution: Aalto University
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
Funder: China Scholarship Council
OpenAlex ID: W7166802648