Work

When Does Auxiliary Modality Matter in Solving Geometric Problems? A Comprehensive Study of Textual, Formal, and Visual Modalities

Year: 2026

Type: conference-paper

Abstract: Large Language Models (LLMs) face challenges in integrating linguistic and spatial reasoning, which limits their performance on geometry problems.While prior work has attempted to bridge this gap using diagram parsers with multimodal models, a systematic comparison of how various auxiliary modalities and their combinations affect performance has been lacking.To address this, we present a systematic study of four auxiliary modalities-formal diagram facts (CDL), natural language representations (T CDL ), diagram descriptions (DES), and image augmentations (IMG)-on a range of open-and closed-source multimodal LLMs.Our analysis reveals a compelling dichotomy in the effectiveness of these modalities.While formal representations like CDL and T CDL offer a modest performance lift, diagram descriptions (DES) cause a dramatic split: they significantly boost the accuracy of open-source LLMs which often struggle with visual parsing, while often misleading more capable closed-source models and causing a performance drop.This highlights a critical trade-off between augmenting input with helpful information and introducing misleading noise, demonstrating that the efficacy of auxiliary modalities is heavily dependent on the inherent capabilities of the underlying model.

Source: Conference of the European Chapter of the Association for Computational Linguistics

DOI: 10.18653/v1/2026.eacl-short.4

Authors: Hyuk Namgoong, Jeesu Jung, Yerim Han, Sangkeun Jung

Institution: Chungnam National University

Language: English

Topic: Multimodal Machine Learning Applications

Subfield: Computer Vision and Pattern Recognition

Field: Computer Science

Domain: Physical Sciences

Open Access status: hybrid