Work

Enhancing Technical Question Answering Quality Through Multimodal Document Segmentation

Year: 2026

Type: article

Abstract: Technical documents present unique multimodal understanding challenges due to their heterogeneous information elements requiring both localized visual interpretation and global contextual reasoning. We introduce a novel two-tiered augmentation strategy that uniquely bridges this gap by generating semantically enriched document fragments through layout-aware segmentation and multimodal annotation. Unlike conventional RAG approaches that process documents as atomic text units, our methodology preserves visual integrity of technical elements while creating a hierarchical evidence structure that balances global context (via Page-Augmented Retrieval) with precise local evidence (via Localized Fragment Augmentation). This approach addresses the fundamental limitation in document AI: the trade-off between preserving spatial relationships and providing task-relevant context. Our method achieves state-of-the-art results, improving GPT-4’s DesignQA performance by 22.4% and demonstrating strong generalizability on ScienceQA (85.3% MC accuracy) and MMMU (66.7% overall). By enabling accurate interpretation of design constraints and functional requirements directly from technical documentation, our framework advances AI systems for engineering design verification and regulatory compliance. Code will be available at https://github.com/LvovDmitro/mmqa-segmentation.

Source: IEEE Access

DOI: 10.1109/access.2026.3655813

Authors: Dmitry Lvov, Ivan Smirnov, Valeria Volokha, Anastasia Laushkina, Alexander Boukhanovsky

Institution: ITMO University

Language: English

Topic: Multimodal Machine Learning Applications

Subfield: Computer Vision and Pattern Recognition

Field: Computer Science

Domain: Physical Sciences

SDG: Industry, innovation and infrastructure

Open Access status: gold

Estimated APC paid: $2,075