Year: 2025
Type: conference-paper
Abstract: Recent advances in large language models (LLMs) and vision-language models (LVLMs) have shown promise across many tasks, yet their scientific reasoning capabilities remain untested, particularly in multimodal settings.We present MMSciBench, a benchmark for evaluating mathematical and physical reasoning through text-only and text-image formats, with human-annotated difficulty levels, solutions with detailed explanations, and taxonomic mappings.Evaluation of state-of-the-art models reveals significant limitations, with even the best model achieving only 63.77% accuracy and particularly struggling with visual reasoning tasks.Our analysis exposes critical gaps in complex reasoning and visual-textual integration, establishing MMSciBench as a rigorous standard for measuring progress in multimodal scientific understanding.The code for MM-SciBench is open-sourced at GitHub 1 , and the dataset is available at Hugging Face 2 .
Authors: Xinwu Ye, Chengfan Li, Siming Chen, Wei Wei, Robert Tang
Institutions: Yale University, Brown University, Fudan University
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: gold
OpenAlex ID: W4412888117