Get started
Work

Do multimodal large language models understand welding?

Year: 2025
Type: article
Abstract: This paper examines the performance of Multimodal LLMs (MLLMs) in skilled production work, with a focus on welding. Using a novel data set of real-world and online weld images, annotated by a domain expert, we evaluate the performance of two state-of-the-art MLLMs in assessing weld acceptability across three contexts: RV & Marine, Aeronautical, and Farming. While both models perform better on online images, likely due to prior exposure or memorization, they also perform relatively well on unseen, real-world weld images. Additionally, we introduce WeldPrompt, a prompting strategy that combines Chain-of-Thought generation with in-context learning to mitigate hallucinations and improve reasoning. WeldPrompt improves model recall in certain contexts but exhibits inconsistent performance across others. These results underscore the limitations and potentials of MLLMs in high-stakes technical domains and highlight the importance of fine-tuning, domain-specific data, and more sophisticated prompting strategies to improve model reliability. The study opens avenues for further research into multimodal learning in industry applications. • We evaluate MLLMs’ performance in assessing weld quality. • We introduce WeldPrompt, a strategy using Chain-of-Thought and in-context learning. • MLLMs perform better on online than real-world weld images, showing limited generalization. • WeldPrompt boosts recall in some contexts but trades off precision in different applications. • MLLM limitations in welding offer insights for future XAI research in manufacturing.
DOI: 10.1016/j.inffus.2025.103121
Authors: Grigorii Khvatskii, Yong Suk Lee, Corey Angst, Maria Gibbs, Robert Landers, Nitesh V. Chawla
Language: English
Open Access status: hybrid
Estimated APC paid: $4,740
OpenAlex ID: W4408733089
FWCI: 3.982
Cites: 38
Cited by: 11
Related to: 10