Work

TongueBench: performance of prompt-engineered large multimodal models in Traditional Chinese Medicine tongue diagnosis

Year: 2026

Type: article

Abstract: Introduction: Traditional Chinese Medicine (TCM) tongue diagnosis requires reconstruction of multiple visual signs across tongue morphology, body color, coating texture, and coating color. We developed TongueBench as a prompt-controlled, ontology-constrained evaluation protocol for application programming interface (API)-accessed large multimodal models (LMMs), without task-specific training. Methods: Nine LMMs were evaluated on 2,694 tongue images with 22 expert-adjudicated labels under direct prompting (DP), knowledge-guided prompting (KGP), and knowledge-guided reflective prompting (KGRP). For the primary LMM-versus-supervised comparison, archived LMM predictions were matched to the same 270 test images used by three supervised baselines. Sensitivity analyses assessed a fixed label order, training-only frequencies, and no conflict removal. Raw-output auditing distinguished model-generated ontology violations from their removal by deterministic post-processing. Results: Across the nine evaluated models, KGRP increased mean Micro-F1 from 47.57% to 48.51%, Macro-F1 from 32.41% to 33.60%, and sample-level Jaccard from 34.45% to 35.32%, while reducing Hamming loss from 26.39% to 25.54%. Model-level changes were heterogeneous. Image-level bootstrap intervals described the fixed nine-model mean contrast under an unverified image-independence assumption. Supervised models had better observed aggregate scores on the matched split, but patient-level separation could not be established. Sensitivity analyses retained the direction of the mean KGRP-versus-DP changes. Discussion: TongueBench characterizes modest prompt-associated changes, parsing failures, repeated-output variation, and incomplete label-set reconstruction. These observations do not establish general prompt superiority, patient-level generalizability, or clinical utility.

Source: Frontiers in Artificial Intelligence

DOI: 10.3389/frai.2026.1943004