Work

Evaluation Boundaries in Multimodal Skin Lesion Classification: Data Leakage, Patient Reuse, and Cross-Release Overlap

Year: 2026
Type: article
Abstract: Multimodal skin-lesion classifiers combine dermoscopic images with clinical metadata, but reported gains may depend on how related images, lesions, patients, and dataset releases are separated between model development and testing. This study evaluates how these boundaries affect classification performance and the measured benefit of metadata fusion. HAM10000, PAD-UFES-20, and International Skin Imaging Collaboration (ISIC) releases are assessed using image-level, lesion-level, patient-level, content-based, and cross-release protocols. Image-only, metadata-only, late-fusion, metadata-gated, and cross-attention models are compared under matched settings with repeated seeds and confidence intervals. In HAM10000, with a fine-tuned ConvNeXt-Small encoder, random image-level splitting increased the macro-averaged F1 score (macro-F1) by 9.48 percentage points relative to lesion-disjoint evaluation, whereas cross-attention changed macro-F1 by only −0.34, −0.86, and +1.11 points across the tested protocols. On PAD-UFES-20, 32.22–40.46% of test patients under lesion-disjoint evaluation were also represented in training through different lesions, and cross-attention macro-F1 was 0.8085 under lesion-disjoint and 0.7124 under patient-disjoint evaluation. Retaining shared HAM10000 identities, and thereby their acquisition source, in the ISIC 2019 model-development pool was associated with an 18.2-point macro-F1 difference. For the datasets and models tested, evaluation boundaries produced larger macro-F1 changes than the fusion methods, highlighting the need to align the separation unit with the intended generalization target.
Source: Mathematics
Institution: Keimyung University
Language: English
Subfield: Oncology
Field: Medicine
Open Access status: gold
Estimated APC paid: $3,137
OpenAlex ID: W7236620619
FWCI:
Cites: 27
Cited by:
Related to: 10