Sex classification from hand X-ray images in pediatric patients: How zero-shot Segment Anything Model (SAM) can improve medical image analysis
Year: 2025
Type: article
Abstract: The potential to classify sex from hand data is a valuable tool in both forensic and anthropological sciences. This work presents possibly the most comprehensive study to date of sex classification from hand X-ray images. The research methodology involves a systematic evaluation of zero-shot Segment Anything Model (SAM) in X-ray image segmentation, a novel hand mask detection algorithm based on geometric criteria leveraging human knowledge (avoiding costly retraining and prompt engineering), the comparison of multiple X-ray image representations including hand bone structure and hand silhouette, a rigorous application of deep learning models and ensemble strategies, visual explainability of decisions by aggregating attribution maps from multiple models, and the transfer of models trained from hand silhouettes to sex prediction of prehistoric handprints. Training and evaluation of deep learning models were performed using the RSNA Pediatric Bone Age dataset, a collection of hand X-ray images from pediatric patients. Results showed very high effectiveness of zero-shot SAM in segmenting X-ray images, the contribution of segmenting before classifying X-ray images, hand sex classification accuracy above 95% on test data, and predictions from ancient handprints highly consistent with previous hypotheses based on sexually dimorphic features. Attention maps highlighted the carpometacarpal joints in the female class and the radiocarpal joint in the male class as sex discriminant traits. These findings are anatomically very close to previous evidence reported under different databases, classification models and visualization techniques. • A zero-prompt strategy for hand segmentation in X-ray images based on zero-shot SAM. • A thorough sex classification study from original and segmented hand X-ray images. • A low-cost deep learning solution for sex classification from hand X-ray images. • Visual explanation of model decisions from the aggregation of attention maps. • Novel use case in biological anthropology: sex estimation of prehistoric handprints.
Institutions: Universitat Jaume I, Universidad Autónoma de Ciudad Juárez
Language: English
Subfield: Archeology
Field: Arts and Humanities
Domain: Social Sciences
Open Access status: hybrid
Estimated APC paid: $3,080
OpenAlex ID: W4414167433