Visual distinctiveness drives memorization beyond rarity in fine-tuned medical imaging models
Year: 2026
Type: preprint
Abstract: Medical AI models trained on patient data can memorize individual samples,exposing them to membership inference attacks. Long-tail theory predicts rareexamples require memorization, yet rarity alone is insufficient: in a skin-lesiondataset, the rarest class is less vulnerable than the commoner ones. Under ran-domly initialised (MAE) and medical-image-pretrained (MedSAM) encoders,the rarest class stays most memorized, but ImageNet- and vision–language-pretrained encoders re-rank the tail toward visually distinctive classes. Across fivedatasets, controlled grayscale manipulation gives causal evidence: the same trans-form produces opposite effects on two equally rare classes, and an independentfeature-space atypicality measure predicts memorization across architectures.Memorization predicts membership inference vulnerability (ρ = 0.36–0.77,p < 0.001), and medical-domain pretraining (MedSAM, BiomedCLIP) does notreduce rare-class risk. DP-LoRA at ε = 1 reduces the most vulnerable class’s memorization by 90% versus a LoRA-only baseline, with modest accuracy coston simple tasks. We release medmem, an open-source memorization-auditing tool.
Source: Research Square
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7204104848