AI detectors are poor western blot classifiers: a study of accuracy and predictive values
Year: 2025
Type: article
Abstract: The recent rise of generative artificial intelligence (AI) capable of creating scientific images presents a challenge in the fight against academic fraud. This study evaluates the efficacy of three free web-based AI detectors in identifying AI-generated images of western blots, which is a very common technique in biology. We tested these detectors on AI-generated western blot images (n = 48, created using ChatGPT 4) and on authentic western blots (n = 48, from articles published before the rise of generative AI). Each detector returned a very different sensitivity (Is It AI?: 0.9583; Hive Moderation: 0.1875; and Illuminarty: 0.7083) and specificity (Is It AI?: 0.5417; Hive Moderation: 0.8750; and Illuminarty: 0.4167), and the predicted positive predictive value (PPV) for each was low. This suggests significant challenges in confidently determining image authenticity based solely on the current free AI detectors. Reducing the size of western blots reduced the sensitivity, increased the specificity, and did not markedly affect the accuracy of the three detectors, and only slightly improved the PPV of one detector (Is It AI?). These findings highlight the risks of relying on generic, freely available detectors that lack sufficient reliability, and demonstrate the urgent need for more robust detectors that are specifically trained on scientific contents such as western blot images.
Source: PeerJ
DOI: 10.7717/peerj.18988
Author: Romain-Daniel Gosselin
Institution: Centre Hospitalier Universitaire Vaudois
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: gold
Estimated APC paid: $1,395
OpenAlex ID: W4407794105