ArtefactSense: Do LLMs Notice When a Wearable Biosignal Is Unreadable?
Year: 2026
Type: preprint
Abstract: Language models are now asked to read raw wearable signals, yet real recordings can be corrupted by motion, lost skin contact, amplifier saturation or mains hum. A trustworthy reader should then flag the window instead of answering. ArtefactSense, the dataset, generator, scorer and harness we release, injects graded artefacts into real ECG, PPG and EMG windows and asks a question whose ground truth and usability label come from a reference pipeline; the frozen release holds 1500 items, 21.8% of them unusable, read by Claude and open-weight models as text and images. AURC, the area under the error-rate curve as more items are answered (lower is better), is 0.125–0.282 for the Claude models and 0.565–0.847 for open-weight ones, yet no model abstains on more than 23.5% (Claude) or 58.7% (open-weight) of unusable items. Claude's stated confidence on unusable items piles at 0.5, so the share it answers confidently is 36.7–65.1% at a 0.5 cut-off and 0.0–12.5% at a 0.7 cut-off. A non-LLM baseline of classic signal-quality rules, fit leave-one-subject-out on a separate tuning set, flags 85.9% of unusable items, and gating a Claude model with it leaves at most 10.1% unflagged. Under matched generation settings Haiku and Sonnet still leave 52.4–62.7% of unusable items unflagged, and a quality-first instruction raises that share in 5 of 6 rows. Dataset, code and scorer are released at https://doi.org/10.5281/zenodo.24185517.
Authors: Kandukuri, Tharun Reddy, Mallia, Giorgio
Language: English
Topic: ECG Monitoring and Analysis
Subfield: Cardiology and Cardiovascular Medicine
Field: Medicine
Domain: Health Sciences
Open Access status: green
OpenAlex ID: W7225996001
