Evaluating Spacecraft Telemetry Anomaly Detection Beyond Clean Benchmarks: A Forensic Study on Real ESA Mission Data with Ground-System Integration
Year: 2026
Type: preprint
Abstract: Spacecraft telemetry anomaly detectors are usually evaluated on a fixed positive population, which hides how they behave once evaluation is widened. We report a forensic case study on reconstructed real ESA Mission-1 telemetry that follows one detector through that widening. A frozen, interpretable robust-evidence detector (Detector v1) detects all 30 evaluated Class-3 events; against a predetermined population of 170 label-negative/unlabelled controls it additionally activates on 40 (23.53%), a control-detection rate that is not a confirmed operational false positive rate because the controls were never certified nominal. A structural analysis finds this activation strongly associated with channel dimensionality and identifies a detector-level dependency (100% detection in the highest-dimensionality stratum) that a direct population comparison narrows but does not eliminate as an explanation. A hypothesis-driven redesign (Detector v2), evaluated on the same paired 30-positive/60-control population as v1, does not improve on it. An external protocol/domain stress test on OPSSAT-AD which does not itself run Detector v1 shows near-chance discrimination under a substitute robust-threshold protocol. An actual Yamcs 5.13.2 ground-system integration ingests real ESA telemetry over UDP and reaches numerical equivalence on five of six replay cases, with the sixth blocked by incomplete historical archive coverage rather than any detector or protocol fault. We report all of these results, including the negative and unresolved ones, and treat the resulting evidence chain not any single metric as the paper's contribution: a documented account of what a realistic, honestly audited evaluation and deployment attempt actually looks like, together with an explicit list of what remains unestablished.
Author: Thokal, Aaditya
Institution: University of Mumbai
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7225562700