Work

Transformer-Guided Serial Knowledge Distillation for High-Precision Anomaly Detection

Year: 2025

Type: article

Abstract: Unsupervised anomaly detection(AD) remains a notable challenge in computer vision research, due to the inherent absence of annotated anomalous data and the unpredictable nature of anomaly manifestations. To address these challenges, a novel Transformer-based knowledge distillation framework is proposed through a hierarchical architecture designed for improved anomaly recognition. This three-stage architecture consists of a fixed pretrained teacher network serving as the upstream feature extractor, a dedicated multi-feature aggregation and filtering module integrated with Vision Transformer components as the intermediate processor, and a trainable student network functioning as the downstream reconstruction module. Extensive evaluations on benchmark datasets demonstrate that the proposed method achieves high performance in both AD accuracy and anomaly localization precision, while also maintaining strong generalization across diverse anomaly types. Notably, it shows particular effectiveness in industrial inspection scenarios, where anomalous patterns are subtle and training data are strictly limited to normal samples.

Source: IEEE Access

DOI: 10.1109/access.2025.3584892

Authors: Danyang Wang, Bingyan Wang

Institutions: Henan University of Urban Construction, Zhejiang University of Science and Technology

Language: English

Topic: Anomaly Detection Techniques and Applications

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: gold

Estimated APC paid: $2,075

Funders: Natural Science Foundation of Henan Province, Key Scientific Research Project of Colleges and Universities in Henan Province