A lightweight knowledge distillation framework for few shot unsupervised industrial surface anomaly detection and localization
Year: 2026
Type: article
Abstract: Industrial surface anomaly detection under data-scarce, unsupervised conditions requires models that are both label-efficient and computationally lightweight. This paper presents LKDF-AD, a knowledge-distillation framework combining a frozen pretrained teacher with a compact student network (385,168 parameters), trained using only k = 1–10 normal reference images per category, without any labeled anomalies. Evaluated across 15 MVTec AD categories and five shot counts, the framework achieves 77.88–86.19% image-level and 90.59–94.87% pixel-level AUROC, and generalizes to the independent BTAD benchmark (73.6–90.99% image-level AUROC), with a larger BTAD pixel-AUROC/PRO gap flagged as an open question. Measured end-to-end (teacher plus student together, as deployed), the framework uses 1.93× fewer parameters (11.56 M vs. 22.35 M) and achieves 1.57× lower inference latency (4.436 ms vs. 6.954 ms) than a full-size STFPM-style baseline trained under an identical protocol. Under category-level paired testing (using 15 independent category means, the statistically appropriate unit given repeated measurements across shot counts), neither the image-level nor the pixel-level advantage over STFPM reaches statistical significance (two-sided p = 0.7197 and p = 0.188, respectively). Consequently, the framework’s established advantage over STFPM is its directly measured computational efficiency (1.93× fewer end-to-end parameters, 1.57× lower end-to-end latency), rather than accuracy. In contrast, against the RegAD-inspired baseline, both accuracy advantages remain highly significant under the same corrected test (two-sided p = 0.00201 for image-level and p = 0.00153 for pixel-level). A random, completely untrained student achieves only near-chance performance (52.7% image-level, 46.6% pixel-level AUROC), confirming that the observed localization signal reflects genuine distillation rather than an architectural artifact. A dedicated investigation of the framework’s weakest category (screw, near-chance performance) tested a rotation-sensitivity hypothesis directly across the full test set; the result was a significant correlation in the direction opposite to that hypothesized, and we report the mechanism as unresolved rather than settled. These results position the framework as a genuinely lightweight, empirically characterized — rather than definitively superior — approach to few-shot unsupervised industrial anomaly detection.
Source: Scientific Reports
Author: Xian Gao
Institution: University of Glasgow
Language: English
Subfield: Industrial and Manufacturing Engineering
Field: Engineering
Domain: Physical Sciences
Open Access status: gold
Estimated APC paid: $2,690
OpenAlex ID: W7222296774
FWCI:
Cites:
Cited by:
Related to: