Work

Topology-aware unsupervised intrusion detection from network flows in encrypted traffic

Year: 2026
Type: article
Abstract: The rapid growth of encrypted network traffic challenges intrusion detection systems that rely on payload inspection. We propose a topology-aware unsupervised intrusion detection framework that models network flows as interaction graphs and applies a graph neural autoencoder to identify structural anomalies from flow metadata alone, without inspecting packet payloads. The autoencoder is trained with a topology-aware augmentation strategy (Gaussian feature noise, edge feature masking, node feature masking) together with edge-level contrastive learning. We evaluate the framework on six datasets organized into eight conditions, spanning enterprise, cloud, industrial IoT and testbed networks as well as end-to-end encrypted DNS-over-HTTPS (DoH) tunneling and command-and-control (C2) traffic over TLS and QUIC. Against six unsupervised baselines, including GraphIDS, Anomal-E, KitNET, and a variational autoencoder, the framework attains the highest mean ROC-AUC on both the general benchmarks (0.986) and the encrypted conditions (0.936). A benign-vs-benign origin control reveals a dataset-origin confound in cross-corpus DoH evaluation (origin-AUC 0.942), so the encrypted-DoH claim rests on a same-origin setting, where the proposed model reaches ROC-AUC 0.797. Without recalibration on the target, the model loses 3.0% (relative) of its mean ROC-AUC under cross-environment transfer and attains a mean cross-domain ROC-AUC of 0.911, compared with 0.415–0.609 for the baselines, and it processes a two-minute window in under 18 ms on a CPU, the highest throughput among the timed models.
Institution: Gachon University
Language: English
Open Access status: hybrid
Estimated APC paid: $2,460
OpenAlex ID: W7203831398
FWCI:
Cites: 28
Cited by:
Related to: 10