Magic-Detector: A Multi-Feature Fusion Framework with Siamese Networks for Black-Box AI-Generated Text Detection
Year: 2026
Type: conference-paper
Abstract: The rapid development of large language models has significantly improved the fluency, coherence, and semantic quality of machine-generated text, making it increasingly difficult to distinguish AI-generated content from human-written text. Existing detection methods often rely on token-level probability distributions, model-specific features, or training data from specific generators, which limits their effectiveness in black-box scenarios where the target generation model is inaccessible or unknown. To address this challenge, this paper proposes Magic-Detector, a multi-feature fusion framework for black-box AI-generated text detection based on Siamese networks. Specifically, a weaker language model is first employed to infer a latent question from the input text and generate a reconstructed answer. Then, a Siamese network is used to capture representation-level differences between the original text and its reconstructed counterpart. In addition, statistical and semantic features, including n-gram probability and text similarity, are integrated to enhance the discriminative capability of the detector. The fused features are finally fed into a binary classifier to determine whether the input text is human-written or AI-generated. Experimental results on in-domain and out-of-domain datasets demonstrate that Magic-Detector achieves strong detection performance and generalization ability. Furthermore, adversarial robustness experiments show that the proposed method maintains higher accuracy than the RoBERTa baseline under different attack strategies. Unlike probability-based detectors, Magic-Detector does not require access to the token probabilities of the target generation model, making it suitable for detecting text generated by black-box or unknown large language models.
Authors: Pengcheng Su, Huiqi Luo, Yanzi Wang, Fei Shang
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W7204175307