Year: 2025
Type: article
Abstract: Securing the interaction layer of large language models (LLMs) against emerging threats is paramount for their responsible and ethical deployment. These powerful AI systems, while offering flexibility, are highly susceptible to prompt injection, abuse and the generation of malicious behaviors. In this paper, we propose a novel, ethical governance approach for responsible prompt engineering, based on pre inference semantic threat detection. This methodology involves converting user prompts into semantically rich embedding vectors using Sentence Transformer (all-MiniLM-L6-v2). These vectors are then classified using a Random Forest model into various cybersecurity threat categories, such as Malware Attacks, Cryptographic Attacks, Intrusion Attack, Evasion attack, and several others. Validated extensively on the Cysecbench dataset, a comprehensive benchmark of real-world cybersecurity prompts, the presented system shows a strong multi-class classification ability with an overall macro averaged precision of 0.91, recall of 0.89, F1-score 0.90, and an accuracy of 91.3%. This approach aligns with the SPIN framework and provides a scalable solution to reduce ethical risks in LLM deployment, thereby contributing to the development of responsible AI system.
Author: Prof. Sanjay Kumar Sharma
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: diamond
OpenAlex ID: W4411206263