Red Teaming Generative Ai Applications: Threat Modeling and Mitigation Strategies
Year: 2025
Type: conference-paper
Abstract: As generative AI systems grow in adoption and complexity, they introduce novel security, safety and alignment risks that challenge traditional evaluation and defense paradigms. To address these, we focus on a structured five-phase red teaming workflow consisting of reconnaissance, enumeration, exploitation, impact realization and persistence specifically tailored to GenAI’s unique threat landscape. Through real- world case studies and examples, we illustrate how adversaries exploit model vulnerabilities, bypass alignment mechanisms and cause persistent harm. We also identify emerging GenAI security tools and map each red teaming phase to actionable mitigations that support safe deployment. Our goal is to connect AI safety theory with practical adversarial resilience for researchers, developers and policymakers.
Authors: Lukić, Matea, Poledica, Ana, Milosevic, Pavle
Institution: University of Belgrade
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7106320067
FWCI:
Cites:
Cited by:
Related to: