Year: 2026
Type: conference-paper
Abstract: Pluralism alignment with AI has the sophisticated and necessary goal of creating AI that can coexist with and serve morally multifaceted humanity.Research towards pluralism alignment has many efforts in enhancing the learning of large language models (LLMs) to accomplish pluralism.Although this is essential, the robustness of LLMs to produce moral content over pluralistic values is still under exploration.Inspired by the astonishing persuasion abilities via jailbreak prompts, we propose to leverage jailbreak attacks to study LLMs' internal pluralistic values.In detail, we develop a morality dataset with 10.3K instances in two categories: Value Ambiguity and Value Conflict.We further formalize four adversarial attacks with the constructed dataset, to manipulate LLMs' judgment over the morality questions.We evaluate both the large language models and guardrail models which are typically used in generative systems with flexible user input.Our experiment results show that there is a critical vulnerability of LLMs and guardrail models to these subtle and sophisticated moral-aware attacks.
Authors: Ying Su, Zheng Mingen, Weili Diao, Haoran Li
Institutions: South China University of Technology, Beihang University
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W7166879586