Year: 2024
Type: conference-paper
Abstract: Prompt engineering has emerged as a key area of focus in natural language processing, with role-playing techniques gaining attention for their potential to enhance model interaction and generation. Despite this interest, there is limited research on how role-playing prompts influence mathematical reasoning tasks. This paper examines the use of role-playing prompts with language models and highlights significant limitations in their effectiveness. Our findings reveal unexpected and counterintuitive results: these role-playing prompts do not improve reasoning abilities and may even degrade performance. We attribute this degradation to a mismatch between the role assumed in the prompt and the problem-solving skills required, where the capabilities provided by the role do not align with the abilities needed to solve the problem. Our findings challenge prevailing assumptions and underscore the need for a more nuanced approach to prompt engineering, highlighting the importance of ensuring that role-based strategies are well-suited to the specific cognitive demands of reasoning tasks.
Authors: Zhiguang Han, Zijian Wang
Institutions: Nanyang Technological University, The University of Sydney
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
SDG: Quality education
Open Access status: gold
OpenAlex ID: W4403052263