Work

Rethinking the Bounds of LLM Reasoning: Are Multi-Agent Discussions the Key?

Year: 2024

Type: conference-paper

Abstract: Recent progress in LLMs discussion suggests that multi-agent discussion improves the reasoning abilities of LLMs.In this work, we reevaluate this claim through systematic experiments, where we propose a novel group discussion framework to enrich the set of discussion mechanisms.Interestingly, our results show that a single-agent LLM with strong prompts can achieve almost the same performance as the best existing discussion approach on a wide range of reasoning tasks and backbone LLMs.We observe that the multi-agent discussion performs better than a single agent only when there is no demonstration in the prompt.Further study reveals the common interaction mechanisms of LLMs during the discussion. 1

Source: Proceedings of the conference - Association for Computational Linguistics. Meeting

DOI: 10.18653/v1/2024.acl-long.331

Authors: Qineng Wang, Zihao Wang, Ying Su, Hanghang Tong, Yangqiu Song

Institutions: Zhejiang University, Hong Kong University of Science and Technology, University of Illinois Urbana-Champaign

Language: English

Topic: Topic Modeling

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: hybrid

Funder: National Natural Science Foundation of China

Award: G4682916240