Enhancing Logical Reasoning in Large Language Models via Multi-Stage Ensemble Architecture with Adaptive Attention and Decision Voting
Year: 2024
Type: conference-paper
Abstract: Increasing difficulty of logical reasoning tasks suggests the deficiency of conventional large language models (LLMs) in solving multi-step reasoning problems. This paper presents a novel multi-stage ensemble method to enhance LLMsQS reasoning capability. The design includes cascading networks, in which dynamic candidates are selected as the inputs and an adaptive self-attention mechanism and voting are applied for decision- making, which together enhance the model's accuracy and robustness. Based on the confidence levels, candidate selection is carried out and the candidates are then refined utilizing multi-head adaptive attention. Then, decision voting fuses reasoning results of several different models, which improves the stability and diversity of the model. Proposed different from state-of-the-art methods, the multi-stage ensemble approach significantly enhances the logic reasoning ability of the model, especially in more complex multi-step problems according to the experimental results. An ablation study validates that the adaptive attention mechanism and ensemble decision method contribute positively to enhancing the model performance. Not only does this out-perform classical models in accuracy, but it also strikes a great balance in reasoning efficiency and model interpretability, thus establishing a new paradigm for AI-based logical reasoning tasks.
Authors: Wenqing Zhang, Zhan Cheng, Fu Lei, Beichen Liu, Dian Gu, Zeyu Wang
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W4410250736