Work

Over-Reasoning and Redundant Calculation of Large Language Models

Year: 2024

Type: conference-paper

Abstract: Large language models (LLMs) can solve problems step-by-step.While this chainof-thought (CoT) reasoning boosts LLMs' performance, it is unclear if LLMs know when to use CoT and whether those CoT are always necessary to answer the question.This paper shows that LLMs tend to generate redundant calculations and reasoning on a manually constructed math QA dataset, GSM8K-Zero.GSM8K-Zero is constructed such that the questions can be answered without any calculations, but LLMs, including Llama-2 models and Claude-2, tend to generate lengthy and unnecessary calculations to answer the questions.We also conduct experiments to explain why LLMs generate redundant calculations and reasonings.

Source: Conference of the European Chapter of the Association for Computational Linguistics

DOI: 10.18653/v1/2024.eacl-short.15

Authors: Cheng-Han Chiang, Hung-yi Lee

Institution: National Taiwan University

Language: English

Topic: Topic Modeling

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: hybrid

Funders: National Applied Research Laboratories, Delta Electronics

OpenAlex ID: W4411638675