Work

Recall Mechanism and Multi-Head Attention for Numerical Reasoning

Year: 2025

Type: article

Abstract: Numerical reasoning is a challenging question-answering task in artificial intelligence (AI) that requires both reading comprehension and numerical computation capabilities. Although recent approaches have made significant progress in reasoning, two critical issues remain: (1) information tends to be gradually lost as the network deepens due to the complexity of deep learning models, and (2) the performance of multi-step reasoning is suboptimal, leaving room for improvement in accuracy. To address these issues, we propose a model with a recall mechanism and multi-head attention for numerical reasoning (RMMANR). The recall mechanism prevents the embedding information of questions and passages from being forgotten as the model deepens, while the multi-head attention mechanism analyzes possible solutions for numerical reasoning. RMMANR consists of two main components: an encoder and a decoder. The encoder leverages RoBERTa to encode the question and passage into contextual embeddings. The decoder, which consists of four modules (RM Module, Selector, MA Module, and Program Solver), generates inference steps and answers based on contextual information from the encoder. We implement our model using PyTorch and evaluate it on the FINQA dataset, a benchmark for numerical reasoning in the financial domain. Experimental results demonstrate that RMMANR outperforms several baseline models, achieving superior accuracy.

Source: Applied Sciences

DOI: 10.3390/app15073528

Authors: Linjia Lai, Tien-Ping Tan, Bocan Zeng

Institutions: Universiti Sains Malaysia, Putian University

Language: English

Topic: Topic Modeling

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: gold

Estimated APC paid: $2,896

Funders: Putian Science and Technology Bureau, Putian University