Year: 2024
Type: article
Abstract: The semantic understanding of numbers requires association with context.However, powerful neural networks overfit spurious correlations between context and numbers in training corpus can lead to the occurrence of contextual bias, which may affect the network's accurate estimation of number magnitude when making inferences in real-world data.To investigate the resilience of current methodologies against contextual bias, we introduce a novel out-ofdistribution (OOD) numerical question-answering (QA) dataset that features specific correlations between context and numbers in the training data, which are not present in the OOD test data.We evaluate the robustness of different numerical encoding and decoding methods when confronted with contextual bias on this dataset.Our findings indicate that encoding methods incorporating more detailed digit information exhibit greater resilience against contextual bias.Inspired by this finding, we propose a digit-aware position embedding strategy, and the experimental results demonstrate that this strategy is highly effective in improving the robustness of neural networks against contextual bias.
Authors: Xuehao Du, Ping Ji, Wei Qin, Lei Wang, Yunshi Lan
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: diamond
OpenAlex ID: W4403140229