Work

Mixed Information Bottleneck for Location Metonymy Resolution Using Pre-trained Language Models

Year: 2025

Type: article

Abstract: Metonymy resolution (MR) is a crucial challenge in natural language understanding and information retrieval. Recent large-scale pre-trained language models have shown promising results in various natural language processing (NLP) tasks, including MR. Despite these achievements, current models still struggle in many real-world scenarios. Since these models rely heavily on contextual information and ignore entity information, they are prone to extract irrelevant features and overfit when fine-tuned with less training data. In this article, we propose a mixed information bottleneck framework to address the above issues, which learns optimal data representations based on the principle of minimal sufficiency. Our model can effectively mitigate irrelevant features in context and entity by using different types of information bottlenecks for entity and context information separately while reducing the dimensionality of latent representations. We show that our approach achieves state-of-the-art performance on three benchmark datasets for location MR, outperforming previous Bert-based methods by a large margin. Ablation studies and qualitative analysis show the effectiveness of our models in reducing dimensionality while extracting more relevant features.

Source: ACM Transactions on Asian and Low-Resource Language Information Processing

DOI: 10.1145/3774933

Authors: Hao Wang, Tang Li, Siyuan Du, Xiao Wei

Institution: Shanghai University

Language: English

Topic: Topic Modeling

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: bronze

Funder: National Natural Science Foundation of China

Award: G3879535228