Work

Factual Knowledge in Language Models: Robustness and Anomalies under Simple Temporal Context Variations

Year: 2025

Type: conference-paper

Abstract: This paper explores the robustness of language models (LMs) to variations in the temporal context within factual knowledge.It examines whether LMs can correctly associate a temporal context with a past fact valid over a defined period, by asking them to differentiate correct from incorrect contexts.The LMs' ability to distinguish is analyzed along two dimensions: the distance of the incorrect context from the validity period and the granularity of the context.To this end, a dataset called TimeStress is introduced, enabling the evaluation of 18 diverse LMs.Results reveal that the best LM achieves a perfect distinction for only 11% of the studied facts, with errors, certainly rare, but critical that humans would not make.This work highlights the limitations of current LMs in temporal representation.

DOI: 10.18653/v1/2025.l2m2-1.1

Authors: Hichem Ammar Khodja, Frederic Bechet, Quentin Brabant, Alexis Nasr, Gwénolé Lecorvé

Institutions: Centre National de la Recherche Scientifique, Orange (France), Aix-Marseille Université, Laboratoire d’Informatique et Systèmes, International Laboratory on Learning Systems

Language: English

Topic: Topic Modeling

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: gold

OpenAlex ID: W4412944323

FWCI:

Cites:

Cited by:

Related to: 10