Work

MAVEN-FACT: A Large-scale Event Factuality Detection Dataset

Year: 2024

Type: conference-paper

Abstract: Event Factuality Detection (EFD) task determines the factuality of textual events, i.e., classifying whether an event is a fact, possibility, or impossibility, which is essential for faithfully understanding and utilizing event knowledge.However, due to the lack of highquality large-scale data, event factuality detection is under-explored in event understanding research, which limits the development of EFD community.To address these issues and provide faithful event understanding, we introduce MAVEN-FACT, a large-scale and high-quality EFD dataset based on the MAVEN dataset.MAVEN-FACT includes factuality annotations of 112, 276 events, making it the largest EFD dataset.Extensive experiments demonstrate that MAVEN-FACT is challenging for both conventional fine-tuned models and large language models (LLMs).Thanks to the comprehensive annotations of event arguments and relations in MAVEN, MAVEN-FACT also supports some further analyses and we find that adopting event arguments and relations helps in event factuality detection for fine-tuned models but does not benefit LLMs.Furthermore, we preliminarily study an application case of event factuality detection and find it helps in mitigating event-related hallucination in LLMs.Our dataset and codes can be obtained from https: //github.com/THU-KEG/MAVEN-FACT

Source: Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP)

DOI: 10.18653/v1/2024.findings-emnlp.651