Work

The ClimateCheck Shared Task: Scientific Fact-Checking of Social Media Claims about Climate Change

Year: 2025

Type: conference-paper

Abstract: Misinformation in public discourse on global and significant issues like climate change is often facilitated through social media.However, current systems do not address factchecking climate-related claims against trustworthy, evidence-based sources, such as scientific publications.To address this, we organised the ClimateCheck shared task at the 5th Scholarly Document Processing (SDP) Workshop, co-located with ACL 2025 in Vienna, Austria.The task featured two subtasks: I. Abstracts retrieval given a claim, and II.Claim verification based on the retrieved abstract.Cli-mateCheck had 27 registered users with active participation from 13 teams, ten of which submitted results for the first subtask and three for the second.The winning team achieved a Recall@10 score of 0.66 and a Binary Preference score of 0.49 for subtask I, and an F1 score of 0.73 for subtask II.Their method combined sparse retrieval using BM25, an ensemble of fine-tuned cross-encoder models using BGErerankers, and LLMs for classification.

Source: Workshop on Scholarly Document Processing (SDP)

DOI: 10.18653/v1/2025.sdp-1.24

Authors: Raia Abu Ahmad, Aida Usmanova, Georg Rehm

Institutions: Leuphana University of Lüneburg, German Research Centre for Artificial Intelligence, Humboldt-Universität zu Berlin

Language: English

Topic: Misinformation and Its Impacts

Subfield: Sociology and Political Science

Field: Social Sciences

Domain: Social Sciences

SDG: Climate action

Open Access status: hybrid

Funder: Deutsche Forschungsgemeinschaft