Winning ClimateCheck: A Multi-Stage System with BM25, BGE-Reranker Ensembles, and LLM-based Analysis for Scientific Abstract Retrieval
Year: 2025
Type: conference-paper
Abstract: The ClimateCheck shared task addresses the critical challenge of grounding social media claims about climate change in scientific literature.This paper details our winning approach for solving two subtasks.For abstract retrieval, we propose a multi-stage pipeline:(1) initial candidate generation from a corpus of ∼400,000 abstracts using BM25; (2) finegrained reranking of these candidates using an ensemble of BGE-Reranker cross-encoder models, fine-tuned with a specialized training set incorporating both random and hard negative samples; and (3) final list selection based on an RRF-ensembled score.For the verification aspect, we leverage Gemini 2.5 Pro to classify the relationship between claims and the retrieved abstracts.Our system achieved first place in both subtasks.Part of the example code: https://github.com/cklcklcklckl/ climatecheck_1st_solution.
Language: English
Subfield: Information Systems
Field: Computer Science
Domain: Physical Sciences
SDG: Climate action
Open Access status: hybrid
OpenAlex ID: W4412944224