Work

The 2nd Automated Verification of Textual Claims (AVeriTeC) Shared Task: Open-weights, Reproducible and Efficient Systems

Year: 2025

Type: conference-paper

Abstract: In the First Automated Verification of Textual Claims (AVERITEC) shared task, participating teams developed systems that for each claim retrieve evidence from the web and predict its veracity.While there was progress in automated fact-checking for real-world claims, the majority of the systems proposed relied on large closed-weights language models, which rendered them expensive to run and less reproducible.To ameliorate this issue, in this year's edition of the AVERITEC shared task, we required system to use only open-weights models that could be run using a single GPU with 23GBs of RAM, and that systems should take one minute or less to return verdicts accompanied by evidence retrieved from a pre-compiled knowledge store.The shared task received 7 submissions; 6 of which exceeded the accuracy of our baseline on the test set, while they ran in under a minute per claim on the hardware we had specified.The winning team was CTU AIC with an AVERITEC score of 33.17%.In this paper we describe the shared task in detail and highlight key findings.

DOI: 10.18653/v1/2025.fever-1.15