Year: 2025
Type: conference-paper
Abstract: We introduce NSF-SCIFY, a comprehensive dataset of scientific claims and investigation proposals extracted from National Science Foundation award abstracts.While previous scientific claim verification datasets have been limited in size and scope, NSF-SCIFY represents a significant advance with 2.8 million claims from 400,000 abstracts spanning all science and mathematics disciplines.We present two focused subsets: NSF-SCIFY-MATSCI with 114,000 claims from materials science awards, and NSF-SCIFY-20K with 135,000 claims across five NSF directorates.Using zero-shot prompting, we develop a scalable approach for joint extraction of scientific claims and investigation proposals.We demonstrate the dataset's utility through three downstream tasks: non-technical abstract generation, claim extraction, and investigation proposal extraction.Fine-tuning language models on our dataset yields substantial improvements, with relative gains often exceeding 100%, particularly for claim and proposal extraction tasks.Our error analysis reveals that extracted claims exhibit high precision but lower recall, suggesting opportunities for further methodological refinement.NSF-SCIFY enables new research directions in large-scale claim verification, scientific discovery tracking, and meta-scientific analysis 1 .
Authors: Delip Rao, Weiqiu You, Eric Wong, Chris Callison-Burch
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W4416033693