Year: 2026
Type: conference-paper
Abstract: Automated Fact-Checking has largely focused on verifying general knowledge against static corpora, overlooking high-stakes domains like law where truth is evolving and technically complex.We introduce CaseFacts, a benchmark for verifying colloquial legal claims against U.S. Supreme Court precedents.Unlike existing resources that map formal texts to formal texts, CaseFacts challenges systems to bridge the semantic gap between layperson assertions and technical jurisprudence while accounting for temporal validity.The dataset consists of 6,294 claims categorized as SUP-PORTED, REFUTED, or OVERRULED.We construct this benchmark using a multi-stage pipeline that leverages Large Language Models (LLMs) to synthesize claims from expert case summaries, employing a novel semantic similarity heuristic to efficiently identify and verify complex legal overrulings.Experiments with state-of-the-art LLMs reveal that the task remains challenging; notably, augmenting models with unrestricted web search degrades performance compared to closed-book baselines due to the retrieval of noisy, non-authoritative precedents.We release CaseFacts 1 to spur research into legal fact verification systems. Case Input (Marbury v. Madison)Facts of the case: William Marbury had been appointed Justice of the Peace in the District of Columbia, but his commission was not delivered... Question: Do the plaintiffs have a right to receive their commissions?Can they sue for their
Institution: The University of Texas at Arlington
Language: English
Field: Social Sciences
Domain: Social Sciences
Open Access status: hybrid
OpenAlex ID: W7166854127