Proof Automation with Large Language Models
Year: 2024
Type: conference-paper
Abstract: Interactive theorem provers such as Coq are powerful tools to formally guarantee the correctness of software. However, using these tools requires significant manual effort and expertise. While Large Language Models (LLMs) have shown promise in automatically generating informal proofs in natural language, they are less effective at generating formal proofs in interactive theorem provers. In this paper, we conduct a formative study to identify common mistakes made by LLMs when asked to generate formal proofs. By analyzing 520 proof generation errors made by GPT-3.5, we found that GPT-3.5 often identified the correct high-level structure of a proof, but struggled to get the lower-level details correct. Based on this insight, we propose PALM, a novel generate-then-repair approach that first prompts an LLM to generate an initial proof and then leverages targeted symbolic methods to iteratively repair low-level problems. We evaluate PALM on a large dataset that includes more than 10K theorems. Our results show that PALM significantly outperforms other state-of-the-art approaches, successfully proving 76.6% to 180.4% more theorems. Moreover, PALM proves 1270 theorems beyond the reach of existing approaches. We also demonstrate the generalizability of PALM across different LLMs.
Authors: Minghai Lu, Benjamin Delaware, Tianyi Zhang
Institution: Purdue University West Lafayette
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
Funder: National Science Foundation
Awards: CAREER: Regularizing Large Language Models for Safe and Reliable Program Generation, SHF: Small: Automated Verification and Synthesis of Input Generators in Property-Based Testing Frameworks, Proto-OKN Theme 1: Knowledge Graph Construction for Resilient, Trustworthy, and Secure Software Supply Chains
OpenAlex ID: W4403536246