Year: 2025
Type: preprint
Abstract: Although formal methods are capable of producing reliable software, they have seen minimal adoption in everyday programming. Automatic code generation using large language models is becoming increasingly widespread, but it rarely considers producing strong correctness guarantees. In this study, we explore the ability of LLMs to produce verified code in three verification languages (Dafny, Nagini, and Verus). To do so, we use manually curated datasets derived from the state-ofthe-art Python benchmark, HumanEval. We also assess what types of information are sufficient to achieve good-quality results.
Source: arXiv (Cornell University)
Authors: Shefer, Aleksandr, Engel, Igor, Alekseev, Stanislav, Berezun, Daniil, Verbitskaia, Ekaterina, Podkopaev, Anton
Institutions: Constructor University, Neapolis University Pafos
Language: English
Subfield: Software
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W6891941702