Improving the Practicality of Program Synthesis: Prototyping, Context Granularity, and Self-Correction
Year: 2026
Type: dissertation
Abstract: Program synthesis is the task of automatically generating a computer program that satisfies a set of constraints known as a specification. It has the potential to make programming accessible to people with limited or no programming experience. Despite decades of progress, program synthesis has seen limited deployment in real-world settings. This thesis argues that the reasons for this limited deployment have changed over time. Most synthesis systems decompose into three core components: a grammar, a solver, and a ranker. Grammar constraints make traditional systems tractable but restrict their generality. Large language models relax these constraints and can generate much larger programs across a broader range of languages. This flexibility reinforces a longstanding bottleneck. A model may generate code that appears correct and even passes the available tests while still failing to satisfy the user's intent. This thesis examines both eras through four studies. The first study uses semi-structured interviews with eleven researchers and practitioners to examine why interactive synthesis systems are difficult to prototype. The findings show that their core components are tightly coupled. Builders can neither anticipate a synthesizer's behavior before it exists nor faithfully simulate its outputs once it is running. They therefore substitute a partially working "half-baked" synthesizer for a prototype. The second study shows that increasing context does not produce proportional improvements in repair performance. Small dependence-aware contexts recover most or all of the full-file performance at a fraction of the token cost. The third study finds that self-correction improves correctness only briefly and inconsistently. Most of the improvement occurs during the first revision, and recovery depends more on task-level solvability than on the correction loop. The fourth study introduces a cross-model property-based evaluation in which an independent model audits programs that pass the public tests. This evaluation reveals correctness failures that example-based test suites cannot detect.
Author: Holloway, Tyler
Language: English
Subfield: Information Systems
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7223163765
