Work

Program Structure-aware Language Models: Targeted Software Testing beyond Textual Semantics

Year: 2026

Type: conference-paper

Abstract: Recent advances in large language models for test case generation have improved branch coverage via prompt-engineered mutations.However, they still lack principled mechanisms for steering models toward specific high-risk execution branches, limiting their effectiveness for discovering subtle bugs and security vulnerabilities.We propose GLMTest, the first program structure-aware LLM framework for targeted test case generation that seamlessly integrates code property graphs and code semantics using a graph neural network and a language model to condition test case generation on execution branches.This structured conditioning enables controllable and branch-targeted test case generation, thereby potentially enhancing bug and security risk discovery.Experiments on realworld projects show that GLMTest built on a Qwen2.5-Coder-7B-Instructmodel improves branch accuracy from 27.4% to 50.2% on Test-GenEval benchmark compared with state-ofthe-art LLMs, i.e., Sonnet-4.5 from Claude and 4o-mini from OpenAI.

Source: Findings of the Association for Computational Linguistics: ACL (Findings of ACL)

DOI: 10.18653/v1/2026.findings-acl.540

Authors: Khang Tran, Khoa Nguyen, Cristian Borcea, Hai Phan

Institution: New Jersey Institute of Technology

Language: English

Topic: Software Testing and Debugging Techniques

Subfield: Software

Field: Computer Science

Domain: Physical Sciences

Open Access status: hybrid

Funder: National Science Foundation

Awards: Collaborative Research: SaTC: CORE: Medium: Understanding the Impact of Privacy Interventions on the Online Publishing Ecosystem, CyberCorps Scholarship for Service (Renewal): Secure Computing Initiative