Work

LLPut: Investigating Large Language Models for Bug Report-Based Input Generation

Year: 2025

Type: article

Abstract: Failure-inducing inputs play a crucial role in diagnosing and analyzing software bugs. Bug reports typically contain these inputs, which developers extract to facilitate debugging. Since bug reports are written in natural language, prior research has leveraged various Natural Language Processing (NLP) techniques for automated input extraction. With the advent of Large Language Models (LLMs), an important research question arises: how effectively can generative LLMs extract failure-inducing inputs from bug reports? In this paper, we propose LLPut, a technique to empirically evaluate the performance of three open-source generative LLMs -- LLaMA, Qwen, and Qwen-Coder -- in extracting relevant inputs from bug reports. We conduct an experimental evaluation on a dataset of 206 bug reports to assess the accuracy and effectiveness of these models. Our findings provide insights into the capabilities and limitations of generative LLMs in automated bug diagnosis.

Source: Maryland Shared Open Access Repository (USMAI Consortium)

DOI: 10.13016/m2ymbn-ka55

Authors: Hasan, Alif Al, Saha, Subarna, Imran, Mia Mohammad, Zaman, Tarannum Shaila

Language: English

Topic: Software Engineering Research

Subfield: Information Systems

Field: Computer Science

Domain: Physical Sciences

Open Access status: green

Funder: National Science Foundation

Awards: CRII: SHF: An Automated and User-centered Framework for Reproducing System-level Concurrency Bugs by Analyzing Bug Reports, CRII: SHF: An Automated and User-centered Framework for Reproducing System-level Concurrency Bugs by Analyzing Bug Reports

OpenAlex ID: W6959473574

FWCI:

Cites:

Cited by:

Related to: