Work

Improving Fault Localization with External Oracle by Using Counterfactual Execution

Year: 2024

Type: article

Abstract: We present Flex , a new approach to improve fault localization with external oracles. Spectrum-based fault localization techniques estimate suspicious statements based on the execution trace of the test suite. State-of-the-art techniques rely on test oracles that internally exist in the program. However, programs often have external oracles that observe their behavior from outside. This in turn hinders fine-grained and accurate estimation of suspicious statements in practice because the correctness of each execution can only be observed at termination. In this article, we aim to address this problem by observing counterfactual execution traces, which enable fine-grained estimation even without precise internal oracles. We observe two types of counterfactual scenarios related to different types of test cases: When the branch condition is set to a Boolean constant, (1) if most of the passing test cases still pass, we consider the newly executed statements in the branch statement as unrelated to the failure; (2) if failing test case still fails, we also consider the originally executed statements as unrelated to the failure. We evaluated the performance on widely used C and Java programs. Flex improves the accuracy of state-of-the-art SBFL techniques on C and Java programs by 24% and 22% on average, respectively.

Source: ACM Transactions on Software Engineering and Methodology

DOI: 10.1145/3695997

Authors: Jongchan Park, Tae Eun Kim, Dongsun Kim, Kihong Heo

Institutions: Korea Advanced Institute of Science and Technology, Kyungpook National University

Language: English

Topic: Software Testing and Debugging Techniques

Subfield: Software

Field: Computer Science

Domain: Physical Sciences

Open Access status: hybrid

Funder: National Research Foundation of Korea

Award: G3418871914