Work

Towards Scalable Evaluation of Software Understanding: A Methodology Proposal

Year: 2025

Type: conference-paper

Abstract: In reverse engineering our goal is to build systems that help people to understand software. However, the field has not converged on a way to measure software understanding. In this paper, we make the case that understanding should be measured via performance on understanding-questions. We propose a method for constructing understanding-questions and evaluating answers at scale. We conduct a case study in which we apply our method and compare Ghidra’s default auto analysis with an analysis that supports binary constructs that are specific to Objective-C.

DOI: 10.1145/3733822.3764672

Authors: Florian Magin, Magdalena Wache, Fabian Scherf, Cléo Fischer, Jonas Zabel

Institutions: Fraunhofer Institute for Secure Information Technology, National Research Center for Applied Cybersecurity ATHENE

Language: English

Topic: Software Engineering Research

Subfield: Information Systems

Field: Computer Science

Domain: Physical Sciences

Open Access status: green

OpenAlex ID: W4415721987

FWCI: 1.175

Cites: 9

Cited by: 2

Related to: