Year: 2025
Type: article
Abstract: ABSTRACT GUI ripping explores the graphical user interface of an application to build a model which can express the application behavior. The ripped GUI model is useful in various software engineering tasks. Traditional GUI ripping techniques depend on the underlying GUI frameworks to provide the GUI structure information. They are difficult to work across platforms or on nonnative applications where the GUI structure information cannot easily be obtained. This work introduces cvrip, a visual GUI ripping framework, to address the problem. cvrip visually analyzes the GUI screen for ripping and does not rely on the underlying GUI frameworks. We introduce many new techniques to enable efficient visual GUI ripping, for example, a YOLO v5‐based model to detect executable widgets, a state recognition acceleration method for fast model updating, and several GUI exploration strategies taking the characteristics of imperfect visual analysis into account. Experiments are conducted to evaluate many technique choices in visual GUI ripping and compare the solution with the traditional style ripping. The results show that cvrip can get competitive exploration coverage compared to traditional approaches. This suggests visual GUI ripping is a direction worthy of more future studies.
DOI: 10.1002/smr.70009
Authors: Heji Huang, Ju Qian, Wenduo Jia, Yiming Jin
Institution: Nanjing University of Aeronautics and Astronautics
Language: English
Subfield: Software
Field: Computer Science
Domain: Physical Sciences
Open Access status: bronze
Award: G1637344061
OpenAlex ID: W4407866018