Year: 2025
Type: conference-paper
Abstract: This paper proposes a method for automatic GUI component detection for the IBM i system (formerly and still more commonly known as AS/400).We introduce a human-annotated dataset consisting of 1,050 system screen images, in which 381 images are screenshots of IBM i system screens in Japanese.Each image contains multiple components, including text labels, text boxes, options, tables, instructions, keyboards, and command lines.We then develop a detection system based on state-of-the-art deep learning models and evaluate different approaches using our dataset.The experimental results demonstrate the effectiveness of our dataset in constructing a system for component detection from GUI screens.By automatically detecting GUI components from the screen, AS400-DET has the potential to perform automated testing on systems that operate via GUI screens.
Authors: Thanh Tran, Son T. Luu, Quan Bui, Shoshin Nomura
Institution: Japan Advanced Institute of Science and Technology
Language: English
Subfield: Software
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W4414630232