Thumb: A forensic automation framework leveraging MLLMs and OCR on Android device
Year: 2025
Type: article
Abstract: The forensic of Android devices is challenging due to automated thumbnail generation by applications and the operating system, complicating attribution to specific user actions. This paper presents the design, implementation, and evaluation of a forensic framework, Thumb, which performs real-time experiments on physical Android devices. Thumb integrates multimodal large language models (MLLM) and Optical Character Recognition (OCR) to capture on-screen information and simulate user interactions, while extracting data from internal storage to monitor changes in cached and thumbnail files. A proof-of-concept implementation demonstrates the framework's accuracy across various applications, highlighting its potential to simplify Android forensic analysis. However, current MLLM limitations and the framework's structure pose challenges in complex scenarios and detailed data analysis.
Authors: Dingjie Shang, Amin Sakzad, Stuart W. Hall
Institution: Monash University
Language: English
Topic: Digital and Cyber Forensics
Subfield: Information Systems
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
Estimated APC paid: $3,100
OpenAlex ID: W4411358869