Year: 2026
Type: preprint
Abstract: Quantification of mouse scratching behavior is essential for preclinical itch research. However, manual video annotation is labor-intensive, and automated detection remains challenging in terms of accuracy and generalizability. We present Scratch-AID+, a computational framework dedicated to automated mouse scratching detection, including a diverse and progressive set of model families and learning strategies: classic ResNet-based detectors, task-adaptive fine-tuning of video foundation models, and in-context learning of advanced multimodal large language models. To simulate real-world use cases, we benchmark on a public dataset containing annotated video recordings from a source laboratory with additional cross-laboratory videos for independent testing, and we explore different data preprocessing and training configurations. We find that classic models suffer severe performance degradation on external data under our realistic setting with sparse scratching events. In contrast, fine-tuned video foundation models substantially improve the generalizability, achieving significantly better detection performance. Furthermore, frozen multimodal large language models demonstrate promising out-of-the-box ability using only few-shot demonstrations.We also report the effects of different video preprocessing strategies and ablations. Our findings highlight the improved performance and generalizability of the adapted foundation models for automated scratching behavior analysis and shed light on the real-world deployment of such detectors. Our code is publicly available at github.com/PennShenLab/scratch-aid-plus.
Institutions: University of Pennsylvania, Carnegie Mellon University
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7221595276