Year: 2026
Type: preprint
Abstract: Markerless pose estimation has emerged as a powerful technique for animal behavior quantification, capable of high resolution tracking of body parts. Many neuroscience labs rely on tools like DeepLabCut and SLEAP, which provide accessible interfaces but restrict users to a narrow set of models and configurations. In this work, we adopt MMPose an open source, general-purpose computer vision library to build a workflow for training and evaluating multiple state-of-the-art models on animal video datasets. We benchmark these models in two scenarios: (1) a complex maze assay with occlusions and varied backgrounds, and (2) a simpler open field arena with a high-contrast background. Our results show that a bottom-up model (DEKR) delivers the highest accuracy in the complex task, whereas lighter-weight models (e.g., SLEAP) offer superior speed highlighting a clear trade-off between accuracy and throughput. We also evaluate a recently published foundation model (TopViewMouse-5K) trained on a large top-view mouse dataset to test its generalization. It performs poorly on our tasks at zero-shot, and even when we combine its data with our training set, we observe no consistent benefit. These findings emphasize the importance of context-specific model selection and the need for more diverse training data to create generalizable pose models. By leveraging a general-purpose vision library, researchers can flexibly choose models that best suit their experimental needs. This work illustrates how adopting advanced computer vision frameworks can accelerate behavioral neuroscience and genetics research, paving the way for more scalable, reproducible, and sensitive analysis of animal behavior.
Authors: Jessica D. Choi, Vivek Kumar
Institutions: Tufts University, Jackson Laboratory
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
Awards: G2630913556, Sequencing Mutant Mice With Altered Cocaine Responses, Dissection of Addiction Relevant Signal Integration by Cyfip2 through Precise Genome Engineering, G386266877, G4417668518, Machine learning based frailty index for the genetically diverse mice, Training Program in Precision Genetics of Aging, Alzheimer's Disease and Related Dementias, G5320618627, G5505655183, G6149589070, Application of Machine Vision to Determine the Influence of Sleep States and Social Interactions on Vulnerability to Drug Addiction, G854360585, G9000226295, G9000229113, G9000259377, G9000274825, Cross Scale Interrogation of NPD Genes (SING) in mice, Establishment and Characterization of Novel Mutant Mouse Models for the Addiction Research Community, Nathan Shock Center for Excellence in Basic Biology of Aging, Machine learning based frailty index for the genetically diverse mice
OpenAlex ID: W7147571053