Year: 2025
Type: preprint
Abstract: Animal behavior unfolds continuously in time, yet quantitative analyses often require segmenting it into discrete, interpretable states. Although manual annotation can achieve this, it remains slow, subjective, and difficult to scale. Most automated pipelines use tracked body parts to infer actions, but are limited by tracking quality, and discard much of the visual information contained in raw videos. Here we present FERAL (Feature Extraction for Recognition of Animal Locomotion), a supervised video-understanding toolkit that bridges this gap by mapping raw video directly to frame-level behavioral labels, bypassing the need for pose estimation. Across benchmarks, FERAL outperforms state-of-the-art pose- and video-based baselines: on a benchmarking dataset of mouse social interaction, it surpasses Google's Videoprism using just a quarter of the training data. FERAL generalizes across species, recording conditions, and levels of behavioral organization: from single-animal locomotion to complex social interactions and emergent collective dynamics. Released as a user-friendly, open-source package, FERAL overcomes the challenges of traditional approaches, integrates easily with existing analysis pipelines, and can be deployed locally or on cloud servers with a few clicks. By mapping raw video directly to annotated behavior, FERAL lowers the barrier to scalable, cross-species behavioral quantification and broadens the range of behavioral analyses possible in both the lab and the wild.
Authors: Peter Skovorodnikov, Jacopo Razzauti, Blair R. Costelloe, Friederike Buck, Vikram Chandra, Dominic D. Frank, Tomas Kay, Benjamin Koger, Orli Snir, Janet Zhao, Iain D. Couzin, Daniel J.C. Kronauer, Leslie B. Vosshall
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
Funders: Nvidia, European Commission, Boehringer Ingelheim Fonds, Deutsche Forschungsgemeinschaft, Gottfried Wilhelm Leibniz Universität Hannover, Universität Konstanz, National Institutes of Health, Weill Cornell Medical College, Leibniz-Gemeinschaft, National Institute on Deafness and Other Communication Disorders, National Institute of General Medical Sciences, HORIZON EUROPE European Innovation Council
Awards: G203253094, G2038046918, Quantum Artificial Neural Networks with Tensor Network Algorithmus, G2088408388, G2381975105, G2463333092, G2600198664, Advanced simulation, analysis and interpretation of network structures in biological data, BABOTS: The design and control of small swarming biological animal robots, G434337789, Neuronal Control of Motor State Transitions, System zur umfassenden Verhaltensanalyse von Fischgruppen, Weill Cornell/Rockefeller/Sloan Kettering MST Program, Neurodevelopmental adaptations prime olfactory circuits for complex pheromonal communication in the clonal raider ant.
OpenAlex ID: W4416285092