Work

ENMA: Integrating Human Affect into Reinforcement Learning for Legged Locomotion

Year: 2026
Type: preprint
Abstract: As we introduce robots into increasingly more demanding social environments like office s and hospitals, a need for robots who not only avoid people, but also make them feel comfortable while interacting with them. Integrating affect into the gait of the robot is not sufficient as it does not iclude live reactions of the robot. Human in the loop reinforcement learning where humans annotate their reactions to robots behaviour fails as reinforcement learning needs hundreds or thousands steps to learn sufficiently, and human annotators simply do not have capacity to do so reliably and objectively. Thats where we introduce ENMA, a middleware sitting inbetween high level controller or teleoperator and a low level controller of the robot. It takes in the command of high level controller, robots state and positions and velocities of people around the robot and modulates the command of the high-level controller, based on a predicted affect metrics(valence/arousal) of the nearest person during training, before passing it to the low-level controller. These predicted affect metrics come from a lightweight 3D ResNet model called CAESAR, which we trained on videos of robots, animals and humans and pseudolabels generted by teacher pipeline consiting of Tribe V2 and Swift 4D transformer. ENMA itself is a policy using spiking neural networks, which have achieved similar benchmarks as classical ANNs, are more power efficient and better suited for processing of dynamic data, as they are more neuromorphic then their analog couterparts. We train ENMA in Isaac Lab where we simulate the quadrupedal robot Unitree Go2 interacting with a human avatar with a POV camera. We then let CAESAR annotate this feed online during training which helps us eliminate the need of human annotators in the reinforcement learning process. We validate our framework both in simulation and real-world deployment on a physical Unitree Go2 quadruped across four distinct human-robot interaction scenarios. Our results demonstrate that biologically grounded affective constraints can be successfully integrated into mid-level robot behavior to yield smooth, predictable, and socially appropriate navigation. Preprint, version 1 (November 2026). Video of the real-robot experiments: https://www.youtube.com/watch?v=ZvLrn8Owd1M
Language: English
Open Access status: green
OpenAlex ID: W7233576555
Cites:
Cited by:
Related to: