A Simplified Event-based Impulsive Control Approach for Stable, Efficient, and Robust Locomotion Using Deep Reinforcement Learning
Year: 2024
Type: article
Abstract: Biological evidence indicates that the actuation system in humans and legged animals is characterized by impulsiveness rather than continuity; i.e., control actions are concentrated within a specific phase of the motion cycle (the stance phase), while the rest of the cycle is passive. Based on this observation, we propose a simple event-based impulsive controller to generate walking cycles for legged robots. To improve optimization speed, we parametrize the controller-applied forces as a Gaussian function of time and employ a deep reinforcement learning method to optimize the controller parameters. To further enhance learning speed, an autoencoder is utilized to address the high dimensionality in the state space. Additionally, we employ a three-phase reward-shaping approach to further improve learning speed and achieve better results. In phase one, the reward function focuses on stability and forward motion to learn stable locomotion. In phase two, the reward function is modified to achieve stable locomotion with lower control effort and desired forward velocity. In phase three, the reward function remains the same as in phase two but places more emphasis on forward velocity regulation. The proposed controller, state encoder, and learning process can be implemented on a group of legged robots with actuation at the leg contact point with the ground. In this paper, the proposed approach is tested on a simulated single-legged robot. In addition, the controller robustness is analyzed considering different types of external disturbances. The simulation results indicate the efficacy of the proposed method as a bio-inspired control approach for legged locomotion.
Authors: Bahareh Sadat Mortazavi, Rezvan Nasiri, Majid Nili Ahmadabadi
Institution: University of Tehran
Language: English
Subfield: Biomedical Engineering
Field: Engineering
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7084056299
FWCI:
Cites:
Cited by:
Related to: