Get started
Work

SKYWALKER:Dual-Agent Reinforcement Learning for Autonomous Crawling in a Planar Microgravity Analog Sim-to-Real Evaluation at ESA’s Orbital Robotics Lab

Year: 2025
Type: article
Abstract: Autonomous robotic assembly in space requires robots that can move and reattach themselves in microgravity without human intervention. We present SKYWALKER, a proof- of-concept dual-agent reinforcement learning system enabling a free-floating robot to “crawl” between custom made anchor points. Our approach divides the task between two cooperating reinforcement learning (RL) agents: one controlling a 7 degrees of freedom (DOF) manipulator to dock at anchors, and another repositioning the robot’s base by pushing off the grasped anchor. Trained entirely in a planar frictionless simulation (analogous to microgravity) using Proximal Policy Optimization (PPO), the agents learned to coordinate and achieve sequential anchor traversal. We deployed the learned arm policy on a physical air-bearing robot at the European Space Agency’s Orbital Robotics Lab, where it successfully performed autonomous anchor docking. Initially, the base-moving policy exhibited unsafe upward forces and was replaced with a conservative scripted motion to ensure system safety during demonstration. However, in follow-up laboratory trials at Aalborg University, we refined the training rewards and applied stricter safety thresholds. Under these updated conditions, the learned base policy successfully executed autonomous base translations, completing the dual-agent architecture in hardware. This result confirms the feasibility of fully learned control for free-floating crawling behaviors, with both agents transferring from simulation to the real system. These results mark an encouraging step toward RL-driven space robotics. We discuss the challenges encountered in transferring learned behaviors to real hardware, especially for dynamic base motions, safety verification, and scaling to full 3D on-structure locomotion along spacecraft surfaces. This early-stage work highlights both the potential of AI-based control for on-orbit assembly and the critical need for robust validation techniques when applying learning in space environments. The success of the updated base- motion policy following safety-driven refinement reinforces that careful constraint design can unlock safe and reliable deployment of reinforcement learning systems in space robotics.
Authors: Davidsson, Dadi Hrannar; id_orcid 0009-0001-5173-1036, De Salas, Bruno Miguelez, Gadensgaard Svendsen, Glenn, Dueholm Kristiansen, Rasmus, Wang Hansen, Gustav, Melnyk, Artem, Eddin Kawara, Alaa, Bork Nielsen, Thor, Argjaboða, Jón Collis Á, Bøgh, Simon; id_orcid 0000-0002-5960-4365, Mortensen, Anton Bjørndahl
Language: English
Open Access status: green
OpenAlex ID: W7134220459
FWCI:
Cites:
Cited by:
Related to: