Work

Guidance for Closed-Loop Transfers using Reinforcement Learning with Application to Libration Point Orbits

Year: 2020

Type: conference-paper

Abstract: While human presence in cislunar space continues to expand, so too does the demand for ‘lightweight’ automated on-board processes. In nonlinear dynamical environments, computationally efficient guidance strategies are challenging. Many traditional approaches rely on either simplifying assumptions in the dynamical model or abundant computational resources. The proposed controller employs the use of the nonlinear equations of motion without imposing a heavy workload on a flight computer. The guidance framework is nevertheless able to leverage high-performance computing by separating the training from the resulting controller. Practical examples demonstrate the flexibility of a reinforcement learning approach, and suggest extendability to higher-fidelity domains.

Source: AIAA SciTech Forum

DOI: 10.2514/6.2020-0458

Authors: Nicholas B. LaFarge, Daniel Miller, Kathleen C. Howell, Richard Linares

Institutions: Purdue University West Lafayette, Massachusetts Institute of Technology

Language: English

Topic: Spacecraft Dynamics and Control

Subfield: Aerospace Engineering

Field: Engineering

Domain: Physical Sciences

Open Access status: green

Funders: National Aeronautics and Space Administration, Purdue University, Massachusetts Institute of Technology

Award: AT THE PRESENT TIME DEEP-SPACE MISSIONS ARE PLANNED IN THE SAME WAY AS THEY HAVE BEEN FOR DECADES: USING BRUTE FORCE COMPUTATIONS OVER MANY MISSION PARAMETERS. SUCH CONSIDERATIONS INCLUDE THE MASS THRUST AND POWER GENERATION CAPABILITIES OF A PROBE AND/OR THE DESTINATIONS THAT MAKE UP ITS JOURNEY. NEW TECHNOLOGIES HOWEVER ARE POSED TO DRAMATICALLY ENHANCE THIS PROCESS. ADVANCES IN REINFORCEMENT LEARNING (RL) AND DEEP NEURAL NETWORKS OFFER PERFORMANCE THAT CAN MATCH OR EVEN SURPASS THAT OF CURRENT STATE-OF-THEART METHODS IN COMPLEX TASKS WITH LARGE DESIGN SPACES. THIS PROPOSAL SEEKS TO APPLY THIS TECHNOLOGY TO THE GENERATION AND CONTROL OF ENERGY EFFICIENT LOW-THRUST SPACECRAFT TRAJECTORIES. THE PROPOSED WORK WILL USE AN ACTOR-CRITIC BASED POLICY APPROACH TO EXPLORE TRAJECTORY PROBLEMS. THE INNOVATION IN THIS METHOD IS THAT THE CRITIC WILL USE CONCEPTS FROM TRANSFER LEARNING TO GENERALIZE CONTROL PROBLEMS OVER MANY APPLICATIONS. THE LEARNING AGENT WILL THEREFORE BE ABLE TO LEARN THE FEATURES OF AN OPTIMIZED TRAJECTORY FROM THOSE THAT IT HAD PREVIOUSLY GENERATED AND THEN USE THIS KNOWLEDGE TO DISCOVER NEW TRAJECTORY SOLUTIONS TO OTHER PROBLEMS. THIS WORK WILL ALSO EXPLORE AN ACTOR-CRITIC FRAMEWORK FOR CONTINUOUS ACTION PROBLEMS WHERE CONTINUOUS CONTROL IS DESIRED AND FOR PROBLEMS WITH DYNAMIC CONSTRAINTS. BY APPLYING RL TO THE DESIGN OF INTERPLANETARY MISSIONS FUTURE SPACECRAFT WILL BE ABLE TO MORE EASILY FOLLOW LOW-THRUST TRAJECTORIES AND TAKE ADVANTAGE OF MULTI-BODY DYNAMICS INSTEAD OF THE TRADITIONAL HOHMANN TRANSFER. NON-STANDARD ORBITS ABOUT LIBRATION POINTS WOULD ALSO BE MADE MORE AVAILABLE. FURTHERMORE BY NOT REQUIRING LARGE COMPUTATIONAL RESOURCES THIS TECHNOLOGY COULD BE APPLIED TO FUTURE MANNED MISSIONS FOR ONBOARD MISSION PLANNING. THE INCREASED FREEDOM IN MISSION DESIGN AFFORDED BY THIS TECHNOLOGY WOULD OFFER SIGNIFICANT BENEFITS TO BOTH NASA AND OTHER SPACEFLIGHT PROGRAMS