Transformer-Enhanced Physics Informed Deep Reinforcement Learning for Quadrotor Navigation
Year: 2026
Type: article
Abstract: Unmanned Aerial Vehicles (UAVs) are increasingly deployed across civilian and military domains due to their mobility, flexibility, and low energy consumption. Traditional UAV path-planning and navigation techniques often suffer from high computational cost, limited adaptability, and suboptimal path generation, motivating the use of learning-based solutions. Deep Reinforcement Learning (DRL) enables UAVs to learn optimal navigation strategies directly from interactions with the environment, handling continuous and high-dimensional action spaces typical of UAV dynamics. However, conventional DRL methods struggle to capture long-term temporal dependencies. This work presents a goal-conditioned DRL framework for multirotor navigation toward a target destination, enhanced with a Transformer-encoder and a Long Short-Term Memory (LSTM) layer. Particularly, both architectures address the DRL limitations by modeling long-range dependencies through a self-attention mechanism and preserving temporal dependencies. The framework integrates two stable, probabilistic DRL algorithms, PPO and A2C. Compared with their standalone counterparts, the Transformer-enhanced architectures improve validation stability and trajectory consistency, with the most pronounced improvement observed for PPO. Furthermore, a novel reward function for continuous action spaces is proposed, combining multiple contributions with a physics-inspired term based on the Least Action Principle (LAP), which encourages efficient and dynamically consistent flight. A simulation pipeline is developed to evaluate the approach. Training is first performed on a quadrotor dynamic model, followed by Software-in-the-Loop (SIL) on PX4 and Gazebo validation and Hardware-in-the-Loop (HIL) deployment on a Raspberry-Pi (RPI). The proposed pipeline is designed to facilitate model prototyping while supporting a structured progression toward real-world flight testing. The results obtained from the simulated experiments show that the Transformer-enhanced architecture improves trajectory robustness compared to baseline DRL policies, while the proposed reward formulation promotes smoother and more action-consistent flight behavior. These findings demonstrate the effectiveness of combining transformer-based temporal modeling with physics-informed reward design for autonomous UAV navigation.
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
Estimated APC paid: $3,090
OpenAlex ID: W7225810875
FWCI:
Cites:
Cited by:
Related to: