Hierarchical Multi-Agent Reinforcement Learning for Cooperative Jamming in Non-Stationary Adversarial Environments
Year: 2026
Type: article
Abstract: This work addresses cooperative jamming decision making in non-stationary adversarial environments with intelligent anti-jamming strategies. Existing approaches struggle to simultaneously handle the high-dimensional combinatorial optimization of heterogeneous jamming resources and the environmental dynamics induced by adaptive adversaries. To tackle these challenges, we propose a cooperative jamming decision making method based on Hierarchical Multi-Agent Parameterized Deep Q-Network (H-MAP-DQN). A three-layer hierarchical parameterized action space is constructed to decouple target selection, beam-frequency decisions, and continuous resource allocation, effectively mitigating the curse of dimensionality. Furthermore, a dual-head coordination architecture is designed to achieve implicit task allocation via intention prediction and competition coefficient mechanisms under the Decentralized Training Decentralized Execution (DTDE) paradigm. Addition ally, an adaptive prioritized experience replay mechanism is introduced to dynamically monitor non-stationarity intensity and adjust experience sampling weights, effectively addressing stale experiences in non-stationary adversarial environments of varying intensities. Simulation results in environments featuring pseudo-random frequency hopping and adaptive power bandwidth adjustment demonstrate that the proposed method achieves a 75.07% overall suppression rate and 96.59% high value target suppression rate, matching the performance of MASAC under the Centralized Training Decentralized Execution (CTDE) framework while significantly outperforming MAPPO, FD-CJDM,andD3QNbaselines. Moreover, against reinforcement learning-based adaptive anti-jamming opponents, the proposed method maintains stable suppression performance, validating the effectiveness and robustness of the hierarchical architecture, implicit coordination, and adaptive PER.
Institution: Space Engineering University
Language: English
Subfield: Aerospace Engineering
Field: Engineering
Domain: Physical Sciences
Open Access status: hybrid
Estimated APC paid: $2,645
OpenAlex ID: W7204459525