CM-MADDPG:a cooperative decision-making algorithm for multi-unmanned aerial vehicle game confrontation
CSTR:
Author:
Affiliation:

(1.Key Laboratory of Technology and Equipment of Tianjin Urban Air Transportation System (Civil Aviation University of China), Tianjin 300300, China; 2.College of Transportation Science and Engineering, Civil Aviation University of China, Tianjin 300300, China; 3.School of Artificial Intelligence, Shenyang University of Technology, Shenyang 110870, China)

Clc Number:

TP181;TP273

Fund Project:

undefined

  • Article
  • |
  • Figures
  • |
  • Metrics
  • |
  • Reference
  • |
  • Related
  • |
  • Cited by
  • |
  • Materials
  • |
  • Comments
    Abstract:

    To address the cumbersome reward design of existing decision-making methods in complex scenarios of multi-unmanned aerial vehicle game confrontation and to improve corresponding training stability and sample efficiency, this paper proposed a Critic and Meta enhanced multi-agent deep deterministic policy gradient (CM-MADDPG) algorithm by combining multi-agent reinforcement learning with Meta-reinforcement learning. First, this paper introduced appropriate structural and learning enhancements to the centralized Critic side to improve the value estimation accuracy of cross-unmanned aerial vehicle interaction and suppress overestimation, thereby making training more stable and achieving higher steady-state performance. Second, a Meta-strategy was adopted during the training period for adaptive scheduling (such as dynamically adjusting loss weights and exploration intensity) to improve sample efficiency and convergence speed without increasing inference-time overhead. Finally, under identical experimental conditions, red unmanned aerial vehicles respectively adopting multi-agent deep deterministic policy gradient (MADDPG), Critic multi-agent deep deterministic policy gradient (Critic-MADDPG), Meta enhanced multi-agent deep deterministic policy gradient (Meta-MADDPG), and CM-MADDPG conducted a game confrontation against blue unmanned aerial vehicles with random maneuvering. Simulation results show that CM-MADDPG achieves significant improvements in final win rate, rewards of each unmanned aerial vehicle, and convergence speed. This proves that CM-MADDPG successfully alleviates sparse reward and training instability issues without increasing inference computational overhead and improves convergence speed and sample efficiency. The research results of this paper provide a new idea and approach for theories of multi-unmanned aerial vehicle game confrontation.

    Reference
    Related
    Cited by
Get Citation
Related Videos

Article Metrics
  • Abstract:
  • PDF:
  • HTML:
  • Cited by:
History
  • Received:October 14,2025
  • Revised:
  • Adopted:
  • Online: August 29,2026
  • Published:
Article QR Code