Abstract:To address the cumbersome reward design of existing decision-making methods in complex scenarios of multi-unmanned aerial vehicle game confrontation and to improve corresponding training stability and sample efficiency, this paper proposed a Critic and Meta enhanced multi-agent deep deterministic policy gradient (CM-MADDPG) algorithm by combining multi-agent reinforcement learning with Meta-reinforcement learning. First, this paper introduced appropriate structural and learning enhancements to the centralized Critic side to improve the value estimation accuracy of cross-unmanned aerial vehicle interaction and suppress overestimation, thereby making training more stable and achieving higher steady-state performance. Second, a Meta-strategy was adopted during the training period for adaptive scheduling (such as dynamically adjusting loss weights and exploration intensity) to improve sample efficiency and convergence speed without increasing inference-time overhead. Finally, under identical experimental conditions, red unmanned aerial vehicles respectively adopting multi-agent deep deterministic policy gradient (MADDPG), Critic multi-agent deep deterministic policy gradient (Critic-MADDPG), Meta enhanced multi-agent deep deterministic policy gradient (Meta-MADDPG), and CM-MADDPG conducted a game confrontation against blue unmanned aerial vehicles with random maneuvering. Simulation results show that CM-MADDPG achieves significant improvements in final win rate, rewards of each unmanned aerial vehicle, and convergence speed. This proves that CM-MADDPG successfully alleviates sparse reward and training instability issues without increasing inference computational overhead and improves convergence speed and sample efficiency. The research results of this paper provide a new idea and approach for theories of multi-unmanned aerial vehicle game confrontation.