期刊检索

  • 2026年第58卷
  • 2025年第57卷
  • 2024年第56卷
  • 2023年第55卷
  • 2022年第54卷
  • 2021年第53卷
  • 2020年第52卷
  • 2019年第51卷
  • 2018年第50卷
  • 2017年第49卷
  • 2016年第48卷
  • 2015年第47卷
  • 2014年第46卷
  • 2013年第45卷
  • 2012年第44卷
  • 2011年第43卷
  • 2010年第42卷
  • 第1期
  • 第2期

主管单位 中华人民共和国工业和信息化部 主办单位 哈尔滨工业大学 主编 李隆球 国际刊号ISSN 0367-6234 国内刊号CN 23-1235/T

期刊网站二维码
微信公众号二维码
引用本文:高志云,唐艺宸,靳舒葳,梁宇玲.CM-MADDPG:面向多无人机博弈对抗的协同决策算法[J].哈尔滨工业大学学报,2026,58(8):173.DOI:10.11918/202510031
Gao Zhiyun,Tang Yichen,Jin Shuwei,Liang Yuling.CM-MADDPG:a cooperative decision-making algorithm for multi-unmanned aerial vehicle game confrontation[J].Journal of Harbin Institute of Technology,2026,58(8):173.DOI:10.11918/202510031
【打印本页】   【HTML】   【下载PDF全文】   【查看/发表评论】  【下载PDF阅读器】  【关闭】
过刊浏览    高级检索
本文已被:浏览 815次   下载 71次 本文二维码信息
码上扫一扫!
分享到: 微信 更多
CM-MADDPG:面向多无人机博弈对抗的协同决策算法
高志云1,2,唐艺宸2,靳舒葳2,梁宇玲3
(1.天津市城市空中交通系统技术与装备重点实验室(中国民航大学),天津 300300; 2.中国民航大学 交通科学与工程学院,天津 300300; 3.沈阳工业大学 人工智能学院,沈阳 110870)
摘要:
为解决多无人机博弈对抗中现有决策方法在复杂场景下奖励设计烦琐,提升算法训练稳定性与样本效率。将多智能体强化学习与元强化学习结合,提出评论家与元策略联合增强的多智能体深度确定性策略梯度(Critic and Meta enhanced multi-agent deep deterministic policy gradient,CM-MADDPG)算法。首先,通过在集中式评论家侧引入适度的结构与学习增强,提升跨机交互的价值估计精度并抑制过估计,从而使训练更稳定,稳态性能更高。其次,在训练期间采用元策略进行自适应调度(如动态调整损失权重与探索强度),提升样本效率与收敛速度,且不增加推理期开销。最后,在相同实验条件下,红方无人机分别采用多智能体深度确定性策略梯度(multi-agent deep deterministic policy gradient,MADDPG)、评论家增强的多智能体深度确定性策略梯度(Critic multi-agent deep deterministic policy gradient,Critic-MADDPG)、元策略增强的多智能体深度确定性策略梯度(Meta enhanced multi-agent deep deterministic policy gradient,Meta-MADDPG),以及CM-MADDPG与随机机动的蓝方无人机进行博弈对抗。仿真结果表明,CM-MADDPG在最终胜率、各无人机奖励,以及收敛速度等指标上均取得显著提升。证明CM-MADDPG在不增加推理计算开销的前提下成功缓解稀疏奖励与训练不稳定问题,提高收敛速度与样本效率。研究成果可为多无人机博弈理论提供一种新思路和途径。
关键词:  多无人机博弈对抗  多智能体深度确定性策略梯度  元策略增强  集中式评论家  多智能体强化学习  样本效率
DOI:10.11918/202510031
分类号:TP181;TP273
基金项目:国家自然科学基金(62403471);中央高校基本科研业务费(3122023QD18);天津市城市空中交通系统技术与装备重点实验室(TJKL-UAM-202402)
CM-MADDPG:a cooperative decision-making algorithm for multi-unmanned aerial vehicle game confrontation
Gao Zhiyun1,2,Tang Yichen2,Jin Shuwei2,Liang Yuling3
(1.Key Laboratory of Technology and Equipment of Tianjin Urban Air Transportation System (Civil Aviation University of China), Tianjin 300300, China; 2.College of Transportation Science and Engineering, Civil Aviation University of China, Tianjin 300300, China; 3.School of Artificial Intelligence, Shenyang University of Technology, Shenyang 110870, China)
Abstract:
To address the cumbersome reward design of existing decision-making methods in complex scenarios of multi-unmanned aerial vehicle game confrontation and to improve corresponding training stability and sample efficiency, this paper proposed a Critic and Meta enhanced multi-agent deep deterministic policy gradient (CM-MADDPG) algorithm by combining multi-agent reinforcement learning with Meta-reinforcement learning. First, this paper introduced appropriate structural and learning enhancements to the centralized Critic side to improve the value estimation accuracy of cross-unmanned aerial vehicle interaction and suppress overestimation, thereby making training more stable and achieving higher steady-state performance. Second, a Meta-strategy was adopted during the training period for adaptive scheduling (such as dynamically adjusting loss weights and exploration intensity) to improve sample efficiency and convergence speed without increasing inference-time overhead. Finally, under identical experimental conditions, red unmanned aerial vehicles respectively adopting multi-agent deep deterministic policy gradient (MADDPG), Critic multi-agent deep deterministic policy gradient (Critic-MADDPG), Meta enhanced multi-agent deep deterministic policy gradient (Meta-MADDPG), and CM-MADDPG conducted a game confrontation against blue unmanned aerial vehicles with random maneuvering. Simulation results show that CM-MADDPG achieves significant improvements in final win rate, rewards of each unmanned aerial vehicle, and convergence speed. This proves that CM-MADDPG successfully alleviates sparse reward and training instability issues without increasing inference computational overhead and improves convergence speed and sample efficiency. The research results of this paper provide a new idea and approach for theories of multi-unmanned aerial vehicle game confrontation.
Key words:  multi-unmanned aerial vehicle game confrontation  multi-agent deep deterministic policy gradient  Meta-strategy enhancement  centralized Critic  multi-agent reinforcement learning  sample efficiency

友情链接LINKS