Abstract:To achieve the escape of a hypersonic vehicle from an interceptor in a multi-role game scenario of “target-interceptor-defender”, it is necessary to execute a cooperative maneuver strategy with the defender. However, due to the limitations of the detection device, hypersonic vehicles face the problem of cooperative maneuver decision-making with imperfect, incomplete, and intermittent strong information constraints. To address this, this paper proposed an end-to-end cooperative maneuver decision-making approach by integrating a multi-agent deep reinforcement learning algorithm, enabling hypersonic vehicles to make cooperative maneuver decisions under strong information constraints and achieve successful evasion. First, the research scenario was modeled as a decentralized partially observable Markov decision process, and an observation information sharing stacking mechanism was proposed for the design of local observation state spaces under the strong information constraints. Second, to address the sparse reward problem in multi-agent deep reinforcement learning, a cooperative decision-making reward function was constructed by integrating game relationships and zero-effort miss distance, enhancing training efficiency in complex game scenarios. Finally, a multi-agent cooperative decision-making network architecture was designed, comprising the agents basic networks and the top value decomposition network. This architecture extracted spatio-temporal trajectory features from imperfect, incomplete, and intermittent information, enabling policy coordination among agents and cooperative maneuver decision-making for vehicles. Research results demonstrate that hypersonic vehicles equipped with the proposed intelligent cooperative maneuver decision-making approach can successfully evade in multi-role game scenarios under strong information constraints. The proposed approach exhibits outstanding performance and robustness in numerical simulations, including typical game scenarios and Monte Carlo tests.
