基于DQN的快速路合流区CAV换道决策方法

Decision method of CAV lane change in expressway merging area based on DQN

摘要:

为解决快速路合流区的交通拥堵和安全问题,保障智能网联车辆（connected and automated vehicle,CAV）在快速路合流区高效、安全、舒适、稳定地行驶,采用深度强化学习方法之DQN（deep q-network）算法,综合考虑车辆行驶安全性、高效性与舒适性等因素,建立了用于神经网络训练的奖励函数模型,提出了快速路合流区CAV换道决策方法。应用开源自动驾驶仿真场景highway-env,搭建快速路合流区仿真环境,对其主线路段及匝道进行仿真试验。仿真试验结果表明:相比于智能驾驶人模型（intelligent driver model,IDM）和highway-env中换道决策方法,采用所提出的快速路合流区CAV换道决策方法时,CAV能够快速达到22.22 m/s行驶的稳定状态,同时避免频繁换道及加减速行为,并优化了车头时距,很大程度上提高了交通流运行效率和乘车舒适性。研究成果在智能交通领域中,可为智能网联环境下城市快速路合流区的车辆通行提供了一种新方法,为未来智能网联车辆的换道提供了决策方法。

Abstract:

In order to tackle traffic congestion and safety issues in expressway merging areas and to ensure efficient, safe, comfortable, and stable travel of connected and automated vehicles (CAVs) in these areas, this study employs the DQN (deep q-network) algorithm from deep reinforcement learning. By considering factors such as vehicle safety, efficiency, and comfort, a reward function model for neural network training has been established, and a CAV lane-change decision-making method for merging areas has been proposed. Using the open-source highway-env simulation scenario, a simulation environment for expressway merging areas has been set up, and experiments have been conducted on the mainline and ramps. The results of the simulation experiments show that compared to the intelligent driver model (IDM) and the lane-change decision-making method in highway-env, the proposed CAV lane-change decision-making method enables CAVs to quickly reach a stable driving state at a speed of 22.22 m/s. It also reduces frequent lane changes and acceleration/deceleration behaviors, and optimizes the time-headway between vehicles. This significantly improves the efficiency of traffic flow and ride comfort. The research findings provide a new method for vehicle traffic management in urban expressway merging areas under intelligent networked conditions. They also offer a decision-making approach for lane changes in future connected and automated vehicles.