Abstract:The optimization of customized bus route with multiple boarding sites and alighting sites considering heterogeneous fleet is studied. Aiming at minimizing passenger trip cost and bus operating cost, the problem is formulated as a mathematical programming model where some constraints indicating the boarding and alighting mode, site number, the length of bus route, the number of customized buses are considered. The traditional methods are not efficient in solving large-scale mathematical models. So a deep reinforcement learning procedure based on Actor-Critic dual neural network is proposed. Here the customized bus is regarded as agent. The learning environment of the agent is established by the unsupervised clustering approach. And then the state space-action space-reward function of the agent is presented and a Actor-Critic double neural network is given to select and evaluate boarding sites and alighting site. So a deep reinforcement learning algorithm based on Actor-Critic dual neural network is obtained. The coordinate data of boarding-alighting sites and resident trip is collected together to set up testing experiment. These hyper-parameters, which is composed of learning rate, discount factor, batch size and number of neurons, are debugged by New Actor neural network and Old Actor neural network. The customized bus route scheme based on multiple boarding sites and alighting sites can be obtained by the proposed deep reinforcement learning algorithm. The performance measures, which consist of the layout of boarding and alighting sites, customized bus route distance, bus occupancy rate and bus type allocation, verify the rationality of the customized bus route scheme. The deep reinforcement learning algorithm based on single Actor neural network is used as a comparison testing. The convergence speed of proposed algorithm is improved by 28.57% and the reward value is increased by 13.65%. Furthermore, compared with the heuristic algorithms, this paper also verifies that the proposed method can quickly and efficiently complete the model solution, and reasonably determine the location of the customized bus stops and the specific route direction.