SoC-VRP: A Deep-Reinforcement-Learning-Based Vehicle Route Planning Mechanism for Service-Oriented Cooperative ITS

Abstract

With the rapid development of emerging information technology and its increasing integration with transportation systems, the Intelligent Transportation System (ITS) is entering a new phase, called Cooperative ITS (C-ITS). It offers promising solutions to numerous challenges in traditional transportation systems, among which the Vehicle Routing Problem (VRP) is a significant concern addressed in this work. Considering the varying urgency levels of different vehicles and their different traveling constraints in the Service-oriented Cooperative ITS (SoC-ITS) framework studied in our previous research, the Service-oriented Cooperative Vehicle Routing Problem (SoC-VRP) is firstly analyzed, in which cooperative planning and vehicle urgency degrees are two vital factors. After examining the characteristics of both VRP and SoC-VRP, a Deep Reinforcement Learning (DRL)-based prioritized route planning mechanism is proposed. Specifically, we establish a deep reinforcement learning model with Rainbow DQN and devise a prioritized successive decision-making route planning method for SoC-ITS, where vehicle urgency degrees are mapped to three priorities: High for emergency vehicles, Medium for shuttle buses, and Low for the rest. All proposed models and methods are implemented, trained using various scenarios on typical road networks, and verified with SUMO-based scenes. Experimental results demonstrate the effectiveness of this hybrid prioritized route planning mechanism.

Navigation Route Planning for Tourism Intelligent Connected Vehicle Based on the Symmetrical Spatial Clustering and Improved Fruit Fly Optimization Algorithm

“…Representative studies on the ICV route problems and intelligent tourism transportation route-planning mainly include the following: Hou et al [5] proposed a route-planning mechanism based on deep reinforcement learning (DRL). A deep reinforcement learning model was established using Rainbow DQN, and the prioritized successive decisionmaking route-planning method was designed.…”

“…The gaps between our proposed model and related work, as well as the advantages of the proposed work, are mainly reflected in the following aspects: Reference [5] focuses on constructing the deep learning algorithm to achieve priority sorting of vehicles of different levels, without considering the fairness of tourist services. Our method reflects the fairness of all ICVs and ICV routes, allowing each tourist to enjoy the same POI recommendations and ICV route planning services.…”

“…This binary tree is the spatial decision tree based on the feature attribute recommendation, denoted as Subfigure (1) shows the first step of the algorithm: generating the natural attribute labels; . The maximum value is stored at the root node, the second maximum value is stored at the leftmost child node of the lower level, and the minimum value is stored at the rightmost child node of the lower level; Subfigure (5) shows the fifth step of the algorithm: comparing the recommendation degrees of all POIs. The larger value is stored in the upper level or left child node, and the smaller value is stored in the lower level or right child node, so that the recommendation degree of any upper-level child node is greater than that of the lower level, and the recommendation degree of any left child node in any level is larger than that of the right child node; Subfigure (6) shows the sixth step of the algorithm: constructing the second decision tree; Subfigure (7) shows the seventh step of the algorithm: constructing the decision trees for all the natural attributes to form a decision forest.…”

“…Node n sub (7) is found, then a feasible path is formed, keep n sub (7) , as shown in the Figure 9(4). (5) Step 4: Select n sub (8) or n sub (11) . Node n sub (11) is found, then a feasible path is formed, keep n sub (11) , as shown in the Figure 9(5).…”

“…To meet the matching algorithm between the tourist interests and POI feature attributes in POI recommendation, the demands that the tourist inputs into the ICV decision-making system should meet the following conditions: ⃝ Collect and determine all the ICV moving sub-intervals, and determine the starting and ending points of each sub-interval; 2 ⃝ Determine two control nodes for the ICV moving sub-interval, with each control node being an ICV transfer station or a POI; 3 ⃝ Obtain all the connected roads between two control nodes in the city map, which can directly or indirectly connect the two control points; 4 ⃝ Search for the road intersections from the starting control point to the ending control point. Encode each intersection when it is searched, and store it in the sub-interval node database; 5 ⃝ Repeat the same searching for all sub-intervals; 6 ⃝ Search for the distance between adjacent nodes in the city map and store it in the sub-interval node distance database. We use the natural attribute classifications of POIs shown in Table 1 and the tourist requirements shown in Table 2, combined with the quantitative feature attributes of POIs, and calculate the POI feature attribute recommendation degrees ζ(f T(i) , f P(i) ).…”