TRAFFIC LIGHT CONTROL METHOD FOR URBAN ROAD NETWORK BASED ON EXPECTED RETURN ESTIMATION

Abstract

The present application discloses a traffic light control method for an urban road network based on expected return estimation, which uses C-V2X wireless communication technology to obtain real-time information of all vehicles and traffic state in the road network from vehicle-mounted terminals, and adaptively and dynamically controls the phase transformation of the traffic light. According to the present application, the expected returns of keeping the current phase and executing phase switch are calculated by estimating the timely driving distance and the future driving distance of the passable vehicles in the next green light duration in combination with the proposed road priority traffic index. By comparing the expected returns of keeping the current phase or switching to other phases, the best phase is selected, so as to make as many passable vehicles travel farther as possible in the next green light duration. Therefore, the efficiency of traffic will be improved.

Claims

1. A traffic light control method for an urban road network based on expected return estimation, comprising: step 1, obtaining road information of the urban road network, comprising connectivity relation of all roads and current traffic light information of intersections, wherein it is assumed that each road comprises lanes of three directions: turning left, going straight or turning right; wherein a traffic light at each intersection comprises four phases: phase 1: turn left on a South-North direction, phase 2: go straight on the South-North direction, phase 3: turn left on a West-East direction, phase 4: go straight on the West-East direction; and wherein the road information comprises: a length of a road, it is assumed that maximum speed limits of all roads are the same, and a distance between a tail of a current road fleet and an upstream intersection; step 2, obtaining information of all vehicles in the urban road network from vehicle-mounted terminals by Cellular Vehicle Networking (Cellular-V2X or C-V2X) wireless communication technology, comprising an instantaneous speed of a vehicle and a position on the road expressed as a distance from a last intersection; and step 3, obtaining current phase information for each intersection in the urban road network, calculating a total expected return of all incoming lanes that keep a current phase in a next traffic light cycle and a maximum total expected return of all incoming lanes that switch to the other three phases, and selecting an optimal phase after comparison; when an executed phase in the next traffic light cycle is the same as the current phase, a green light duration of the vehicle being T; and when the executed phase in the next traffic light cycle is different from the current phase, the green light duration of the vehicle being T−t, where t is a red light duration when a phase is switched; wherein calculating the total expected return of all incoming lanes comprises: (3.1) multiplying by a road priority index, a sum of a timely driving distance of a vehicle in a lane and a future driving distance of the vehicle, as an expected return of each incoming lane, and summing expected returns of all incoming lanes as a total expected return of a certain phase; wherein calculating the timely driving distance of the vehicle comprises: calculating a distance and time that the vehicle needs to travel to reach an intersection according to a driving speed of the vehicle, an acceleration of the vehicle, a maximum speed limit of the road, a length of a road and a distance from the upstream intersection; and calculating, for all vehicles capable of passing through the intersection, a driving distance of the vehicle within the green light duration; (3.2) adding a distance that the vehicle needs to travel to reach the intersection calculated in step (3.1) to a length of a road of an outgoing lane and subtracted a queue length of the outgoing lane corresponding to a direction of turning left, going straight or turning right; determining whether an obtained result is less than the driving distance of the vehicle within the green light duration; if not, the timely driving distance of the vehicle being the driving distance of the vehicle within the green light duration, and the future driving distance of the vehicle being 0; and if yes, the timely driving distance of the vehicle and the future driving distance of the vehicle being calculated as follows:
drive.sub.distance-f=d+L.sub.2−q.sub.f ${\mathrm{future}}_{\mathrm{dist}ance-f}=\alpha *p*\frac{1}{L_2-q_f}*\left(D_T-\left(d+L_2-q_f\right)\right)$ where drive.sub.distance-f represents the timely driving distance of the vehicle in the outgoing lane, future.sub.distance-f represents the future driving distance of the vehicle in the outgoing lane, wherein the outgoing lane comprises lanes in three directions: turning left, going straight or turning right, which is represented by f, and the vehicle enter one of the lanes with a certain probability. q.sub.f represents the queue length of the outgoing lane; d is the distance that the vehicle still needs to travel to reach the intersection, L.sub.2 is the road length of the outgoing lane, and D.sub.T is the driving distance of the vehicle within the green light duration; p is a probability that an outgoing lane which the vehicle enters in at a downstream intersection is under the green light, and a is a loss coefficient of the future driving distance, which is an empirical coefficient; and (3.3) multiplying the timely driving distance and future driving distance of the vehicle in three directions of turning left, going straight or turning right calculated in step (3.2), by a probability that the vehicle turns left, goes straight or turns right, respectively, and calculating a sum of probabilities that the vehicle turns left, goes straight and turns right, obtain timely driving distances and future driving distances of all vehicles capable of passing through the intersection.

2. The traffic light control method for an urban road network based on expected return estimation according to claim 1, wherein in step 1, each intersection comprises a north-south dual-direction lane and an east-west dual-direction lane, wherein the intersection has traffic lights, and the traffic lights comprise a green light for allowing passing and a red light for forbidding passing.

3. The traffic light control method for an urban road network based on expected return estimation according to claim 1, wherein each phase comprises an incoming lane and three outgoing lanes, and the outgoing lanes comprise directions of turning left, going straight or turning right, and a vehicle to turn right is not controlled by the traffic lights and is capable of turning right at any time.

4. The traffic light control method for an urban road network based on expected return estimation according to claim 1, wherein in step (3.1), the time that the vehicle needs to travel to reach the intersection is calculated as follows: ${\mathrm{remain}}_{time}=\{\begin{array}{cc}\frac{-v+\sqrt{v^2+2a*d}}{\alpha },& \frac{V^2-v^2}{2*a}\ge d\\ \frac{V-v}{a}+\frac{d}{V}-\frac{V^2-v^2}{2*a*v},& \frac{V^2-v^2}{2*a}<d\end{array}$ where remain.sub.time represents the time that the vehicle needs to travel to reach the intersection, v is the driving speed, a is the acceleration, and V is a maximum speed limit of a road where the vehicle is located; and wherein when remain.sub.time is less than the green light duration, the vehicle is capable of passing the current intersection.

5. The traffic light control method for an urban road network based on expected return estimation according to claim 4, wherein in step (3.1), the driving distance of the vehicle within the green light duration is calculated as follows: $D_T=\{\begin{array}{cc}v*T^{\prime }+\frac{a*T^{\prime 2}}{2},& \frac{V-v}{a}\ge T^{\prime }\\ T^{\prime }*v+\frac{{\left(V-v\right)}^2}{2*a},& \frac{V-v}{a}<T^{\prime }\end{array}$ where D.sub.T represents the vehicle within the green light duration, T′ with a value of T or T−t, represents the green light duration.

6. The traffic light control method for an urban road network based on expected return estimation according to claim 1, wherein in step (3.1), the road priority index is calculated as follows:
priority.sub.factor=normal(queue.sub.length)*normal(avg_delay)*normal(avg_travel.sub.time) where priority.sub.factor represents the road priority index, queue.sub.length represents a queue length of the incoming lane, the queue length of the incoming lane is the total number of a vehicle with a speed less than 0.01m/s, normal represents a dimensionless treatment of three factors by a Min-Max method, avg_travel.sub.time represents an average driving time of all vehicles in the incoming lane, and avg_delay represents an average delay of the vehicles in the incoming lane, and the avg_delay is calculated as follows:
avg_delay=1−avg_speed/speed_limit where avg_speed represents an average speed of all vehicles in the incoming lane, and speed_limit is a maximum speed limit in the incoming lane.

7. The traffic light control method for an urban road network based on expected return estimation according to claim 1, wherein in step (3.2), the probability p that a lane of a direction which vehicle enters in at the downstream intersection is under the green light is calculated as follows: $p=\frac{1}{4}*\left(1+\frac{t}{T}\right).$

8. The traffic light control method for an urban road network based on expected return estimation according to claim 1, wherein in step (3.3), a probability that the vehicle turns left, goes straight or turns right is p.sub.1, p.sub.2, p.sub.3, respectively, and a sum of p.sub.1, p.sub.2, p.sub.3 is 1.

9. The traffic light control method for an urban road network based on expected return estimation according to claim 1, wherein in step (3), based on an estimated total expected return of keeping the current phase and an estimated maximum total expected return of phase switching, when the maximum return of phase switching is a multiple β of an expected return of keeping the current phase, the current phase is switched to a phase with the maximum total return of phase switching, and otherwise, the current phase is kept, where β is an empirical value.

Description

BRIEF DESCRIPTION OF DRAWINGS

[0026] FIG. 1 is a flow chart of a traffic light control method based on expected return estimation;

[0027] FIG. 2 is a schematic diagram of a traffic signal phase at an intersection;

[0028] FIG. 3 is the simulation visualization interface of a CBEngine traffic simulation engine; and

[0029] FIG. 4 is an intersection (circled intersection) in a CBEngine traffic simulation engine.

DETAILED DESCRIPTION OF THE INVENTION

[0030] The present application will be described in detail with reference to the attached drawings.

[0031] As shown in FIG. 1, the present application provides a traffic light control method for an urban road network based on expected return estimation, which includes the following steps.

[0032] Step 1: An intersection of urban roads is defined, including north-south dual-direction lanes and east-west dual-direction lanes. There are traffic lights at the intersection, including a green light and a red light. The green light allows passing, but the red light does not. The traffic lights at each intersection include four phases: phase 1: turn left on South-North direction, phase 2: go straight on South-North direction, phase 3: turn left on West-East direction, phase 4: go straight on West-East direction. Vehicles turning right are not controlled by traffic lights and can turn right at any time. Each phase includes an incoming lane and three outgoing lanes, such as phase 1. The incoming lane is two lanes numbered 1 and 2 in FIG. 2, and the outgoing lanes are six lanes numbered 3, 4, 5, 6, 7 and 8 in FIG. 2, respectively. Vehicles entering the lane in the direction of turning left in the north (the lane numbered 2 in FIG. 2) can enter the lanes corresponding to the outgoing lanes in three directions (numbered 2 in FIG. 2) of turning left, going straight and turning right (lanes numbered 6, 7 and 8 in FIG. 2). The duration of a signal cycle is T (unit, s), that is, the duration of a phase green light is T. If the phases are switched, there is a red light duration of t (unit, s), and vehicles in all phases are impassable, then the duration of the switched phase green light is T−t. The value of T herein is 30 s.

[0033] Step 2: The road information of the urban road network is obtained. The road information includes the connectivity of all roads and the current phase information of each intersection, assuming that each road includes lanes of three directions: turning left, going straight or turning right. The road information includes the length of the road, the maximum speed limit of the road (assuming that the maximum speed limits of all roads are the same), and the distance between the tail of the current road fleet and the upstream intersection.

[0034] Step 3: Using C-V2X(Cellular-V2X) wireless communication technology, the information of all vehicles in the road network is obtained from the vehicle-mounted terminal, including the position of vehicles on the road, which is expressed as the distance (unit, m) from the last intersection, and speed (unit, m/s).

[0035] Step 4: For each intersection in the road network, the phase information of the intersection is obtained. The position, speed, acceleration and road information of the vehicles on the roads connected to the current intersection are used to estimate the sum of the farthest driving distances of all vehicles in the vehicles in the incoming lane that can pass through the intersection during the green light duration, and a priority traffic index that can reflect the congestion degree of the incoming lane is introduced and multiplied with the sum of the farthest driving distances. If the executed phase of the next traffic light cycle is the same as the current phase, the green light duration of the vehicle is T, and if the executed phase of the next traffic light is different from the current phase, the green light duration of the vehicle is T−t.

[0036] Further, the third step is realized by the following sub-steps:

[0037] (1) Assuming that the current phase is phase and the other three switchable phases are phase.sub.1, phase.sub.2, phase.sub.3.

[0038] (2) According to the information of the current intersection and the downstream intersection, the road information connected with the two intersections, and the information of vehicles driving on the connected roads, the expected return when the executed phase of the next traffic light cycle is the same as the current phase is estimated, that is, the expected return when the executed phase of the next traffic light cycle is phase. At this time, because the phase remains unchanged, the green light duration in the next traffic light cycle is T. For this phase, the sum of the expected returns of all the incoming lanes is calculated, and t the expected return Reward.sub.keep of one of the incoming lane is calculated as follows:

Reward.sub.keep=(drive.sub.distance+future.sub.distance)*priority.sub.factor

where Reward.sub.keep indicates the expected return of an incoming lane, drive.sub.distance indicates the timely driving distance, future.sub.distance indicates the future driving distance, and priority.sub.factor indicates the road priority index.

[0039] drive.sub.distance and future.sub.distance are calculated as follows.

[0040] For any vehicle Veh driving in the incoming lane, assuming its driving speed is v, its acceleration is a, the maximum speed limit of the road where the vehicle Veh is located is V, the road length is L.sub.1, and the distance from the upstream intersection is dis, then the distance that the vehicle still needs to travel to reach the intersection is:

d=L.sub.1−dis

[0041] The time required for the vehicle to reach the intersection is:

[00006]${\mathrm{remain}}_{\mathrm{time}}=\{\begin{array}{cc}\frac{-v+\sqrt{v^2+2a*d}}{a},& \frac{V^2-v^2}{2*a}\ge d\\ \frac{V-v}{a}+\frac{d}{V}-\frac{V^2-v^2}{2*a*v},& \frac{V^2-v^2}{2*a}<d\end{array}$

[0042] If remain.sub.time, the vehicle can pass the current intersection. For all vehicles that can pass through the intersection, their driving distances in time T are calculated:

[00007]$D_T=\{\begin{array}{cc}v*T+\frac{a*T^2}{2},& \frac{V-v}{a}\ge T\\ T*v+\frac{{\left(V-v\right)}^2}{2*a},& \frac{V-v}{a}<T\end{array}$

[0043] For the corresponding three outgoing lanes, assuming that the road length is L.sub.2, and the queue lengths of the lanes for turning left, going straight and turning right are q.sub.1, q.sub.2, q.sub.3, respectively, the distances from the tail of the fleet to the current intersection are L.sub.2−q.sub.1, L.sub.2−q.sub.2, L.sub.2−q.sub.3, respectively.

[0044] For any vehicle that can pass through the current intersection, it is assumed that the probability of entering the lanes for turning left, going straight or turning right is p.sub.1, p.sub.2, p.sub.3, where p.sub.1+p.sub.2+p.sub.3=1. If the vehicle drives into the lane for turning left:

[00008]$\mathrm{When}d+L_2-q_1\ge D_T,\mathrm{then}$${\mathrm{drive}}_{\mathrm{distance}-\mathrm{left}}=D_T$${\mathrm{future}}_{\mathrm{distance}-\mathrm{left}}=0$$\mathrm{When}d+L_2-q_1<D_T,\mathrm{then}$${\mathrm{drive}}_{\mathrm{distance}-\mathrm{left}}=d+L_2-q_1$${\mathrm{future}}_{\mathrm{distance}-\mathrm{left}}=\alpha *p*\frac{1}{L_2-q_1}*\left(D_T-\left(d+L_2-q_1\right)\right)$

[0045] If the vehicle drives into the lane for going straight:

[00009]$\mathrm{When}d+L_2-q_2\ge D_T,\mathrm{then}$${\mathrm{drive}}_{\mathrm{distance}-\mathrm{through}}=D_T$${\mathrm{future}}_{\mathrm{distance}-\mathrm{through}}=0$$\mathrm{When}d+L_2-q_2<D_T,\mathrm{then}$${\mathrm{drive}}_{\mathrm{distance}-\mathrm{through}}=d+L_2-q_2$${\mathrm{future}}_{\mathrm{distance}-\mathrm{through}}=\alpha *p*\frac{1}{L_2-q_2}*\left(D_T-\left(d+L_2-q_2\right)\right)$

[0046] If the vehicle drives into the lane for turning right:

[00010]$\mathrm{When}d+L_2-q_3\ge D_T,\mathrm{then}$${\mathrm{drive}}_{\mathrm{distance}-\mathrm{right}}=D_T$${\mathrm{future}}_{\mathrm{distance}-\mathrm{right}}=0$$\mathrm{When}d+L_2-q_3<D_T,\mathrm{then}$${\mathrm{drive}}_{\mathrm{distance}-\mathrm{right}}=d+L_2-q_3$${\mathrm{future}}_{\mathrm{distance}-\mathrm{right}}=\alpha *p*\frac{1}{L_2-q_3}*\left(D_T-\left(d+L_2-q_3\right)\right)$

where α is the loss coefficient of the future driving distance, the loss coefficient is the empirical coefficient due to the loss of the future driving distance caused by the start delay or braking of the preceding queuing vehicle, and the value here is 0.8, and p is a probability that the lane of the direction which the vehicle enters in at the downstream intersection is under a green light.

[00011]$p=\frac{1}{4}*\left(1+\frac{t}{T}\right)$

[0047] Then the timely driving distance and future driving distance of all vehicles that can pass through the intersection in the outgoing lane are respectively:

drive.sub.distance=drive.sub.distance-left*p.sub.1+drive.sub.distance-through*p.sub.2+drive.sub.distance-right*p.sub.3

future.sub.distance=future.sub.distance-left*p.sub.1+future.sub.distance-through*p.sub.2+future.sub.distance-right*p.sub.3

[0048] The road priority index priority.sub.factor is calculated as follows:

priority.sub.factor=normal(queue.sub.length)*normal(avg_delay)*normal(avg_travel.sub.time)

[0049] queue.sub.length represents the queue length of the incoming lane, which is the total number of vehicles with a speed less than 0.01 m/s.

[0050] avg_delay represents the average delay of vehicles of the incoming lane,

avg_delay=1−avg_speed/speed_limit

where avg_speed indicates the average speed of all vehicles in the incoming lane, and speed_limit is the maximum speed limit in the incoming lane.

[0051] avg_travel.sub.time represents the average driving time of all vehicles in the lane.

[0052] Normal means that the Min-Max method is adopted to carry out dimensionless treatment on the three factors, respectively.

[0053] The expected returns of other lanes in this phase are calculated in the same way as above. After the calculation is completed according to the above method, the total expected return of the phase is the sum of the expected returns of all incoming lanes.

[0054] (3) According to the information of the current intersection and the downstream intersection, the road information connected with the two intersections, and the information of the vehicles driving on the connected roads, the return when the phase is switched in the next traffic light cycle is estimated, that is, the expected return of any switched phase phase.sub.1∈{phase.sub.1, phase.sub.2, phase.sub.3} is estimated. The expected return of one incoming lane is calculated as follows:

[00012]${\mathrm{Reward}}_{{\mathrm{change}}_{{\mathrm{phase}}_i}}=\left({\mathrm{drive}}_{\mathrm{distance}}+{\mathrm{future}}_{\mathrm{distance}}\right)*{\mathrm{priority}}_{\mathrm{factor}}$

where

[00013]${\mathrm{Reward}}_{{\mathrm{change}}_{{\mathrm{phase}}_i}}$

represents the expected return of phase.sub.i, drive.sub.distance represents the corresponding timely driving distance, future.sub.distance represents the corresponding future driving distance, and priority.sub.factor represents the corresponding road priority index. drive.sub.distance and future.sub.distance are calculated as follows:

[0055] For any vehicle Veh traveling in the incoming lane, assuming its speed is v, its acceleration is a, the maximum speed limit of the road where the vehicle Veh is located is V, the length of the road is L.sub.1, and the distance from the upstream intersection is dis, then the distance that the vehicle still needs to travel to reach the intersection is:

d=L.sub.1−dis

[0056] The time required for the vehicle to reach the intersection is:

[00014]${\mathrm{remain}}_{\mathrm{time}}=\{\begin{array}{c}\frac{-v+\sqrt{v^2+2a*d}}{a},\frac{V^2-v^2}{2*a}\ge d\\ \frac{V-v}{a}+\frac{d}{V}-\frac{V^2-v^2}{2*a*v},\frac{V^2-v^2}{2*a}<d\end{array}$

[0057] If remain.sub.time<T−t, the vehicle can pass the current intersection. For all vehicles that can pass through the intersection, the driving distances thereof in time T−t are calculated:

[00015]$D_T=\{\begin{array}{c}v*\left(T-t\right)+\frac{a*{\left(T-t\right)}^2}{2},\frac{V-v}{a}\ge T-t\\ \left(T-t\right)*v+\frac{{\left(V-v\right)}^2}{2*a},\frac{V-v}{a}<T-t\end{array}$

[0058] For the corresponding three outgoing lanes, assuming that the road length is L.sub.2, and the queue length of the lane for turning left, going straight and turning right is q.sub.1, q.sub.2, q.sub.3, respectively, the distances from the tail of the motorcade to the current intersection are L.sub.2−L.sub.2−q.sub.2, L.sub.2−q.sub.3, respectively.

[0059] For any vehicle that can pass through the current intersection, it is assumed that the probability of entering the lane for turning left, going straight and turning right is p.sub.1, p.sub.2, p.sub.3, where p.sub.1+p.sub.2+p.sub.3=1. If it drives into the left turn lane:

[00016]$\mathrm{when}d+L_2-q_1\ge D_T,$${\mathrm{drive}}_{\mathrm{distance}-\mathrm{left}}=D_T$${\mathrm{future}}_{\mathrm{distance}-\mathrm{left}}=0$$\mathrm{when}d+L_2-q_1<D_T,$${\mathrm{drive}}_{\mathrm{distance}-\mathrm{left}}=d+L_2-q_1$${\mathrm{future}}_{\mathrm{distance}-\mathrm{left}}=\alpha *p*\frac{1}{L_2-q_1}*\left(D_T-\left(d+L_2-q_1\right)\right)$

[0060] If it drives into the straight lane:

[00017]$\mathrm{when}d+L_2-q_2\ge D_T,$${\mathrm{drive}}_{\mathrm{distance}-\mathrm{through}}=D_T$${\mathrm{future}}_{\mathrm{distance}-\mathrm{through}}=0$$\mathrm{when}d+L_2-q_2<D_T,$${\mathrm{drive}}_{\mathrm{distance}-\mathrm{through}}=d+L_2-q_2$${\mathrm{future}}_{\mathrm{distance}-\mathrm{through}}=\alpha *p*\frac{1}{L_2-q_2}*\left(D_T-\left(d+L_2-q_2\right)\right)$

[0061] If it drives into the right turn lane:

[00018]$\mathrm{when}d+L_2-q_3\ge D_T,$${\mathrm{drive}}_{\mathrm{distance}-\mathrm{right}}=D_T$${\mathrm{future}}_{\mathrm{distance}-\mathrm{right}}=0$$\mathrm{when}d+L_2-q_3<D_T,$${\mathrm{drive}}_{\mathrm{distance}-\mathrm{right}}=d+L_2-q_3$${\mathrm{future}}_{\mathrm{distance}-\mathrm{right}}=\alpha *p*\frac{1}{L_2-q_2}*\left(D_T-\left(d+L_2-q_3\right)\right)$

where α is the loss coefficient of the future driving distance, the loss coefficient is the loss of future driving distance due to the start delay or braking of the preceding queuing vehicle, which is an empirical coefficient and the value of which is 0.8 in this embodiment, and p is the probability that the lane of the direction which the vehicle enters in at the downstream intersection is under the green light.

[00019]$p=\frac{1}{4}*\left(1+\frac{t}{T}\right)$

[0062] Then the timely driving distance and future driving distance of all vehicles that can pass through the intersection in the outgoing lane are respectively:

drive.sub.distance=drive.sub.distance-left*p.sub.1+drive.sub.distance-through*p.sub.2+drive.sub.distance-right*p.sub.3

future.sub.distance=future.sub.distance-left*p.sub.1+future.sub.distance-through*p.sub.2+future.sub.distance-right*p.sub.3

[0063] The road priority index priority.sub.factor is calculated as follows

priority.sub.factor=normal(queue.sub.length)*normal(avg_delay)*normal(avg_travel.sub.time)

[0064] queue.sub.length represents the queue length of the incoming lane, and is the total number of vehicles with a speed less than 0.01 m/s.

[0065] avg_delay represents the average delay of vehicles of the incoming lane,

avg_delay=1−avg_speed/speed_limit

where avg_speed indicates the average speed of all vehicles in the incoming lane, andspeed_limit is the maximum speed limit in the incoming lane. avg_travel.sub.time indicates the average driving time of all vehicles in the lane.

[0066] Normal means that the Min-Max method is adopted to carry out dimensionless treatment on the three factors respectively.

[0067] The expected return of other lanes of the phase phase.sub.i is calculated in the same way as above. After the calculation is completed according to the above method, the total expected return of the phase is the sum of the expected return of all incoming lanes. And the total expected returns of other phases are calculated respectively.

[0068] (4) The phase with the largest total expected return of phase switching is obtained, and the corresponding maximum total expected return is calculated.

[00020]${\mathrm{Reward}}_{\mathrm{change}-\max }=\max \left({\mathrm{Reward}}_{{\mathrm{change}}_{{\mathrm{phase}}_1}},{\mathrm{Reward}}_{{\mathrm{change}}_{{\mathrm{phase}}_2}},{\mathrm{Reward}}_{{\mathrm{change}}_{{\mathrm{phase}}_3}}\right)$

[0069] Reward.sub.change-max represents the maximum total expected return of phase switching, and the corresponding phase is recorded as phase.sub.j.

[0070] Step 5: according to the estimated total expected return of keeping the current phase and the maximum total expected return of phase switching, if the maximum return of phase switching is a certain multiple of the expected return of keeping the current phase, the phase is switched to the phase with the maximum total return, otherwise, the current phase is kept.

[0071] Further, step 5 is realized by the following sub-steps:

[0072] Reward.sub.keep and Reward.sub.change-max values are compared.

[0073] If Reward.sub.change-max≤β*Reward.sub.keep, the current phase is kept.

[0074] If Reward.sub.change-max>β*Reward.sub.keep, the phase to is switched to phase.sub.j.

where β is an empirical value, and the value of β here is 1.6.

[0075] According to this method, based on the urban road network, 2024 intersections, 3010 roads and 10186 traffic flows are set in the CBEngine traffic simulation engine for simulation, as shown in FIGS. 3 and 4. In FIG. 4, a certain intersection in the CBEngine traffic simulation engine is circled, and the traffic flow in the north-south straight lane of the current road is large, so the phase 2, that is, the north-south straight traffic is executed in the next traffic light cycle. This method and the maximum pressure method are used to control the traffic lights respectively, and it is found that the delay index of this method is reduced by 23% compared with that of the maximum queuing pressure method. This method can dynamically control the phase transformation according to the real-time state of road traffic in each traffic light cycle, so that as many vehicles as possible can travel farther in the green light duration, thereby obviously alleviating traffic congestion and improving the travel experience.

[0076] The above-mentioned embodiments are used to explain, rather than limit the present application. Any modification and change made to the present application within the scope of protection of the spirit and claims of the present application shall fall within the scope of protection of the present application.