MULTI-OBJECTIVE OPTIMIZATION CONTROL METHOD AND SYSTEM FOR COOPERATIVE RAMP MERGING OF CONNECTED VEHICLES ON HIGHWAY

Abstract

State data of vehicles in a control area is collected, then time of starting control and selection of controlled vehicles are decided according to a position of a ramp merging vehicle, information is transmitted into a traffic control module by means of a data transmission module, and an artificial intelligence based ramp merging multi-objective control model ensures efficient, safe, and energy-saving operation of overall traffic of a road while completing ramp merging by means of a vehicle traveling track in the cooperative ramp control area. Compared with a traditional method, the present invention greatly promotes merging of the ramp vehicles, and different from other methods having the defect of transforming the problem of ramp merging into the problem of vehicle sequencing, the present invention greatly improves efficiency of ramp merging,

Claims

1. A multi-objective optimization control method for cooperative ramp merging of connected vehicles on a highway, comprising: step 1, collecting state data of vehicles in a control area of the highway, and analyzing and processing the state data, the control area comprising, an intersection of a main road and a ramp, a merging area, and partial road segments of the main road, the ramp and an acceleration lane of the highway, a range of the control area being a communication range of a road side unit, the road side unit being arranged at the intersection of the main road and the ramp of the highway, and the merging area being a preselected area comprising the partial road segments of the acceleration lane and road segments of the main road parallel to the partial road segments of the acceleration lane; step 2, constructing a set of alternatives for a ramp merging vehicle, an auxiliary vehicle, and a guidance vehicle according to the state data of the vehicles in the control area; step 3, inputting the set of alternatives into an artificial intelligence based ramp merging multi-objective control model separately, and further determining selection of the auxiliary vehicle and the guidance vehicle by means of an optimal value strategy; step 4, according to the auxiliary vehicle, the guidance vehicle, and the merging vehicle that are selected, controlling and adjusting accelerations of the auxiliary vehicle and the guidance vehicle, to ensure safe merging of the ramp merging vehicle from the acceleration lane into the main road in the selected merging area; and step 5, collecting state data of the ramp merging vehicle and the auxiliary vehicle, of which the accelerations are adjusted, and returning to step 4 to adjust accelerations at a next moment.

2. The multi-objective optimization control method for cooperative ramp merging of connected vehicles on a highway of claim 1, wherein in step 1, the state data of the vehicles comprises positions, speeds, powder battery states, and corresponding moment information of the vehicles in the control area.

3. The multi-objective optimization control method for cooperative ramp merging of connected vehicles on a highway of claim 1, wherein in step 1, the analyzing and processing the state data comprises: analyzing the data, extracting features, and fusing information.

4. The multi-objective optimization control method for cooperative ramp merging of connected vehicles on a highway according to claim 1, wherein in steps 2 and 3, the optimal value strategy selected for the ramp merging vehicle, the guidance vehicle, and the auxiliary vehicle is as follows: i) selecting the merging vehicle: setting a vehicle, of which a front bumper is closest to a ramp exit line, on the ramp as the merging vehicle, and obtaining state information of all the vehicles in the control area in first t time steps; ii) according to position information of all the vehicles, preliminary selecting each z main road vehicles in the rear of and in front of the merging vehicle as alternative vehicles of the auxiliary vehicle and the guidance vehicle on the basis of front and rear relations between main line vehicles and the merging vehicle; iii) in all the alternative vehicles, selecting two vehicles that are adjacent to each other as a group of guidance vehicle and auxiliary vehicle, to construct an alternative set AL of the guidance vehicle and the auxiliary vehicle of the merging vehicle; and iv) using a traversal method to substitute combinations in the alternative set AL into the artificial intelligence based ramp merging multi-objective control model separately, and determining the merging vehicle, the auxiliary vehicle, and the guidance vehicle that are finally selected on the basis of a value function Q.sub.π of the model.

5. The multi-objective optimization control method for cooperative ramp merging of connected vehicles on a highway according to claim 4, wherein the state information of all the vehicles in the control area in the first t time steps comprises speeds, positions, and accelerations.

6. The multi-objective optimization control method for cooperative ramp merging of connected vehicles on a highway according to claim 4, wherein in step 4) of the optimal value strategy selected for the merging vehicle, the auxiliary vehicle, and the guidance vehicle, constructing an objective function and constraints of the artificial intelligence based ramp merging multi-objective control model successfully merged into a moment comprises: i) setting t̂ as a safe merging moment to construct a position and speed relation to be satisfied by the guidance vehicle, the auxiliary vehicle, and the ramp merging vehicle at the safe merging moment: $\begin{array}{cc}{x}_m\left(\widehat{t}\right)\le x_l\left(\widehat{t}\right)-\left(L_1+s_0\right)-\tau v_m\left(\widehat{t}\right)& \text{­­­(1)}\end{array}$ $\begin{array}{cc}{x}_m\left(\widehat{t}\right)\ge x_f\left(\widehat{t}\right)+\left(L_1+s_0\right)+\tau v_f\left(\widehat{t}\right)& \text{­­­(2)}\end{array}$ $\begin{array}{cc}{v}_m\left(\widehat{t}\right)\le v_l\left(\widehat{t}\right)& \text{­­­(3)}\end{array}$ $\begin{array}{cc}{v}_m\left(\widehat{t}\right)\ge v_f\left(\widehat{t}\right)& \text{­­­(4)}\end{array}$ $\begin{array}{cc}{x}_m\left(\widehat{t}\right)\in \left[d_{min},d_{max}\right]& \text{­­­(5)}\end{array}$ x.sub.l, v.sub.l and α.sub.l representing a position, a speed, and an acceleration of the guidance vehicle; x.sub.f, v.sub.f and α.sub.f representing a position, a speed, and an acceleration of the auxiliary vehicle; x.sub.m, v.sub.m and a.sub.m representing a position, a speed, and an acceleration of the ramp merging vehicle; τ being a constant time interval, L.sub.1 being a length of the vehicle, s.sub.0 being a pause gap, d.sub.min and d.sub.maxbeing a start point and an end point of the merging area respectively, and the merging area having a length of d.sub.max - d.sub.min; and the equation sequentially representing, from top to bottom, that the ramp merging vehicle is at the rear of the guidance vehicle, the ramp merging vehicle is in front of the auxiliary vehicle, the ramp merging vehicle and the guidance vehicle have a consistent speed, the ramp merging vehicle and the auxiliary vehicle have a consistent speed, and the ramp merging vehicle is safely merged from the acceleration lane into the main road in the selected merging area; and ii) under the condition of satisfying 1), further constructing an objective function C comprising, but not limited to, objectives of driving comfort, vehicle energy consumption, traffic efficiency, etc.; $\begin{array}{cc}minC\left(\widehat{t}\right)=min{\displaystyle {\int }_0^{\widehat{t}}\left(c^1{C}_1\left(\widehat{t}\right)+c^2{C}_2\left(\widehat{t}\right)+.Math.+c^n{C}_n\left(\widehat{t}\right)\right)}& \text{­­­(6)}\end{array}$ C.sub.n(t̂) representing cost functions of different objectives, and c.sup.n representing parameters.

7. The multi-objective optimization control method for cooperative ramp merging of connected vehicles on a highway according to claim 4, wherein in step 4) of the optimal value strategy selected for the merging vehicle, the guidance vehicle and the auxiliary vehicle, the artificial intelligence based ramp merging multi-objective control model is solved by using a reinforcement learning actor-critic algorithm, which specifically comprises: i) establishing a state space $S$ and a behavior space $\mathcal{A}$ : selecting, according to the state data of the guidance vehicle, the auxiliary vehicle, and the ramp merging vehicle, six-dimensional state information s = {x.sub.l, x.sub.m, x.sub.f, v.sub.l, v.sub.m, v.sub.f} to represent the most relevant influence factor in an environment, s ∈ S, and selecting a control behavior strategy a = {a.sub.m, a.sub.f}, a ∈ A on the basis of a control object; ii) establishing an optimal objective: according to a vehicle constraint relation at a safe merging moment t̂, constructing an optimal objective space set $g^{\ast }=\left\{g_1^{\ast },g_2^{\ast },g_3^{\ast },g_4^{\ast },g_5^{\ast }\right\}$ for ramp merging, $g*\in \mathcal{G},\mathcal{G}$ being an objective space set, $g_1^{\ast }$ representing that equation (1) that the merging vehicle is at the rear of the guidance vehicle is satisfied, $g_2^{\ast }$ representing that equation (2) that the ramp merging vehicle is in front of the auxiliary vehicle is satisfied, $g_3^{\ast }$ representing that equation (3) that the ramp merging vehicle and the guidance vehicle have a consistent speed is satisfied, $g_4^{\ast }$ representing that equation (4) that the ramp merging vehicle and the auxiliary vehicle have a consistent speed is satisfied, and $g_5^{\ast }$ representing that equation (5) that the ramp merging vehicle is safely merged from the acceleration lane into the main road in the selected merging area is satisfied; iii) constructing an objective space: on the basis of classifications included by the optimal objective space set $g^{\ast }=\left\{g_1^{\ast },g_2^{\ast },g_3^{\ast },g_4^{\ast },g_5^{\ast }\right\}\left(\text{in}2\right),$ establishing the objective space set $\mathcal{G}$ to satisfy g = {g.sub.1, g.sub.2, g.sub.3, g.sub.4, g.sub.5}, g.sub.1 representing a position relation between the ramp merging vehicle and the guidance vehicle, g.sub.2 representing a position relation between the ramp merging vehicle and the auxiliary vehicle, g.sub.3 representing a speed relation between the ramp merging vehicle and the guidance vehicle, g.sub.4 representing a speed relation between the ramp merging vehicle and the auxiliary vehicle, and g.sub.5 representing a relation between a position of the ramp merging vehicle and the merging area; iv) constructing rewards: with a reward function being $r:\mathcal{S}\times \mathcal{A}\times \mathcal{G}.fwdarw.R$ , introducing, according to safe, efficient and comfortable traveling requirements, at least two short-term objective rewards under the condition that the reward r(s,a,g*) at each time step includes a long-term objective merging reward R.sub.m(t), the long-term objective merging reward R.sub.m(t) that must be inc3luded being represented as follows: ; $\begin{array}{cc}\begin{array}{l}{R}_m\left(t\right)=\\ \left\{0,\begin{array}{c}1,\text{safe successful merging}\\ \text{the merging vehicle can}\left(\text{may}\right)\text{be merged, but}{g}^{\ast }isnotsatisfied\\ -1,\text{others}\end{array}\right)\end{array}& \text{­­­(7)}\end{array}$ v) obtaining a data link: according to state, objective, strategy and reward data of the t.sup.th time step obtained from 1) to 4), obtaining the data link (s.sub.t||g*,a,r,s.sub.t+1||g*), and storing the data into an intelligent optimization module, s||g* representing a connection between a state s and an objective g*; vi) expanding data exploration, and optimizing the objective space: further providing a multi-experience replay based virtual objective construction algorithm, to introduce virtual objectives, to optimize the objective space, and simultaneously expand the data exploration; and vii) in each time step, according to the data link stored by the intelligent optimization module, on the basis of an actor-critic algorithm framework, training a merging control strategy by means of a deep neural network taking θ.sub.A as a parameter, the strategy directly outputting an action to control acceleration magnitudes and states of the ramp merging vehicle and the auxiliary vehicle and objective inputs, strategy optimization aiming at to find an optimal behavior strategy a, to maximize a return expectation of a whole trip, and finally, the optimal control strategy being output by means of forward delivery of a trained network: a. = π (s,g | θ.sub.A).

8. The multi-objective optimization control method for cooperative ramp merging of connected vehicles on a highway according to claim 7, wherein the short-term objective rewards in construction of the rewards of the reinforcement learning actor-critic algorithm comprises, but not limited to, an energy-saving reward R.sub.e(t), a comfort reward R.sub.p(t), an efficient traffic reward R.sub.s(t), and a battery state reward R.sub.b(t); and a coupled Chebyshev based multi-objective reward optimization method for the reinforcement learning actor-critic algorithm comprises: (i) determining optimized reward terms: under the condition of assuming that the number of a variety of introduced real-time short-term objectives after successful merging isn.sub.r, determining the number of the optimized reward terms for a merging problem as n.sub.r; (ii) determining a super-ideal optimal value of each reward: constructing the super-ideal optimal value $r_i^{\ast \ast }=r_i^{\ast }+{\vartheta }_i\left(i=1,2,.Math.,n_r\right)$ of each objective, $r_i^{\ast }$ being an ideal value, and ϑ.sub.i being a constant according to selection of empirical data, and representing a degree that the super-ideal optimal value is better than the ideal value; (iii) constructing a generalized weighted Chebyshev optimization model for a multi-objective problem: with setting λ.sub.i as a Chebyshev weight of a short-term objective, transforming the multi-objective problem into a generalized weighted Chebyshev multi-objective problem as follows: $\begin{array}{cc}\min \left\{F_c+\rho {\displaystyle {.Math.}_{i=1}^{n_r}\left(r_i^{\ast \ast }-r_i\right)}\right\}& \text{­­­(14)}\end{array}$ The following condition being satisfied: $\begin{array}{cc}\begin{array}{l}{F}_c-{\lambda }_i\left(r_i^{\ast \ast }-r_i\right)\ge 01\le i\le n_r\\ r_i\text{being an unrestricted variable}1\le i\le n_r\\ F_c\ge 0\end{array}& \text{­­­(15)}\end{array}$ F.sub.c being a weighted Chebyshev norm, $F_c=\max \left\{{\lambda }_i\left(r_i^{\ast \ast }-r_i\right)\right\}$ representing a maximum deviation between each objective and the super-ideal optimal value; and $\rho {\displaystyle {.Math.}_{i=1}^{n_r}\left(r_i^{\ast \ast }-r_i\right)}$ being a term ensuring the stable algorithm, usually, ρ = 0.001, and a computation equation of λ.sub.i being referred to as follows: $\begin{array}{cc}{\lambda }_i=\left\{\begin{array}{ll}\frac{1}{\left(r_i^{\ast \ast }-r_i\right)}{\left[{\displaystyle {.Math.}_{i=1}^{n_r}\frac{1}{\left(r_i^{\ast \ast }-r_i\right)}}\right]}^{-1}\hfill & r_i\ne r_i^{\ast \ast }\hfill \\ 1\hfill & r_i=r_i^{\ast \ast }\hfill \\ 0\hfill & r_i\ne r_i^{\ast \ast },\text{­­­(16)}\exists i,\text{and}{r}_i=r_i^{\ast \ast }\hfill \end{array}\right)& \end{array}$ (iv) constructing a reward function as: $\begin{array}{cc}\begin{array}{l}{R}_m\left(t\right)=\\ \left\{0,\begin{array}{c}1,\text{safe successful merging}\\ \text{the merging vehicle can}\left(\text{may}\right)\text{be merged, but}{g}^{\ast }isnotsatisfied\\ -1,\text{others}\end{array}\right)\end{array}& \text{­­­(17)}\end{array}$ .

9. The multi-objective optimization control method for cooperative ramp merging of connected vehicles on a highway according to claim 7, wherein the multi-experience replay based virtual objective construction algorithm provided in construction of the rewards of the reinforcement learning actor-critic algorithm comprises: (i) constructing multi-experience virtual objectives and optimizing the objective space at a t time step: on the basis of the objective g* and the data link (s.sub.t||g*,a,r,s.sub.t+1||g*) at the t time step, constructing a fully connected neural network H, g = H.sub.φ(s, g*)taking φ as a parameter, to obtain 1 virtual objectives in a current state: g.sup.1=H.sub.φ(s.sub.t,g*), g.sup.2=Hφ(s.sub.t+1,g*), ..., g.sup.l=H.sub.φ(s.sub.t+l-1,g*); (ii) optimizing data exploration on the basis of the optimized objective space: constructing 1 virtual objective data links on the basis of the obtained 1 virtual objectives: (s.sub.t||g.sup.1,a,r,s.sub.t+1||g.sup.1), (s.sub.t||g.sup.2,a,r,s.sub.t+1||g.sup.2), ..., (s.sub.t||g.sup.l,a,r,s.sub.t+1||g.sup.l); and storing the virtual objective data links into the intelligent optimization module; (iii) optimizing data exploration at a full time step: repeating ① and ② for each time step to complete data exploration and objective space optimization at all time steps; (iv) selecting virtual objective effectiveness on the basis of an artificial intelligence model: training the artificial intelligence model according to an optimized data set, and selecting, according to a training result, an optimal virtual objective data link at each time step; and (v) verifying the fully connected neural network H for the virtual objectives: according to the optimal virtual objective data link at each time step, verifying the parameter φ of the fully connected neural network H for the virtual objectives, to constantly improve accuracy of generation of the virtual objectives, so as to ensure algorithm performance and a training speed. 10. A system based on the multi-objective optimization control method for cooperative ramp merging of connected vehicles on a highway of any one of claims 1-9, comprising: an information collection module, a data transmission module, a traffic control module, and an intelligent optimization module, wherein the information collection module is used for collecting state data of vehicles in a control area of a highway, analyzing and processing the state data, and selecting a ramp merging vehicle, an auxiliary vehicle, and a guidance vehicle; the information collection module comprises an in-vehicle unit, and a road side unit, the road side unit being arranged at an intersection of a main road and a ramp of a highway, the road side unit being used for collecting positions, speeds and corresponding moment information of the vehicles in the control area, and being further used for collecting time of determining the ramp merging vehicle, and time when a front bumper reaches a ramp exit line, and the in-vehicle unit being used for collecting power battery states and corresponding moment information of the vehicles in the control area; the data transmission module is used for using a mobile communication technology as a subject information transmission communication mode to assist one or two of wireless communication modes of wireless fidelity/bluetooth (WiFi/BT), and dedicated short range communications (DSRC) to transmit data between the information collection module and the traffic control module, the traffic control module and the intelligent optimization module; the traffic control module is used for obtaining, according to the state data of the vehicles provided by the information collection module, a behavior strategy a, an objective g and a reward r that are optimal in real time, sending the behavior strategy to the in-vehicle unit for real-time control of the vehicles, and further sending the behavior strategy a, the objective g and the reward r that are optimal to the intelligent optimization module; and the intelligent optimization module is used for storing data sent by the traffic control module, optimizing a ramp merging multi-objective control model on the basis of an alternative set AL of the vehicles selected in step 3) and an optimization algorithm provided in step 4), and transmitting the optimized model to the traffic control module.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0068] FIG. 1 is a schematic diagram of a merging scenario of a highway provided in the present invention; and

[0069] FIG. 2 is an architecture diagram of a multi-objective optimization control method for cooperative ramp merging of connected vehicles on a highway of the present invention.

DETAILED DESCRIPTION OF THE EMBODIMENTS

[0070] The present invention will be further described in detail below in combination with the accompanying drawings and particular embodiments.

[0071] A method provided in the present invention is based on assumptions as follows: 1) network connected vehicles need to have necessary capabilities to transmitting information and implement instructions, that is, the network connected vehicles have the capability to interact information by means of an in-vehicle unit, and the vehicles fully execute control instructions; 2) a ramp control area includes an intersection of a main road and a ramp, a ramp merging area and partial road segments of a main line and the ramp, and a control range of the control area is a communication range of a road side unit; 3) the merging area has a fixed length, i.e., d.sub.m = d.sub.max - d.sub.min; 4) delays in information transmission, data processing and computation, and instruction execution are ignored, that is, it is assumed that a running speed of each module is enough to support running of a system; and 5) influences of lateral movement and temperatures of the vehicles on the vehicles are ignored.

[0072] A multi-objective optimization control system for cooperative ramp merging of connected vehicles on a highway provided in the present invention includes an information collection module, a data transmission module, a traffic control module, and an intelligent optimization module.

[0073] 1) The information collection module collects, by means of an in-vehicle unit and a road side unit, state information such as speeds, positions, and power battery states and corresponding moment information of vehicles in a control area and start moment information of merging of a ramp vehicle (a selected moment of a merging vehicle), and analyzing and processing the data, and the analyzing and processing the data include, but not limited to, analyze the data, extract features, fuse information, etc.; and

[0074] information collection cooperation of the in-vehicle unit and the road side unit included in the information collection module is as follows: [0075] (1) the road side unit is responsible for collecting traffic state information, such as the speeds and positions of the vehicles in the area, and the start moment of merging of the ramp merging vehicle; and [0076] (2) the in-vehicle unit is responsible for collecting vehicle related parameter information, such as engine state information, battery state information, and gear information. A power composition of a network connected vehicle designed in the present invention includes: a fuel vehicle, a pure electric vehicle, a hydrogen energy vehicle, and a hybrid electric vehicle.

[0077] 2) The data transmission module uses a fifth generation mobile communication technology (5G) as a subject information transmission communication mode to assist one or two of wireless communication modes of wireless fidelity/bluetooth (WiFi/BT), dedicated short range communications (DSRC), etc. to transmit data between the in-vehicle unit, the road side unit, and each module; [0078] 3) the traffic control module obtains, according to traffic state information provided by the information collection module, a behavior strategy a, an objective g and a reward r that are optimal in real time, sends the strategy to the in-vehicle unit for real-time control of the vehicles, and further packages a traffic state information set to be sent to the intelligent optimization module; and [0079] 4) the intelligent optimization module includes a training sub-module and a data sub-module. The data sub-module stores data sent by the traffic control module, and applies an artificial intelligence data exploration method to obtain a number of data pairs for storage; and the training sub-module utilizes a reward function to optimize an artificial intelligence based ramp merging multi-objective control model according to the data provided by the data sub-module, and transmits an optimized model to the traffic control module.

[0080] The control area includes an intersection of a main road and a ramp, a ramp merging area, and partial road segments of a main line and the ramp. It is worth noting that a control range of the control area is a communication range of the road side unit, which sees FIG. 1 for details.

[0081] As shown in FIG. 2, a multi-objective optimization control method for cooperative ramp merging of connected vehicles on a highway provided in the present invention includes: [0082] S01 obtain, by an information collection module, state data of vehicles in a control area; [0083] S02 perform, by the information collection module, data analysis, information fusion, etc. on the collected data by means of an intelligent data analysis model; [0084] S03 according to the state data of the vehicles in the control area, construct a set AL of alternatives of a ramp merging vehicle, an auxiliary vehicle and a guidance vehicle; [0085] S04 input the set of alternatives into an artificial intelligence based ramp merging multi-objective control model separately, and further determine selection of the auxiliary vehicle and the guidance vehicle by means of an optimal value strategy; [0086] S05 transmit, by a transmission module, the vehicles selected in S04 and vehicle related collected data to a control module, [0087] S06 confirm, by a traffic control module, the vehicles to which instructions are sent to implement a decision on the basis of the artificial intelligence based ramp merging multi-objective control model, a frame of the artificial intelligence based ramp merging multi-objective control model being a reinforcement learning actor-critic algorithm; [0088] S07 transmit, by the transmission module, decision information to in-vehicle units of controlled vehicles to achieve a vehicle safe, efficient and energy-saving ramp merging objective, and the controlled vehicles are the ramp merging vehicle and the auxiliary vehicle; and [0089] S08 collect state information data pairs of the controlled vehicles under a control strategy, and store the state information data pairs of the controlled vehicles into a data sub-module to form a closed loop control.

[0090] In S03 and S04, a selection method for the merging vehicle, the auxiliary vehicle and the guidance vehicle is based on a framework of the optimal value strategy algorithm framework, and includes: [0091] 1) select the merging vehicle: set a vehicle, of which a front bumper is closest to a ramp exit line, on the ramp as the merging vehicle, and obtain state information, such as speeds, positions, and accelerations, of all the vehicles in the control area in first t time steps; [0092] 2) according to position information of all the vehicles, preliminary select each z main road vehicles in the rear of and in front of the merging vehicle as alternative vehicles of the auxiliary vehicle and the guidance vehicle on the basis of front and rear relations between main line vehicles and the merging vehicle; [0093] 3) in all the alternative vehicles, select two vehicles that are adjacent to each other as a group of guidance vehicle and auxiliary vehicle, to construct an alternative set AL of the guidance vehicle and the auxiliary vehicle of the merging vehicle; [0094] 4) use a traversal method to substitute combinations in the alternative set AL into the artificial intelligence based ramp merging multi-objective control model separately, and determine the merging vehicle, the auxiliary vehicle, and the guidance vehicle that are finally selected on the basis of a value function Q″ of the model; and [0095] 5) control and adjust, by the artificial intelligence based ramp merging multi-objective control model, accelerations of the merging vehicle and auxiliary vehicle according to the selected guidance vehicle, to solve multi-objective optimal control, so as to successfully merge the merging vehicle into a main line, and simultaneously ensuring energy consumption optimization of the merging vehicle and efficient and safe traffic of a road. Specifically, constructing a problem mathematical model and carrying out optimization solution on the basis of reinforcement learning are specifically included.

[0096] The constructing a problem mathematical model includes: [0097] 1) set a safe merging moment as t to construct a position and speed relation to be satisfied by the guidance vehicle, the auxiliary vehicle, and the ramp merging vehicle at the safe merging moment: [0098] x.sub.l, v.sub.l and a.sub.lrepresenting a position, a speed, and an acceleration of the guidance vehicle; [0099] x.sub.f, v.sub.f and a.sub.f representing a position, a speed, and an acceleration of the auxiliary vehicle; x.sub.m, v.sub.m and a.sub.m representing a position, a speed, and an acceleration of the ramp merging vehicle; τ being a constant time interval, L1 being a length of the vehicle, s.sub.0 being a pause gap, d.sub.min and d.sub.maxbeing a start point and an end point of the merging area respectively, and the merging area having a length ofd.sub.max - d.sub.min; and the equation sequentially representing, from top to bottom, that the ramp merging vehicle is at the rear of the guidance vehicle, the ramp merging vehicle is in front of the auxiliary vehicle, the ramp merging vehicle and the guidance vehicle have a consistent speed, the ramp merging vehicle and the auxiliary vehicle have a consistent speed, and the ramp merging vehicle is safely merged from the acceleration lane into the main road in the selected merging area; and [0100] 2) under the condition of satisfying 1), further construct an objective function C including, but not limited to, objectives of driving comfort, vehicle energy consumption, traffic efficiency, etc.; C.sub.n(t̂) representing cost functions of different objectives, and c.sup.n representing parameters.

[0101] As the preferred solution of the method of the present invention, the artificial intelligence based ramp merging multi-objective control model is solved by using a reinforcement learning actor-critic algorithm, which specifically includes:

[0102] 1) establish a state space

[00036]$S$

and a behavior space

[00037]$A$

: selecting, according to the state data of the guidance vehicle, the auxiliary vehicle, and the ramp merging vehicle, six-dimensional state information s = {x.sub.l, x.sub.m, x.sub.f, v.sub.l v.sub.m, v.sub.f} to represent the most relevant influence factor in an environment,

[00038]$s\in S$

, and select a control behavior strategy a = {a.sub.m, a.sub.f},

[00039]$a\in \mathcal{A}$

on the basis of a control object.

[0103] 2) establish an optimal objective: according to a vehicle constraint relation at a safe merging moment t̂, construct an optimal objective space set

[00040]$g*=\left\{g_1^{\ast },g_2^{\ast },g_3^{\ast },g_4^{\ast },g_5^{\ast }\right\}$

for ramp merging,

[00041]$g*\in \mathcal{G},\mathcal{G}$

being an objective space set,

[00042]$g_1^{\ast }$

representing that equation (1) that the merging vehicle is at the rear of the guidance vehicle is satisfied,

[00043]$g_2^{\ast }$

representing that equation (2) that the ramp merging vehicle is in front of the auxiliary vehicle is satisfied,

[00044]$g_3^{\ast }$

representing that equation (3) that the ramp merging vehicle and the guidance vehicle have a consistent speed is satisfied,

[00045]$g_4^{\ast }$

representing that equation (4) that the ramp merging vehicle and the auxiliary vehicle have a consistent speed is satisfied, and

[00046]$g_5^{*}$

representing that equation (5) that the ramp merging vehicle is safely merged from the acceleration lane into the main road in the selected merging area is satisfied;

[0104] 3) construct an objective space: on the basis of classifications included by the optimal objective space set

[00047]$g*=\left\{g_1^{\ast },g_2^{\ast },g_3^{\ast },g_4^{\ast },g_5^{\ast }\right)$

in 2), establish the objective space set g to satisfy g = {g.sub.1,g.sub.2,g.sub.3,g.sub.4,g.sub.5}, g.sub.1 representing a position relation between the ramp merging vehicle and the guidance vehicle, g.sub.2 representing a position relation between the ramp merging vehicle and the auxiliary vehicle, g.sub.3 representing a speed relation between the ramp merging vehicle and the guidance vehicle, g.sub.4 representing a speed relation between the ramp merging vehicle and the auxiliary vehicle, and g.sub.5 representing a relation between a position of the ramp merging vehicle and the merging area; and

[0105] 4) construct rewards: with a reward function beingr:

[00048]$\mathcal{S}\times \mathcal{A}\times \mathcal{G}.fwdarw.R$

, introduce, according to safe, efficient and comfortable traveling requirements, at least two short-term objective rewards under the condition that the reward r(s, a, g*) at each time step includes a long-term objective merging reward QUOTE, [0106] (1) the long-term objective merging reward R.sub.m(t) that must be included being represented as follows:, and [0107] (2) other objectives being capable of including, but not limited to, energy conservation, stability, comfort, efficiency, etc. Construction may refer to: [0108] a. construct a driving energy consumption reward R.sub.e(t): [0109] in consideration of battery efficiency and energy consumption, use an ampere-time integration method to construct a descent relation model of vehicle state of charge (SOC): [0110] V.sub.ocbeing open circuit voltage, R.sub.int being a resistance, P.sub.b(t) being battery power at a time t, and Q.sub.c being battery capacity.

[0111] Therefore, an energy-saving reward includes SOC situations of the ramp merging vehicle and the auxiliary vehicle as follows;

[00051]$\begin{array}{cc}{R}_e\left(t\right)=\Delta SO{C}_m\left(t\right)+\Delta SO{C}_f\left(t\right)& \text{­­­(9)}\end{array}$

[0112] b. construct a battery life reward R.sub.b(t) according to

[00052]$\begin{array}{cc}\Delta SOH\left(t\right)=\frac{1}{2N\left(\left|P_b\left(t\right)\right|\right)E_0\left(0\right)}{\displaystyle {\int }_0^t\left|P_b\left(t\right)\right|dt}& \text{­­­(10)}\end{array}$

[0113] N being the number of battery cycles, and E.sub.0(0) being standard battery capacity energy.

[0114] Therefore, the battery life reward is state of health (SOH) situations of the ramp merging vehicle and the auxiliary vehicle as follows: .

[00053]$\begin{array}{cc}{R}_b\left(t\right)=\Delta SO{H}_m\left(\text{t}\right)+\Delta SO{H}_f\left(\text{t}\right)& \text{­­­(11)}\end{array}$

[0115] c. construct a stability reward R.sub.s(t): without considering lateral movement of the vehicle, see the stability reward as a change of an actual acceleration, the maximum acceleration being a.sub.max = 3 m/s.sup.2, and a specific model is as follows: . [0116] d. construct a comfort reward construction R.sub.p(t): in order to reduce bump of the merging vehicles and improve comfort of passengers, j.sub.max representing a maximum allowable bump value, m/s.sup.3 being taken as the unit, and the model is as follows: .

[0117] A coupled Chebyshev based multi-objective reward optimization method specifically includes: [0118] a. determine optimized reward terms: under the condition of assuming that the number of a variety of introduced real-time short-term objectives after successful merging is n.sub.r, determine the number of the optimized reward terms for a merging problem as n.sub.r; [0119] b. determine a super-ideal optimal value of each reward: construct the super-ideal optimal value of each objective, being an ideal value, and ϑ.sub.i being a constant according to selection of empirical data, and representing a degree that the super-ideal optimal value is better than the ideal value; c. construct a generalized weighted Chebyshev optimization model for a multi-objective problem: with setting λ.sub.i as a Chebyshev weight of a short-term objective, transform the multi-objective problem into a generalized weighted Chebyshev multi-objective problem as follows:

[0120] The following condition being satisfied:

[00059]$F_c-{\lambda }_i\left(r_i{}^{**}-r_i\right)\ge 01\le i\le n_r$

[00060]$r_i\text{being}\text{an}\text{unrestricted}\text{variable}\text{1}\le i\le n_r\left(15\right),$

[00061]$F_c\ge 0$

[0121] F.sub.cbeing a weighted Chebyshev norm,

[00062]$F_c=\max \left\{{\lambda }_i\left(r_i{}^{**}-r_i\right)\right\}$

representing a maximum deviation between each objective and the super-ideal optimal value; and

[00063]$\rho {\displaystyle {.Math.}_{i=1}^{n_r}\left(r_i^{**}-r_i\right)}$

being a term ensuring the stable algorithm, usually, ρ = 0.001, and a computation equation of λ.sub.i being referred to as follows:

[00064]$\begin{array}{cc}{\lambda }_i=\left\{\begin{array}{ll}\frac{1}{\left(r_i^{\ast \ast }-r_i\right)}{\left[{\displaystyle {.Math.}_{i=1}^{n_r}\frac{1}{\left(r_i^{\ast \ast }-r_i\right)}}\right]}^{-1}\hfill & r_i\text{­­­(16)}\ne r_i^{\ast \ast }\hfill \\ 1\hfill & r_i=r_i^{\ast \ast }\hfill \\ 0\hfill & r_i\ne r_i^{\ast \ast },\text{but}\exists i,\text{and}{r}_i=r_i^{\ast \ast }\hfill \end{array}\right)& \end{array}$

[0122] c. construct a reward function as:

[00065]$\begin{array}{cc}{R}_m\left(t\right)=\left\{\begin{array}{c}1,\text{safe}\text{successful}\text{merging}\\ 0,\text{the}\text{merging}\text{vehicle}\text{can}\left(\text{may}\right)\text{be}\text{merged,}\text{but}{g}^{*}isnotsatisfied.\\ -1,\text{others}\end{array}\right)& \text{­­­(17)}\end{array}$

[0123] 5) obtain a data link: according to state, objective, strategy and reward data of the t-th time step obtained from 1) to 4), obtain the data link (s.sub.t||g*, a, r, s.sub.t+1 ||g*), and store the data into an intelligent optimization module, s||g* representing a connection between a state s and an objective g*, and r not including the short-term objective R.sub.e(t), R.sub.s(t) when g* (Rm(t) being valued as 1, safe successful merging) is not completed. connection between a state s and an objective g*, and r not including the short-term objective R.sub.e(t), R.sub.s(t) when g* (Rm(t) being valued as 1, safe successful merging) is not completed.

[0124] 6) expand data exploration, and optimize the objective space: according to 4) and 5), it may be found that data satisfying g* is difficult to obtain, because g* is a fixed final objective and has extremely limited guidance significance for the real-time rewards. Therefore, a multi-experience replay based virtual objective construction algorithm is further provided, to introduce virtual objectives, to optimize the objective space, and simultaneously expand the data exploration. The provided multi-experience replay based virtual objective construction algorithm includes: [0125] (1) construct multi-experience virtual objectives and optimize the objective space at a t time step: on the basis of the objective g* and the data link (s.sub.t||g*, a, r, s.sub.t+1||g*) at the t time step, construct a fully connected neural network H, g = H.sub.φ(s, g*) taking φ as a parameter, to obtain l virtual objectives in a current state: g.sup.1=Hφ(s.sub.t,g*), g.sup.2=H.sub.φ(s.sub.t+1, g*), ..., g.sup.l=H.sub.φ(s.sub.t+l-1,g*); [0126] (2) optimize data exploration on the basis of the optimized objective space: construct l virtual objective data links on the basis of the obtained l virtual objectives: store the virtual objective data links into the intelligent optimization module; [0127] (3) optimize data exploration at a full time step: repeat (1) and (2) for each time step to complete data exploration and objective space optimization at all time steps; [0128] (4) select virtual objective effectiveness on the basis of an artificial intelligence model: [0129] train the artificial intelligence model according to an optimized data set, and select, [0130] according to a training result, an optimal virtual objective data link at each time step; and [0131] (5) verify the fully connected neural network H for the virtual objectives: according to the optimal virtual objective data link at each time step, verify the parameter φ of the fully connected neural network H for the virtual objectives, to constantly improve accuracy of generation of the virtual objectives, and improve a training speed of the algorithm.

[0132] in each time step, according to the data link stored by the intelligent optimization module, on the basis of an actor-critic algorithm framework, train a merging control strategy by means of a deep neural network taking θ.sub.A as a parameter, the strategy directly outputting an action to control acceleration magnitudes and states of the ramp merging vehicle and the auxiliary vehicle and objective inputs, strategy optimization aiming at to find an optimal behavior strategy a, to maximize a return expectation of a whole trip, which specifically includes: [0133] (1) according to a Bellman function, construct an optimal value function as: [0134] Q.sub.π being a value function, γ being a loss factor, s, a, g being a state, a behavior strategy, and an objective at a current time step respectively, and s′, a′, g′ being a state, a behavior strategy, and an objective at a next time step respectively: [0135] (2) according to (1), determine a standard time difference update equation as: [0136] ζ being a learning rate; and [0137] utilize a critic network taking θ.sub.c as a parameter to estimate a Q value according to an update rule of equation (16), so as to construct a loss function of the critic network as follows: [0138] θ.sub.c being a critic network parameter; [0139] (3) utilize a stochastic gradient descent algorithm to update a critical network parameter by minimizing the loss function as follows: [0140] ∇ representing a gradient; and [0141] (4) output an optimal control strategy under the driving state by means of forward delivery of a trained network:

[0142] obtain the optimal control strategy, to achieve optimal control of the highway scenario oriented multi-objective optimization control method and system for cooperative ramp merging of connected vehicles on a highway by means of each module, thereby achieving efficient, safe, and energy-saving driving in the ramp area.

[0143] What is described above is merely the preferred example of the present invention, and is not intended to limit the present invention in any form. Any amendments or equivalent changes made according to the technical essence of the present invention still fall within the scope of the present invention.