Sensing method and sensing device for human driven vehicles under partial VANET (vehicular ad hoc network) environment

11634144 · 2023-04-25

Assignee

Inventors

Cpc classification

International classification

Abstract

A sensing method and a sensing device for HDVs (human driven vehicles) under a partial VANET environment are provided. According to the method, an existence sensing module and a location sensing module are constructed, based on a long-short-term-memory recurrent neural network, with utilizing historical information of motion states of a single CAV (connected and autonomous vehicle) as an input, existence and exact locations of surrounding HDVs of the CAV are outputted. The method is not only applicable to sensing the surrounding HDVs of the single CAV, but also the surrounding HDVs of the multiple CAVs. An estimation result of each CAV is firstly obtained, then the estimation results of the CAVs are checked with confliction criterion, according to checking results, the estimation results of the multiple CAVs are fused, and information about the surrounding HDVs of each CAV is finally outputted.

Claims

1. A sensing method for HDVs (human driven vehicles) under a partial VANET (vehicular ad hoc network) environment comprises steps of: constructing an existence sensing module and a location sensing module, wherein: both of the existence sensing module and the location sensing module adopt state information of a CAV (connected and autonomous vehicle) as an input an output of the existence sensing module is existence of surrounding HDVs of the CAV; and an output of the location sensing module is probability density distributions of distances between the CAV and the surrounding HDVs; for a scenario that a single CAV exists, adopting a first sensing procedure, specifically comprising steps of: estimating existence of each surrounding HDV of the CAV with the existence sensing module; estimating a location of each surrounding HDV with the location sensing module; continuously conducting the first sensing procedure, so as to output estimation results of the surrounding HDVs of the CAV in real-time; for a scenario that multiple CAVs exist, adopting a second sensing procedure, specifically comprising steps of: applying the first sensing procedure to each CAV, so as to obtain existence and location information of surrounding HDVs of each CAV; checking whether there is a confliction among the estimation results from the CAVs with confliction criterion; if the confliction exists, fusing conflicted estimation results, and obtaining final existence and location information of each HDV, wherein: the existence sensing module is constructed based on a LSTM RNN (long-short-term-memory recurrent neural network), comprising two layers, wherein: a first layer is a LSTM layer; an input of the first layer is historical motion information of the CAV, comprising lateral displacement, longitudinal displacement, normalized speed and acceleration; a sequence length of the first layer is set manually; because state decision of the CAV needs to consider five vehicles, respectively an ego-lane leader, a left-lane leader, a left-lane follower, a right-lane leader, and a right lane follower of the CAV, an output of the first layer only adopts the last five data; and a second layer is a sigmoid layer, for mapping the output of the first layer to an interval of [0,1]; and an output of the second layer is five scalars, respectively representing existence probabilities of the HDVs in five surrounding directions of the CAV.

2. A sensing method for HDVs (human driven vehicles) under a partial VANET (vehicular ad hoc network) environment comprises steps of: constructing an existence sensing module and a location sensing module, wherein: both of the existence sensing module and the location sensing module adopt state information of a CAV (connected and autonomous vehicle) as an input an output of the existence sensing module is existence of surrounding HDVs of the CAV; and an output of the location sensing module is probability density distributions of distances between the CAV and the surrounding HDVs; for a scenario that a single CAV exists, adopting a first sensing procedure, specifically comprising steps of: estimating existence of each surrounding HDV of the CAV with the existence sensing module; estimating a location of each surrounding HDV with the location sensing module; continuously conducting the first sensing procedure, so as to output estimation results of the surrounding HDVs of the CAV in real-time; for a scenario that multiple CAVs exist, adopting a second sensing procedure, specifically comprising steps of: applying the first sensing procedure to each CAV, so as to obtain existence and location information of surrounding HDVs of each CAV; checking whether there is a confliction among the estimation results from the CAVs with confliction criterion; if the confliction exists, fusing conflicted estimation results, and obtaining final existence and location information of each HDV, wherein: the location sensing module is constructed based on a LSTM RNN (long-short-term-memory recurrent neural network), comprising three layers, wherein: a first layer is a LSTM layer; an input of the first layer is historical motion information of the CAV, comprising lateral displacement, longitudinal displacement, normalized speed and acceleration; a second layer is a feed forward layer; the second layer adopts an output of the first layer as an input; and an output of the second layer is five classes of variables, respectively corresponding to five surrounding directions of the CAV; and a third layer is a mixture density layer, which constructs a probability density distribution function of locations of the HDVs based on a GMM (Gaussian mixture model) with utilizing the output of the second layer, and obtains estimates.

3. A sensing method for HDVs (human driven vehicles) under a partial VANET (vehicular ad hoc network) environment comprises steps of: constructing an existence sensing module and a location sensing module, wherein: both of the existence sensing module and the location sensing module adopt state information of a CAV (connected and autonomous vehicle) as an input an output of the existence sensing module is existence of surrounding HDVs of the CAV; and an output of the location sensing module is probability density distributions of distances between the CAV and the surrounding HDVs; for a scenario that a single CAV exists, adopting a first sensing procedure, specifically comprising steps of: estimating existence of each surrounding HDV of the CAV with the existence sensing module; estimating a location of each surrounding HDV with the location sensing module; continuously conducting the first sensing procedure, so as to output estimation results of the surrounding HDVs of the CAV in real-time; for a scenario that multiple CAVs exist, adopting a second sensing procedure, specifically comprising steps of: applying the first sensing procedure to each CAV, so as to obtain existence and location information of surrounding HDVs of each CAV; checking whether there is a confliction among the estimation results from the CAVs with confliction criterion; if the confliction exists, fusing conflicted estimation results, and obtaining final existence and location information of each HDV, wherein the confliction criterion is: if two CAVs simultaneously satisfy following three rules, a confliction exists between estimation results of the two CAVs, wherein: a first rule is that the two CAVS are separated by at most one lane; a second rule is that a longitudinal distance between the two CAVs is smaller than a distance threshold; and a third rule is that the two CAVs and surrounding HDVs thereof are probabilistic close; a meaning of probabilistic close is defined as follows: { mi n ( x ^ M EV 1 .Math. x ^ N EV 2 ) ma x ( x ^ M EV 1 .Math. x ^ N EV 2 ) p M EV1 ( x ^ M EV 1 - x ) dx p = mi n ( x ^ M EV 1 .Math. x ^ N EV 2 ) ma x ( x ^ M EV 1 .Math. x ^ N EV 2 ) p N EV 2 ( x ^ N EV 2 - x ) dx p = wherein: EV1 and EV2 respectively represent the two CAVs; M.sub.EV1 represents any one of surrounding HDVs of EV1; N.sub.EV2 represents any one of surrounding HDVs of EV2; x ^ M EV 1 and x ^ N EV 2 respectively represent location estimates of M.sub.EV1 and N.sub.EV2; p M EV 1 ( x ) and p N EV 1 ( x ) respectively represent probability density distributions of M.sub.EV1 and N.sub.EV2; and p is a set threshold.

4. The sensing method, as recited in claim 3, wherein: when estimation results of N CAVs are conflicted, one result with maximum probability is selected from 1+N+1 possible results as a final estimation result; the 1+N+1 possible results are respectively: (1) maintaining all of the estimation results of the N CAVs; (2) maintaining an estimation result of one CAV, and discarding estimation results of the other CAVs, totally N possibility; and (3) weighting probability densities in the estimation results of the N CAVs, and obtaining a final probability density and location estimate of each HDV.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

(1) The patent of application file contains at least one drawing executed in color. Copies of this patent or patent application publication with color drawing(s) will be provided by the Office upon request and payment of the necessary fee.

(2) FIG. 1(a) is a sketch view of an application scenario of the present invention.

(3) FIG. 1(b) is a sketch view of typical devices required by a CAV (connected and autonomous vehicles).

(4) FIG. 2(a) shows a motion trajectory of an EV (ego vehicle).

(5) FIG. 2(b) shows motion trajectories of an ego-lane leader, a left-lane follower and a right-lane follower of the EV.

(6) FIG. 2(c) is a sketch view of the EV and five surrounding HDVs (human driven vehicles).

(7) FIG. 3(a) is a structural sketch view of an existence sensing module according to the present invention.

(8) FIG. 3(b) is a sketch view of an estimation result of the existence sensing module.

(9) FIG. 4 is a structural sketch view of a location sensing module according to the present invention.

(10) FIG. 5 is a sketch view of an estimation result of the location sensing module.

(11) FIG. 6 is a flow chart of a first sensing procedure (P1) when a single CAV exists.

(12) FIG. 7 is a flow chart of a second sensing procedure (P2+) when multiple CAVs exist.

(13) FIG. 8 shows a meaning of a confliction between two CAVs.

(14) FIG. 9(a) shows conflictions between two CAVs.

(15) FIG. 9 (b) shows conflictions among three CAVs.

(16) FIG. 10 shows a meaning of probabilistic close.

(17) FIG. 11 shows four possible results when estimation results of two CAVs are conflicted.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

(18) The present invention is further illustrated with accompanying drawings. The embodiments described below are only preferred embodiments of the present invention and the present invention is not limited thereto. For one of ordinary skill in the art, variants made without departing from the present invention should be all encompassed in the protection scope of the present invention.

(19) FIG. 1(a) is a sketch view of an application scenario of the present invention, and FIG. 1(b) is a sketch view of typical devices required by a CAV (connected and autonomous vehicles). In FIG. 1(a), a road segment with multiple lanes is shown. In a traffic flow, there are two types of vehicles. The first type of vehicles is the CAV 1, which is equipped with three required types of devices. The first type of devices is locating devices 3; a common locating device is a GPS (global position system); with utilizing the locating devices, the coordinate of the CAV at any moment can be obtained. The second type of devices is communication devices 5; with utilizing the communication devices, the CAV can communicate with other vehicles or a road side unit 6 (shown in FIG. 1(a)). The third type of devices is recording devices 4, for storing historical information of the CAV, comprising lateral displacement, longitudinal displacement, speed and acceleration. Moreover, the second type of vehicles in the traffic flow is the HDV (human driven vehicle) 2, which is not equipped with the communication devices and cannot exchange information with the outside world. Therefore, it is unable to obtain the information of the HDV, such as location and existence.

(20) A purpose of the present invention is to estimate and obtain the existence and exact locations of the HDVs, if they exist. The sensing method can be deployed in two ways. The first way is to deploy the sensing method inside each CAV, so that the CAVs can exchange the historical motion state information and cooperatively accomplish the task of sensing the HDVs. The second way is to deploy the sensing method inside the road side unit 6 shown in FIG. 1(a). The road side unit collects the historical information of all the CAVS within the communication range and senses the surrounding HDVs of these CAVs.

(21) FIG. 2(a) shows a motion trajectory of one CAV, namely EV (ego vehicle). The color of the trajectory indicates the instantaneous speed. The warmer color indicates the lower speed, while the cooler color indicates the higher speed. The x-axis represents the lateral distance, while the y-axis represents the longitudinal distance. It can be concluded from the motion trajectory and the speed that: the CAV firstly slows down, then changes the lane, and accelerates thereafter. It can be conjectured that: the reasons why the CAV changes the lane are the ego-lane leader has the lower speed and the left lane provides the enough large space. Thus, it can be inferred that: the ego-lane leader is closer; the left-lane leader is farther; and the left-lane follower is also farther. FIG. 2(b) shows motion trajectories of the ego-lane leader, the left-lane follower and the right-lane follower of the CAV, which proves the above inference. It is intuitive from FIG. 2(a) and FIG. 2(b) that the trajectory-related information can be used to estimate the existence and exact locations of the surrounding vehicles of the CAV. Generally, the state decision of the CAV needs to consider five surrounding vehicles, as shown in FIG. 2(c). In FIG. 2(c), EV represents the ego vehicle; EL represents the ego-lane leader; LL represents the left-lane leader; LF represents the left-lane follower; RL represents the right-lane leader; and RF represents the right-lane follower. The present invention utilizes the information of the CAV to estimate the existence and locations of the HDVs in five surrounding directions of the CAV.

(22) According to the sensing method of the present invention, the existence sensing module and the location sensing module are firstly constructed. The existence sensing module is constructed based on a LSTM RNN (long-short-term-memory recurrent neural network), as shown in FIG. 3(a), comprising two layers, wherein:

(23) a first layer is a LSTM layer; an input of the first layer is a sequence with a length of K; an input of a first LSTM cell is:

(24) [ δ x δ y v v max α ] t - K + 1

(25) wherein: the input is a vector with a length of 4; the subscript t−K+1 represents a moment; t represents a current moment; δx and δy respectively represent the lateral displacement and the longitudinal displacement of the

(26) CAV ; v v max
represents the normalized speed; v represents the real-time speed of the CAV; v.sub.max represents the road speed limit; and a represents the acceleration;

(27) a second layer is a sigmoid layer, for mapping the output of the first layer to an interval of [0,1]; an output of the second layer is five scalars, respectively

(28) ϕ EL EV , ϕ LL EV , ϕ LF EV , ϕ RL EV , and ϕ RF EV
shown in FIG. 3(b), representing the existence probabilities of the HDVs in the five surrounding directions of the CAV; the subscript EV represents the ego vehicle; the subscript EL represents the ego-lane leader of the EV; the subscript LL represents the left-lane leader of the EV; the subscript LF represents the left-lane follower of the EV; the subscript RL represents the right-lane leader of the EV; the subscript RF represents the right-lane follower of the EV; if the scalar is larger than 0.5, the HDV in the corresponding direction exists; conversely, the HDV does not exist.

(29) The location sensing module is constructed based on the LSTM RNN, as shown in FIG. 4, comprising three layers; wherein:

(30) a first layer is for processing a sequence input; a second layer is for calculating parameters of Gaussian mixture density; and a third layer is for calculating a final value of the Gaussian mixture density;

(31) the first layer is a common LSTM layer, which is same as the first layer of the existence sensing module;

(32) the second layer is a common feed forward layer; an output of the second layer is five classes of variables, respectively corresponding to the five surrounding directions of the CAV, expressed as:

(33) { EL : z EL EV β 1 , z EL EV β 2 , .Math. z EL EV β k z EL EV μ 1 , z EL EV μ 2 , .Math. z EL EV μ k and ( a ) z EL EV σ 1 , z EL EV σ 2 , .Math. z EL EV σ k LL : z LL EV β 1 , z LL EV β 2 , .Math. z LL EV β k z LL EV μ 1 , z LL EV μ 2 , .Math. z LL EV μ k and ( b ) z LL EV σ 1 , z LL EV σ 2 , .Math. z LL EV σ k LF : z LF EV β 1 , z LF EV β 2 , .Math. z LF EV β k z LF EV μ 1 , z LF EV μ 2 , .Math. z LF EV μ k and ( c ) z LF EV σ 1 , z LF EV σ 2 , .Math. z LF EV σ k RL : z RL EV β 1 , z RL EV β 2 , .Math. z RL EV β k z RL EV μ 1 , z RL EV μ 2 , .Math. z RL EV μ k and ( d ) z RL EV σ 1 , z RL EV σ 2 , .Math. z RL EV σ k RF : z RF EV β 1 , z RF EV β 2 , .Math. z RF EV β k z RF EV μ 1 , z RF EV μ 2 , .Math. z RF EV μ k and ( e ) z RF EV σ 1 , z RF EV σ 2 , .Math. z RF EV σ k ( 1 )

(34) if the Gaussian mixture distribution has k mixtures (see equation (2)), the number of the outputted variables of the second layer in the equation (1) is k*3*5; “3” represents weight, mean value and standard deviation; “5” represents the five surrounding directions of the CAV; namely five classes; in each class of variables, the superscripts β, μ and σ is respectively represent the weight, mean value and standard deviation of the Gaussian mixture density;

(35) the third layer is a mixture density layer, which constructs a probability density distribution function of locations of the HDVs based on a GMM (Gaussian mixture model) with utilizing the output of the second layer (the output of the third layer is five probability density functions, as shown in FIG. 5); M is assumed as the HDV in one surrounding direction of the EV, and the probability density of the location of M is expressed as:

(36) p M EV ( x ) = .Math. j = 1 k β M EV , j N ( x ; μ M EV , j σ M EV , j ) M { EL , LL , LF , RL , RF } ( 2 )

(37) wherein: k is the number of mixtures in the GMM;

(38) N ( x ; μ M EV , j , σ M EV , j )
represents a Gaussian distribution with the mean value of μ.sub.M.sub.EV,.sub.j and the variance of

(39) 0 σ M EV , j ; β M EV , j
represents the weight of a j.sup.th Gaussian distribution,

(40) μ M EV , j
represents the mean value of the j.sup.th Gaussian distribution, and

(41) σ M EV , j
represents the variance of the j.sup.th Gaussian distribution, which are obtained through calculating the output of the second layer, expressed as:

(42) { β M EV , j = e z M β j .Math. i e z M β i , j = 1 , 2 , .Math. k , σ M EV , j = e z M σ j , j = 1 , 2 .Math. k , μ M EV , j = z z M μ j , j = 1 , 2 .Math. k , ( 3 )

(43) at the moment, the location estimate {circumflex over (x)}M.sub.EV of the vehicle M is:

(44) x ^ M EV = E ( p M EV ( x ) ) = E ( .Math. j = 1 k β M EV , j N ( x ; μ M EV , j , σ M EV , j ) ) = .Math. j = 1 k β M EV , j μ M EV , j . ( 4 )

(45) FIG. 6 is a flow chart of a first sensing procedure (named as P1) when a single CAV exists. The procedure P1 firstly collects the historical motion information of the CAV, then sequentially employs the existence sensing module and the location sensing module, and obtains the existence and location estimates of the surrounding HDVs of the CAV.

(46) FIG. 7 is a flow chart of a second sensing procedure (named as P2+) when multiple CAVs exist. When multiple CAVs exist, the first sensing procedure shown in FIG. 6 is applied to each CAV, so as to sense the surrounding HDVs of each CAV; thereafter, whether a confliction exists among the estimation results of the CAVs is checked.

(47) FIG. 8 shows a meaning of the confliction. In FIG. 8, there are two CAVs, respectively EV1 and EV2; an estimation result of EV1 to the left-lane leader LL.sub.EV1 is x.sub.1; an estimation result of EV2 to the right-lane leader RL.sub.EV2 is x.sub.2; the two estimation results are close, and a part of the estimation results are overlapped, so that the coexistence of LL.sub.EV1 and RL.sub.EV2 is impossible. A more realistic result is to fuse the two distribution functions P.sub.LL.sub.EV1(x) and P.sub.RL.sub.EV2(x). The method provided by the present invention is not only applicable to the confliction between the estimation results of the two CAVs. FIG. 9(a) shows conflictions between two CVAs, and FIG. 9(b) shows conflictions among three CAVs. In FIG. 9(a), RL.sub.EV1 conflicts with RF.sub.EV2; and EL.sub.EV1 conflicts with EV2. In FIG. 9(b), the estimation results of three CAVs are conflicted, namely locations of LL.sub.EV1, LF.sub.EV2, and RL.sub.Ev3 are confliced. For convenient description, Neighboring(EV) is used to represent the set containing the five surrounding vehicles of EV; ExNeighboring(EV) is used to represent the set containing the EV and the five surrounding vehicles thereof; that is to say, ExNeighboring(EV)={EV}UNeighboring(EV).

(48) The confliction criterion is used to check whether there is a confliction. If the two CAVs, EV1 and EV2, simultaneously satisfy following three rules, it is considered that a confliction exists between the estimation results of the two CAVs, wherein:

(49) a first rule is that the two CAVS are separated by at most one lane; when there are two lanes between EV1 and EV2, it is impossible that the estimation results of the two CAVs are conflicted;

(50) a second rule is that a longitudinal distance between the two CAVs is smaller than a distance threshold, namely Δx.sub.EV1,EV2(t)=|x.sub.EV1−x.sub.EV2|<x; wherein: x is a preset distance threshold; because a distance threshold that the leader will influence the follower is generally 120 meters, x can be set as 240 meters; when the second rule is satisfied, ExNeighboring(EV1) may conflict with ExNeighboring(EV2); and

(51) a third rule is that one vehicle in ExNeighboring(EV1) and one vehicle in ExNeighboring(EV2) are probabilistic close; wherein: ExNeighboring(EV) represents the set containing the EV and the HDVs in the five surrounding directions; M.sub.EV1 is assumed as one surrounding HDV of EV1; N.sub.EV2 is assumed as one surrounding HDV of EV2;

(52) x ^ M EV 1 and x ^ N EV 2
respectively represent location estimates of M.sub.EV1 and N.sub.EV2; a meaning of probabilistic close is defined as follows:

(53) { min ( x ^ M EV 1 , x ^ N EV 2 ) max ( x ^ M EV 1 , x ^ N EV 2 ) p M EV 1 ( x ^ M EV 1 - x ) dx p _ _ min ( x ^ M EV 1 , x ^ N EV 2 ) max ( x ^ M EV 1 , x ^ N EV 2 ) p N EV 2 ( x ^ N EV 2 - x ) dx p _ _ ( 5 )

(54) wherein: p is a manually set threshold and is 0.2; as shown in FIG. 10, when satisfying the above equation (5), the possibility that the estimation results of M.sub.EV1 and N.sub.EV2 are overlapped is high.

(55) FIG. 11 shows four possible results when the estimation results of the two CAVs are conflicted, wherein:

(56) a first possible result is to maintain

(57) x ^ M EV 1 and x ^ N EV 2 ,
and a corresponding probability is

(58) p M EV 1 ( x ^ M EV 1 ) * p N EV 2 ( x ^ N EV 2 ) ;
a second possible result is to maintain

(59) x ^ M EV 1
and discard

(60) 0 x ^ N EV 2 ,
and a corresponding probability is

(61) p M EV 1 ( x ^ M EV 1 ) * p M EV 1 ( x ^ N EV 2 ) ;

(62) a third possible result is to maintain

(63) x ^ N EV 2
and discard

(64) x ^ M EV 1 ,
and a corresponding probability is

(65) p N EV 2 ( x ^ M EV 1 ) * p N EV 2 ( x ^ N EV 2 ) ;

(66) a fourth possible result is to fuse

(67) p M EV 1 ( x ) and P N EV 2 ( x ) ,
so as to form a new location probability density P.sub.EV1,M+EV2,N(x), and obtain a final location estimate

(68) x ^ M EV 1 + N EV 2
of the vehicle; wherein:

(69) p M EV 1 + N EV 2 ( x ) = ϕ M EV 2 ϕ M EV 1 + ϕ N EV 2 p M EV 1 ( x ) + ϕ N EV 2 ϕ M EV 1 + ϕ N EV 2 p N EV 2 ( x ) ( 6 )

(70) ϕ M EV 1 and ϕ N EV 2
are output results of the existence sensing module;

(71) x ^ M EV 1 + N EV 2 = ϕ M EV 1 x ^ M EV 1 + ϕ N EV 2 x ^ N EV 2 ϕ M EV 2 + ϕ N EV 2 ( 7 )

(72) a corresponding probability of the fourth possible result is

(73) 0 p M EV 1 + N EV 2 ( x ^ N EV 2 ) * p M EV 1 + N EV 2 ( x ^ M EV 1 ) .

(74) Among the four possible results, one result with maximum probability is selected as a final estimation result, and a final output is shown in FIG. 11.

(75) Similarly, when the estimation results of the N CAVs are conflicted, one result with the maximum probability is selected from 1+N+1 possible results as the final estimation result, wherein the 1+N+1 possible results are respectively:

(76) (1) maintaining all of the estimation results of the N CAVs;

(77) (2) maintaining the estimation result of one CAV, and discarding the estimation results of the other CAVs, totally N possibility; and

(78) (3) weighting the probability densities in the estimation results of the N CAVs, and obtaining the final probability density and location estimate of each HDV.