Lidar based Road Condition Estimation for Passenger Vehicles

Abstract

A method performed by a vehicle system for enabling estimation of a condition of a road comprising obtaining, from a sensor mounted on a vehicle, first data comprising a first plurality of data points, each comprising longitudinal, lateral and vertical coordinates representing dimensions of a part of the road at a first time, obtaining a second plurality of data points, each comprising longitudinal, lateral and vertical coordinates representing dimensions of the part of the road at a second time, wherein the first time is more recent than the second time, segmenting the first plurality of data points into a plurality of segmented data areas based on longitudinal and lateral coordinates of the first plurality of data points, and estimating a respective vertical position in each segmented data area based on vertical coordinates of the second plurality of data points and a motion state of the vehicle at the second time.

Claims

1. A method performed by a vehicle system for enabling estimation of a condition of a road, the method comprising: Obtaining, from a sensor mounted on a vehicle, first data comprising a first plurality of data points, wherein each of the first plurality of data points comprises longitudinal, lateral and vertical coordinates representing respective dimensions of a part of the road at a first time; Obtaining a second plurality of data points, wherein each of the second plurality of data points comprises longitudinal, lateral and vertical coordinates representing respective dimensions of the part of the road at a second time, wherein the first time is more recent than the second time; Segmenting the first plurality of data points into a plurality of segmented data areas based on the longitudinal and lateral coordinates of the first plurality of data points by dividing a representation of the part of the road into the plurality of segmented data areas and assigning each of the first plurality of data points to a segmented data area based on the longitudinal and lateral coordinates of the first plurality of data points; and estimating a respective vertical position in each segmented data area based on the vertical coordinates of the second plurality of data points and a motion state of the vehicle at the second time, wherein the motion state of the vehicle comprises at least one of a longitudinal velocity, a lateral velocity, a vertical velocity, a roll angle, a pitch angle and a yaw angle of the vehicle.

2. The method according to claim 1, further comprising estimating the condition of the road based on the estimated respective vertical positions.

3. The method according to claim 2, further comprising calculating a mean of the vertical coordinates of the first plurality of data points for each segmented data area based on vertical coordinates of the first plurality of data points and wherein the condition of the road is estimated based on the calculated means.

4. The method according to claim 1, wherein the sensor comprises one or more Light Detection and Ranging, LIDAR, sensors.

5. The method according to claim 1, wherein the motion state of the vehicle is estimated based on sensor data received from one or more motion tracking sensor and/or rotation wheel speed sensors of the vehicle.

6. The method according to claim 1, wherein the sensor data is received from an inertial measurement unit, IMU, and/or from a wheel speed sensor, WSS.

7. The method according to claim 2, wherein the condition of the road comprises at least one of a curvature of the road, an inclination of the road, a banking of the road, an anomaly of the road, and a smoothness of the road, wherein the anomaly of the road comprises at least one of a road bump, an undulation, a pothole and a manhole cover.

8. The method according to claim 2, further comprising adjusting a regenerative braking force of the vehicle based on the estimated condition and/or adjusting a speed profile based on the estimated condition.

9. The method according to claim 2, further comprising adjusting a steering, a suspension and/or a speed of the vehicle based on the estimated condition.

10. The method according to claim 2, further comprising obtaining second data indicative of a steering angle to estimate segmented data areas among the plurality of segmented data areas where the vehicle might travel and wherein estimating the condition of the road is further based on the estimated segmented areas.

11. A vehicle system comprising a memory, and a controller configured to: Obtain, from a sensor mounted on a vehicle, first data comprising a first plurality of data points, wherein each of the first plurality of data points comprises longitudinal, lateral and vertical coordinates representing respective dimensions of a part of the road at a first time; Obtain a second plurality of data points, wherein each of the second plurality of data points comprises longitudinal, lateral and vertical coordinates representing respective dimensions of the part of the road at a second time, wherein the first time is more recent than the second time; Segment the first plurality of data points into a plurality of segmented data areas based on the longitudinal and lateral coordinates of the first plurality of data points by dividing a representation of the part of the road into the plurality of segmented data areas and assigning each of the first plurality of data points to a segmented data area based on the longitudinal and lateral coordinates of the first plurality of data points; and Estimate a respective vertical position in each segmented data area based on the vertical coordinates of the second plurality of data points and a motion state of the vehicle at the second time, wherein the motion state of the vehicle comprises at least one of a longitudinal velocity, a lateral velocity, a vertical velocity, a roll angle, a pitch angle and a yaw angle of the vehicle.

12. The vehicle system according to claim 11, wherein the controller is further configured to estimate the condition of the road based on the estimated respective vertical positions.

13. The vehicle system according to claim 12, wherein the controller is further configured to calculate a mean of the vertical coordinates of the first plurality of data points for each segmented data area based on vertical coordinates of the first plurality of data points and wherein the condition of the road is estimated based on the calculated means.

14. The vehicle system according to claim 11, wherein the sensor comprises one or more Light Detection and Ranging, LIDAR, sensors.

15. The vehicle system according to claim 11, wherein the motion state of the vehicle is estimated based on sensor data received from one or more motion tracking sensor and/or rotation wheel speed sensors of the vehicle.

16. The vehicle system according to claim 11, wherein the sensor data is received from an inertial measurement unit, IMU, and/or from a wheel speed sensor, WSS.

17. The vehicle system according to claim 11, wherein the condition of the road comprises at least one of a curvature of the road, an inclination of the road, a banking of the road, an anomaly of the road, and a smoothness of the road, wherein the anomaly of the road comprises at least one of a road bump, an undulation, a pothole and a manhole cover.

18. The vehicle system according to claim 11, wherein the controller is further configured to adjust a regenerative braking force of the vehicle based on the estimated condition and/or adjust a steering, a suspension and/or a speed of the vehicle based on the estimated condition and/or adjust a speed profile based on the estimated condition.

19. The vehicle system according to claim 11, wherein the controller is further configured to obtain second data indicative of a steering angle to estimate segmented data areas among the plurality of segmented data areas where the vehicle might travel and wherein estimating the condition of the road is further based on the estimated segmented areas.

20. A vehicle comprising the vehicle system according to claim 11.

Description

BRIEF DESCRIPTION OF DRAWINGS

[0026] FIG. 1 illustrates a block diagram of a vehicle system according to the disclosure.

[0027] FIG. 2 illustrates a flowchart schematically depicting a method for enabling estimation of a condition of a road according to the disclosure.

[0028] FIGS. 3A and 3B illustrate schematically a vehicle and how to estimate a road inclination according to an embodiment of the disclosure.

[0029] FIG. 4 illustrates schematically a vehicle and how to estimate a road banking according to an embodiment of the disclosure.

[0030] FIGS. 5A and 5B illustrate schematically a vehicle and how to estimate a pothole or a road bump according to an embodiment of the disclosure.

[0031] FIG. 6 illustrates schematically an initialization of the segmented data areas representing a road according to an embodiment of the disclosure.

[0032] FIGS. 7, 8 and 9 illustrate schematically the segmented data areas with current and previous detected data.

DESCRIPTION OF EMBODIMENTS

[0033] FIG. 1 illustrates a block diagram of a vehicle system 102 according to the disclosure. The vehicle system 102 may comprise a processing unit 104, a memory 106, a sensor unit 108, and a communication unit 110. The processing unit 104 is connected to the memory 106, the sensor unit 108, and the communication unit 110. The communication unit 110 may transmit and receive information via Bluetooth, through a wireless communication network, for example, a 4G network, and/or using any other suitable communication technology. The sensor unit 108 may comprise sensors such as Lidar, IMU (accelerometer & gyroscope), wheel speed sensors, steering wheel sensor, etc.

[0034] FIG. 2 illustrates a flowchart schematically depicting a method a method for enabling estimation of a condition of a road according to the disclosure. The method of FIG. 2 may be performed by the vehicle system 102 of FIG. 1.

[0035] In step 202 of the method, the vehicle system obtains, from a sensor mounted on a vehicle, first data comprising a first plurality of data points. Each of the first plurality of data points comprises longitudinal, lateral and vertical coordinates representing respective dimensions of a part of the road at a first time. The sensor may be a three-dimensional (3D) sensor. For instance, the 3D sensor may be a lidar sensor which is part of the sensor unit 108. The lidar sensor may scan the road extending in front of the vehicle and generate the first plurality of data points.

[0036] In step 204 of the method, the vehicle system obtains a second plurality of data points, wherein each of the second plurality of data points comprises longitudinal, lateral and vertical coordinates representing respective dimensions of the part of the road at a second time, and wherein the first time is more recent than the second time. The second plurality of data points may be stored in a memory or in a data base that the vehicle system can access.

[0037] In step 206, the first plurality of data points is segmented into a plurality of segmented data areas or grids based on the longitudinal and lateral coordinates of the first plurality of data points.

[0038] In step 208, a respective vertical position is estimated in each segmented data area based on the vertical coordinates of the second plurality of data points and a motion state of the vehicle at the second time. The motion state of the vehicle may be estimated based on sensor data received from one or more motion tracking sensor and/or rotation wheel speed sensors of the vehicle. The sensor data may be received from the IMU, and/or from the WSS of the sensor unit 108. The motion state of the vehicle may comprise at least one of a longitudinal velocity, a lateral velocity, a roll angle, a pitch angle and a yaw angle of the vehicle.

[0039] The method of FIG. 2 may further comprise, after step 208, another step of estimating the condition of the road based on the estimated respective vertical positions.

[0040] The condition of the road may comprise at least one of a curvature of the road, an inclination of the road as shown in FIGS. 3A and 3B, a banking of the road as shown in FIG. 4, an anomaly of the road, and a smoothness of the road, wherein the anomaly of the road may comprise at least one of a road bump 502 as shown in FIG. 5B, an undulation, a pothole 500 as shown in FIG. 5A and a manhole cover. The smoothness of the road, also called pavement smoothness or roughness of the road, is a measure of minute elevation changes in the pavement surface that indicates the drivers' comfort. It may be estimated using a pavement profile (a measure of minute elevation changes in the pavement surface). Pavement profiles can be visualized as imaginary lines drawn along the surfaces of the pavement. Normally, pavement profiles are measured either longitudinally (down the road) or transversely (across the road).

[0041] In an alternative embodiment, the method of FIG. 2 may comprise calculating a mean of the vertical coordinates of the first plurality of data points for each segmented data area based on vertical coordinates of the first plurality of data points after step 206. In this alternative embodiment, step 208 may comprise estimating the condition of the road based also on the calculated means.

[0042] The method of FIG. 2 may further comprise adjusting a regenerative braking force of the vehicle based on the estimated condition, and/or adjusting a steering, a suspension and/or a speed of the vehicle based on the estimated condition, and/or adjusting a speed profile based on the estimated condition.

[0043] Furthermore, the method may also comprise obtaining second data indicative of a steering angle to estimate segmented data areas among the plurality of segmented data areas where the vehicle might travel and wherein estimating the condition of the road is further based on the estimated segmented areas.

[0044] The step 206 of FIG. 2 may comprise dividing a representation of the part of the road into the plurality of segmented data areas and assigning each of the first plurality of data points to a segmented data area based on the longitudinal and lateral coordinates of the first plurality of data points.

[0045] FIG. 6 shows how a representation of a part of the road in front of the vehicle can be divided into the plurality of segmented data areas 610. The terms grids, virtual grids and segmented data areas will be used through the description to indicate the same. FIG. 6 shows the representation of a part of the road that has been divided into a plurality of 50-centimetre50-centimetre grids 610. This is just a non-limiting example and the representation of the road may be divided into the plurality of segmented data areas in any other way. The array of grids shown in FIG. 6 is represented using a system of reference having as origin the vehicle 300 as shown in FIGS. 3A-3B, 4, and 5A-5B wherein the x axis, y axis and z axis respectively represent in FIGS. 3A, 3B, 4 and 6 a longitudinal direction, a lateral direction, and a vertical direction. The longitudinal, lateral and vertical coordinates of the data points represent respective dimensions of the road in the x, y and z axis. As FIG. 6 shows a top view, the z axis is perpendicular to the array of grids. FIG. 6 also shows the front right tire position 630 and the front left tire position 620 of the vehicle 300.

[0046] The vertical coordinate can be used to determine whether a data point received from the lidar at a time t belongs to the road surface and not to a static or a dynamic object around or over the road surface. The data points with lowest values in the vertical coordinate below a certain threshold have the highest probability of belonging to the road and will form the first plurality of data points. The longitudinal and lateral coordinates of the first plurality of data points can be used then to assign each of the first plurality of data points to one of the grids 610 of FIG. 6.

[0047] A second plurality of data points is obtained, wherein the second plurality of points comprise longitudinal, lateral and vertical coordinates representing respective dimensions of the part of the road at a second time t-1, wherein the first time is more recent than the second time. The grids comprising data points that belong to the first time t and to the second time t-1 are indicated as 702 in FIGS. 7, 8 and 9, and the grids that comprise data points that belong to the first plurality of data points but do not comprise data points that belong to the second plurality of data points are indicated as 704 in FIGS. 7, 8 and 9.

[0048] As said, the motion states of the vehicle (such as longitudinal velocity, lateral velocity, vertical velocity, roll angle 430, pitch angle 330, yaw angle) for previous second time t-1 (in previous 100 milliseconds lidar measurement time stamp), are calculated by processing IMU and Wheel Speed Sensor (WSS) data.

[0049] In the following, Z.sub.t-1 is a measured mean value of the vertical coordinates of the second plurality of data points in each grid 610, Z.sub.t is the measured mean value of the vertical coordinates of the first plurality of data points in each grid 610, {circumflex over (Z)}.sub.t-1|t-1 is the updated grid value of the vertical coordinates of the second plurality of points in each grid 610, {circumflex over (Z)}.sub.t|t is the updated grid value of the vertical coordinates of the first plurality of points in each grid 610 and {circumflex over (Z)}.sub.t|t-1 is the estimated grid value of the vertical coordinate of the first plurality of points in each grid 610.

[0050] First, Z.sub.t is calculated as a mean of measured values of a segmented grid area at time t. Then {circumflex over (Z)}.sub.t|t-1 for a segmented area is calculated from updated value {circumflex over (Z)}.sub.t-1|t-1 from second time step t-1 according to below equation 1. The method proceed then to calculate an updated grid value {circumflex over (Z)}.sub.t|t as an average of values {circumflex over (Z)}.sub.t|t-1 and Z.sub.t according to below equation 2.

[0051] As said, {circumflex over (Z)}.sub.t|t-1 is calculated as follows:

[00001]$\begin{array}{cc}\left[\begin{array}{c}0\\ 0\\ {\hat{Z}}_{t.Math.t-1}\\ 1\end{array}\right]=\left(\begin{array}{cc}R& T\\ N& 1\end{array}\right)\left[\begin{array}{c}0\\ 0\\ {\hat{Z}}_{t-1.Math.t-1}\\ 1\end{array}\right]& \mathrm{Equation}1\end{array}$ [0052] wherein R=R.sub.x()R.sub.y()R.sub.z() is the 33 rotation matrix along x, y, z axis,

[00002]$T=\left(0.1*\left[\begin{array}{c}{v}_{t-1}^x\\ v_{t-1}^y\\ v_{t-1}^z\end{array}\right]\right)$

is the translation vector of size 31 in x, y, z axis, N=[0 0 0] is a zero vector of size 13, 0.1 is a constant difference in seconds between t and t-1 and may have any other value, and v.sub.t-1.sup.x, v.sub.t-1.sup.y, v.sub.t-1.sup.z are respectively the following motion states of the vehicle at the second time t-1: vehicle's longitudinal velocity along x axis, vehicle's lateral velocity along y axis and vehicle's vertical velocity along z axis respectively.

[0053] The velocities v.sub.t-1.sup.x, v.sub.t-1.sup.y, v.sub.t-1.sup.z may be obtained, for instance, by fusion of IMU and WSS sensor data. Furthermore, R.sub.x() is the rotation matrix along x axis, is the vehicle roll angle, R.sub.y() is the rotation matrix along y axis, is the vehicle pitch angle, R.sub.z() is the rotation matrix along z axis, is the vehicle pitch angle and , , and may be obtained from a gyroscope of the vehicle.

[0054] A measured mean value Z.sub.t or mean of the vertical coordinates of the first plurality of data points for each segmented data area can be calculated based on vertical coordinates of the first plurality of data points. The measured mean Z.sub.t-1 of the vertical coordinates of the second plurality of data points for each segmented data area 610 is obtained by calculating the mean of all the data points in said segmented data area 610 that belong to the second plurality of data points.

[0055] Then the grid's estimated respective vertical position {circumflex over (Z)}.sub.t|t-1 can be combined or fused with measured mean value, Z.sub.t. This combination or fusion will give a grid's updated height value {circumflex over (Z)}.sub.t|t. Equation 2 shows the simplest possible fusion or combination of {circumflex over (Z)}.sub.t|t-1 and Z.sub.t by averaging them as below:

[00003]$\begin{array}{cc}{\hat{Z}}_{t.Math.t}=\frac{1}{2}\left({\hat{Z}}_{t.Math.t-1}+Z_t\right)& \mathrm{Equation}2\end{array}$

[0056] Other popular fusion and filtering technique can also be applied there, e.g. Kalman filter.

[0057] A steering wheel sensor may provide second data indicative of a steering angle to estimate segmented data areas among the plurality of segmented data areas where the vehicle might travel and wherein calculating the condition of the road is further based on the Updated value of segmented areas. In FIGS. 7 and 8, {circumflex over (z)}.sub.t|t.sup.left,first and {circumflex over (z)}.sub.t|t.sup.right,first indicate the, updated value of height in grids where the vehicle might travel closest to the vehicle and {circumflex over (z)}.sub.t|t.sup.right,last and {circumflex over (z)}.sub.t|t.sup.left,last indicate the updated value of height in grids where the vehicle might travel farthest to the vehicle on the left and right sides respectively.

[0058] A predicted height of the road h.sub.road ahead or estimated inclination of the road can be obtained using below Equations 3-5 and as shown in FIGS. 4A and 4B:

[00004]$\begin{array}{cc}{h}_{\mathrm{observed}\mathrm{grids}}=\frac{1}{2}\left\{\left({{\hat{Z}}_{t.Math.t}}^{\mathrm{right},\mathrm{last}}-{{\hat{Z}}_{t.Math.t}}^{\mathrm{right},\mathrm{last}-1}-.Math.{{\hat{Z}}_{t.Math.t}}^{\mathrm{right},\mathrm{first}+1}-{{\hat{Z}}_{t.Math.t}}^{\mathrm{right},\mathrm{first}}\right)+\left({{\hat{Z}}_{t.Math.t}}^{\mathrm{left},\mathrm{last}}-{{\hat{Z}}_{t.Math.t}}^{\mathrm{left},\mathrm{last}-1}-.Math.{{\hat{Z}}_{t.Math.t}}^{\mathrm{left},\mathrm{first}+1}-{{\hat{Z}}_{t.Math.t}}^{\mathrm{left},\mathrm{first}}\right)\right\}& \mathrm{Equation}3\end{array}$$\begin{array}{cc}{h}_{\mathrm{vehicle}}=\cos \left({}_{\mathrm{vehicle}}\right)d_{\mathrm{observed}\mathrm{grids}}& \mathrm{Equation}4\end{array}$$\begin{array}{cc}{h}_{\mathrm{road}\mathrm{ahead}}=h_{\mathrm{observed}\mathrm{grids}}+h_{\mathrm{vehicle}}& \mathrm{Equation}5\end{array}$ [0059] where, h.sub.road ahead is the estimated inclination of the road, h.sub.observed grids 310 is a height difference component as shown in FIGS. 3A-3B and is based only on lidar measurements, h.sub.vehicle is a compensation due to current pitch angle 330 (.sub.vehicle) of the vehicle as shown in FIGS. 3A-3B and is obtained by integration of IMU gyroscope pitch rate, and d.sub.observed grids 320 is the length of extension of the grids from last until first as indicated in FIG. 5.

[0060] The lateral inclination b.sub.road ahead, also called forward road bank measurement or height difference in lateral direction of the vehicle, is calculated in a similar way using Equations 6-8 wherein the different parameters of Equations 6-8 are shown in FIG. 4:

[00005]$\begin{array}{cc}{b}_{\mathrm{observed}\mathrm{grids}}=\frac{1}{\mathrm{last}-\mathrm{first}}\{\left({{\hat{Z}}_{t.Math.t}}^{\mathrm{left},\mathrm{last}}-{{\hat{Z}}_{t.Math.t}}^{\mathrm{right},\mathrm{last}}\right)+\left({{\hat{Z}}_{t.Math.t}}^{\mathrm{left},\mathrm{last}-1}-{{\hat{Z}}_{t.Math.t}}^{\mathrm{right},\mathrm{last}-1}\right)+.Math.+\left({{\hat{Z}}_{t.Math.t}}^{\mathrm{left},\mathrm{first}+1}-{{\hat{Z}}_{t.Math.t}}^{\mathrm{right},\mathrm{first}+1}\right)+\left({{\hat{Z}}_{t.Math.t}}^{\mathrm{left},\mathrm{first}}-{{\hat{Z}}_{t.Math.t}}^{\mathrm{right},\mathrm{first}}\right))& \mathrm{Equation}6\end{array}$$\begin{array}{cc}{b}_{\mathrm{vehicle}}=\sin \left({}_{\mathrm{vehicle}}\right)\mathrm{front}\mathrm{wheel}\mathrm{base}& \mathrm{Equation}7\end{array}$$\begin{array}{cc}{b}_{\mathrm{road}\mathrm{ahead}}=b_{\mathrm{observed}\mathrm{grids}}+b_{\mathrm{vehicle}}& \mathrm{Equation}8\end{array}$

where, b.sub.road ahead is the lateral inclination, b.sub.observed grids is a bank difference component based only on lidar measurements, b.sub.vehicle 410 is a compensation due to the current roll angle (.sub.vehicle) 430 of the vehicle obtained by integration of IMU gyroscope pitch rate, and front wheel base 420 is the distance between the centre of the front tires.

[0061] If a grid's mean and variance values are observed to be deviating beyond certain threshold to their adjacent grids, this indicates road anomalies, either a bump 502 or a pothole 500, along the driving direction, as in FIGS. 5A, 5B and 9.

[0062] Those skilled in the art will appreciate that the methods, systems and components described herein may comprise, in whole or in part, a combination of analogue and digital circuits and/or one or more appropriately programmed processors (e.g., one or more microprocessors including central processing units (CPU)) and associated memory, which may include stored operating system software, firmware and/or application software executable by the processor(s) for controlling operation thereof and/or for performing the particular algorithms represented by the various functions and/or operations described herein, including interaction between and/or cooperation with each other as well as transmitters and receivers. One or more of such processors, as well as other digital hardware, may be included in a single ASIC (Application-Specific Integrated Circuitry), or several processors and various digital hardware may be distributed among several separate components, whether individually packaged or assembled into a SoC (System-on-a-Chip).

[0063] Furthermore, the systems, methods and components described, and/or any other arrangement, unit, system, device or module described herein may for instance be implemented in one or several arbitrary nodes comprised in the host vehicle and/or one or more separate devices. In that regard, such a node may comprise an electronic control unit (ECU) or any suitable electronic device, which may be a main or central node. It should also be noted that the these may further comprise or be arranged or configured to cooperate with any type of storage device or storage arrangement known in the art, which may for example be used for storing input or output data associated with the functions and/or operations described herein. The systems, components and methods described herein may further comprise any computer hardware and software and/or electrical hardware known in the art configured to enable communication therebetween.

[0064] While the disclosure has been described with reference to exemplary embodiments, it will be understood by those skilled in the art that various changes may be made and equivalents may be substituted for elements thereof without departing from the scope of the disclosure. In addition, many modifications may be made to adapt a particular situation or material to the teachings of the disclosure without departing from the essential scope thereof. Therefore, it is intended that the disclosure not be limited to the particular embodiments disclosed, but that the disclosure will include all embodiments falling within the scope of the appended claims.