Object Reflectivity Estimation in a LIDAR System

Abstract

Methods, devices, systems, and computer program products for estimating object reflectivity in a light detection and ranging (LIDAR) system are disclosed. The method, for example, includes receiving LIDAR data for a plurality of LIDAR scan cycles. The method also includes generating a dataset from the LIDAR data by accumulating the recorded return signals over the plurality of scan cycles. A data feature associated with an object is identified in the dataset, and one or more parameters of the data feature are identified. An estimated reflectivity of the object may then be determined based on the one or more parameters.

Claims

1. A method for estimating object reflectivity in a light detection and ranging (LIDAR) system, the method comprising: receiving LIDAR data for a plurality of scan cycles, the LIDAR data including recorded return signals; generating a dataset from the LIDAR data by accumulating the recorded return signals over the plurality of scan cycles; identifying in the dataset a data feature associated with an object; identifying one or more parameters of the data feature; and determining an estimated reflectivity of the object based on the one or more parameters.

2. The method according to claim 1, wherein the identifying the one or more parameters of the data feature and the determining the estimated reflectivity of the object comprise applying a machine learning model.

3. The method according to claim 1, wherein the identifying the one or more parameters of the data feature comprises fitting a distribution function to the data feature.

4. The method according to claim 3, wherein: the data feature comprises a peak; and the fitting the distribution function to the data feature comprises: identifying a rising edge of the peak, and fitting a rising edge of the distribution function to the rising edge of the peak.

5. The method according to claim 3, wherein the fitting the distribution function to the data feature comprises: identifying a dip in accumulated signal counts of the dataset; and adjusting a width of the distribution function based on a position of the dip in the dataset.

6. The method according to claim 3, wherein the distribution function is a gaussian.

7. The method according to claim 3, wherein the generating the dataset comprises generating a histogram.

8. The method according to claim 7, wherein the fitting the distribution function to the data feature comprises fitting a shape of the distribution function to the histogram.

9. The method according to claim 3, wherein the determining the estimated reflectivity of the object comprises integrating counts of the fitted distribution function.

10. A processing device for estimating object reflectivity in a light detection and ranging (LIDAR) system, the processing device comprising: an input configured to receive LIDAR data for a plurality of scan cycles, the LIDAR data including recorded return signals; and a processor configured to: generate a dataset from the LIDAR data by accumulating the recorded return signals over the plurality of scan cycles, identify in the dataset a data feature associated with an object, identify one or more parameters of the data feature, and determine an estimated reflectivity of the object based on the one or more parameters.

11. The processing device according to claim 10, wherein: the processor comprises a machine learning module; and the machine learning module is configured to: identify the one or more parameters of the data feature, and determine the estimated reflectivity of the object using a machine learning model.

12. The processing device according to claim 10, wherein: the processor comprises a fitting module; and the fitting module is configured to identify the one or more parameters of the data feature by fitting a distribution function to the data feature.

13. The processing device according to claim 12, wherein: the data feature comprises a peak; and the processor is configured to: identify a rising edge of the peak, and fit a rising edge of the distribution function to the rising edge of the peak.

14. The processing device according to claim 12, wherein the processor is configured to: identify a dip in accumulated signal counts of the dataset; and adjust a width of the distribution function based on a position of the dip in the dataset.

15. The processing device according to claim 12, wherein the distribution function is a gaussian.

16. The processing device according to claim 12, wherein the processor is configured to generate the dataset by generating a histogram.

17. The processing device according to claim 12, wherein the processor is configured to determine the estimated reflectivity of the object by integrating counts of the fitted distribution function.

18. A computer program product for estimating object reflectivity in a light detection and ranging (LIDAR) system, the computer program product comprising instructions that, when executed by a computer, cause the computer to: receive LIDAR data for a plurality of scan cycles, the LIDAR data including recorded return signals; generate a dataset from the LIDAR data by accumulating the recorded return signals over the plurality of scan cycles; identify in the dataset a data feature associated with an object; identify one or more parameters of the data feature; and determine an estimated reflectivity of the object based on the one or more parameters.

19. The computer program product according to claim 18, wherein the instructions, when executed by a computer, cause the computer to: identify the one or more parameters of the data feature and the determine the estimated reflectivity of the object by applying a machine learning model.

20. The computer program product according to claim 18, wherein the instructions, when executed by a computer, cause the computer to: identify the one or more parameters of the data feature by fitting a distribution function to the data feature.

Description

BRIEF DESCRIPTION OF DRAWINGS

[0030] Illustrative embodiments will now be described with reference to the accompanying drawings in which:

[0031] FIG. 1 shows accumulated histogram counts of a single SPAD pixel over time;

[0032] FIG. 2 shows an aerial view of an automotive LIDAR system according to an illustrative embodiment;

[0033] FIG. 3 shows a schematic view of a LIDAR device according to the illustrative embodiment;

[0034] FIG. 4A shows a graph illustrating an example number of integrated peak counts of a SPAD array at different reflectivities at long range distances;

[0035] FIGS. 4B and 4C show example fitted histogram counts for the low and high reflectivities in FIG. 4A;

[0036] FIG. 5A shows a graph illustrating an example number of integrated peak counts of a SPAD array at different reflectivities at short range distances; and

[0037] FIGS. 5B and 5C show example fitted histogram counts for the low and high reflectivities in FIG. 5A.

DETAILED DESCRIPTION

[0038] FIG. 2 shows an aerial view of an automotive LIDAR system 20 according to an illustrative embodiment. The LIDAR system 20 includes a plurality of LIDAR devices 21-24 mounted around the vehicle. Each LIDAR device includes a controller, an emitter and a sensor, described in detail below with reference to FIG. 3. The LIDAR devices 21-24 together provide a combined field of view surrounding the vehicle. Objects 26, such as pedestrians or structures, may enter the field of view of one or more of the LIDAR devices 21-24 to be detected by the LIDAR system 20.

[0039] FIG. 3 shows a schematic illustration of one of the LIDAR devices 21-24 shown in FIG. 2. Each LIDAR device 21-24 includes a controller 31, an emitter 33 and a sensor 32. In use, during a plurality of scan cycles, the controller 31 controls its emitter 33 to emit a series of output signals or light pulses 34 for illuminating its field of view. These signals or pulses 34 reflect off objects 26 located in the device's field of view and return reflected light 36, which is detected by the sensor 32. The photons captured by the sensor 32 during this are recorded in a dataset as an accumulated count over the series of scan cycles. As is discussed in further detail below, the higher the reflectivity of an object 26, the greater the number of photons reflected back onto the sensor 32.

[0040] In this embodiment, the controller 31 accumulates the sensor counts and performs digital signal processing to generate output data. However, in other embodiments, the controller 31 may output the recorded data to a central processor for digital signal processing. For instance, processing may be performed in an electronics control unit located elsewhere in the vehicle.

[0041] The digital signal processing method will now be described. As discussed above, the sensor 32 receives LIDAR data over the plurality of scan cycles and the dataset is generated by the controller 31 by binning the accumulated counts over the plurality of scan cycles. This thereby forms a dataset having the accumulated/integrated signal counts binned into a number of time intervals. A data feature associated with an object is then identified in the dataset, and a distribution function is fitted to the data feature to re-simulate the data lost through saturation of the LIDAR sensor 32. By recovering the undetected lost data in this way, a more accurate estimated reflectivity of the object can be calculated. This process is described in further detail below with reference to FIGS. 4A-4C and 5A-5C.

[0042] In this connection, FIG. 4A shows a graph illustrating an example relationship between object reflectivity and integrated peak counts for measured 41 and theoretical 42 distribution peaks, in this case for longer range objects. As mentioned above, as the reflectivity of an object 26 increases, the number of photons reflected back onto the sensor 32 also increases. This is illustrated by the theoretical plot 42, which shows a linear relationship between reflectivity and the integrated count value. However, in practice, because of the saturation effect of deadtime on SPAD elements within the sensor, the measured integrated count value 41 does not exhibit this linear relationship. Instead, the measured plot 41 is substantially linear at lower reflectivities, but the response is reduced as the reflectivity increases, resulting in a levelling off due to saturation.

[0043] FIG. 4B shows an example accumulated histogram 43 count for a low reflectivity object at long range. As shown, there is a pronounced data feature in the form of a peak 46 associated with the object 26. Accordingly, if a theoretical distribution peak is fitted to this, as shown by the gaussian distribution function 44, it substantially follows the shape of the peak 46 in the measured histogram 43. As such, it can be seen in this scenario that the response of the sensor 32 is therefore linear, and so totaling the accumulated counts for the peak 46 or integrating the estimated count for the gaussian distribution function 44 can both be used to indicate the reflectivity of the object.

[0044] In contrast to the above, FIG. 4C shows an example accumulated histogram 43 for a high reflectivity object 26 at a similarly long range. In this case, the peak 46 associated with the object 26 is much less pronounced because the increased reflectivity has caused saturation of SPAD elements. Consequently, totaling the accumulated counts for this peak 46 would provide a less accurate indication of the object's reflectivity. However, if the gaussian distribution function 44 described above is again fitted to the peak 46 in the measured histogram count 43, the controller 31 is able to simulate linearity. In particular, the controller 31 fits the distribution function 44 by matching the rising edge of the gaussian with the rising edge of the data feature, designated by the initial slope of the measured peak 46.

[0045] The position of this rising edge in the dataset provides a parameter indicting the accumulated counts for the first reflective return signals. As such, these return signals are the least affected by saturation, and thereby provide for accurate fitting of the distribution function 44. Once fitted, the distribution function 44 simulates a linear reflectivity response and hence effectively allows the unsaturated return signals to be recovered by integrating the estimated accumulated count for the distribution function 44. That is, integration can be used to calculate the area beneath the gaussian 44 and thereby indicate the true total accumulated count for photons reflected from the object 26. This thereby indicates the reflectivity of the object in question.

[0046] The above saturation effect is even more pronounced when objects are detected at short range. In this connection, FIG. 5A shows a graph illustrating an example relationship between object reflectivity and integrated peak counts for measured 51 and theoretical 42 distribution peaks for short range objects. As shown, the theoretical plot 42 again exhibits a linear relationship between reflectivity and the integrated count value. In contrast, the measured plot 51 falls off a lot quicker from the linear correlation, indicating that the sensor becomes saturated very quickly as reflectivity increases.

[0047] The above effect can also be seen in the histogram counts when viewing low and high reflectivity objects at short range. In this respect, FIG. 5B shows an example incident photon rate count histogram for a low reflectivity object viewed at short range. As shown, there is again a very pronounced peak 46 in the measured count 43 associated with the object 26, which matches the idealised gaussian distribution function 44. This contrasts with FIG. 5C, which shows an example incident photon rate count histogram for a high reflectivity object 26 viewed at short range.

[0048] In the case of FIG. 5C, the saturation effect caused by deadtime results in a very muted peak 46 in the measured count 43. This is due to the high photon rates reflected back from the object 26. As such, as you move from left to right across the histogram, the measured count 43 quickly drops off as SPAD elements are more saturated during later time bins. Furthermore, once past the true peak photon intensity, the measured photon rate actually begins to drop, resulting in a dip 59 in the histogram. This dip 59 is caused by paralysation of the sensor elements that occurs when the high incident photon rate results in the deadtime blocking the subsequent detection of noise. As such, the position of the dip 59 in the dataset can be used as a parameter to indicate the point at which the last signal photons associated with the object 26 arrive.

[0049] In this scenario, as with the example shown in FIG. 4C, the gaussian distribution function 44 may again be fitted to the peak 46 of the measured histogram count 43 using processing within the controller 31. This matches the initial rising edge incline on the left of the gaussian peak 44 with the rising edge of the peak 46. At the same time, the dip 59 is identified by the controller 31 and used to set the width of the distribution function 44. The combination of these parameters may thereby be used to fit and size the distribution function 44 to the peak 46 in the actual measured count 43 for simulating a linear reflectivity response. As such, the unsaturated accumulated return signal count can again be recovered by integrating counts for the distribution function 44. As in the above example, this allows the reflectivity of the object 26 in question to be determined, despite much of the incident photon rate data being unrecorded.

[0050] With the above, an improved LIDAR method, processing device, and computer program for estimating reflectivity over a range of photo incident rates can be provided. Advantageously this bypasses the inherent limitations of the saturation effect of the sensor, and it may provide improved reflectivity determination and image output resolution, without increasing the cost or complexity of the LIDAR system.

[0051] It will be understood that the embodiments illustrated above show applications only for the purposes of illustration. In practice, embodiments may be applied to many different configurations, the detailed embodiments being straightforward for those skilled in the art to implement.

[0052] In this connection, for example, although the examples above describe the technique in the context of fitting the distribution function to a signal peak in a histogram, it will be understood that the data processing may not require a histogram to be generated. For example, in some embodiments, a controller implementing a machine learning model may be used to identify a data feature associated with an object from the dataset and estimate the reflectivity based on one or more parameters of this data feature. The machine learning model may be trained, for example, using an algorithm and training data including measured incident photon rates for a plurality of objects with known reflectivities.

Example Implementations

[0053] Example 1: A method for estimating object reflectivity in a LIDAR system, the method comprising the steps of: receiving LIDAR data for a plurality of LIDAR scan cycles; generating a dataset from the LIDAR data by accumulating the recorded return signals over the plurality of scan cycles; identifying a data feature associated with the object in the dataset; identifying one or more parameters of the data feature; and determining an estimated reflectivity of the object based on the one or more parameters.

[0054] Example 2: A method according to claim 1, wherein the steps of identifying the one or more parameters of the data feature and determining an estimated reflectivity of the object comprises applying a machine learning model.

[0055] Example 3: A method according to claim 1, wherein the step of identifying the one or more parameters of the data feature comprises fitting a distribution function to the data feature.

[0056] Example 4: A method according to claim 3, wherein the data feature is a peak, and the step of fitting the distribution function to the data feature comprises identifying the rising edge of the peak and fitting the rising edge of the distribution function to the rising edge of the peak.

[0057] Example 5: A method according to claim 1 or 2, wherein the step of fitting the distribution function to the signal comprises identifying a dip in the accumulated signal counts in the dataset and adjusting the width of the distribution function based on the position of the dip in the dataset.

[0058] Example 6: A method according to any of claims 3 to 5, wherein the distribution function is a gaussian.

[0059] Example 7: A method according to any of claims 3 to 6, wherein the step of generating the dataset comprises generating a histogram.

[0060] Example 8: A method according to claim 7, wherein the step of fitting the distribution function to the signal comprises fitting a shape of the distribution function to the histogram.

[0061] Example 9: A method according to any of claims 3 to 8, wherein the step of determining an estimated reflectivity comprises integrating counts of the fitted distribution function.

[0062] Example 10: A processing device for estimating object reflectivity in a LIDAR system, the device comprising: an input for receiving LIDAR data for a plurality of LIDAR scan cycles; and a processor for generating a dataset from the LIDAR data by accumulating the recorded return signals over the plurality of scan cycles, for identifying a data feature associated with the object in the dataset, for identifying one or more parameters of the data feature, and for determining an estimated reflectivity of the object based on the one or more parameters.

[0063] Example 11: A processing device according to claim 10, wherein the processor comprises a machine learning module, and the machine learning module identifies the one or more parameters of the data feature and determines an estimated reflectivity of the object using a machine learning model.

[0064] Example 12: A processing device according to claim 10, wherein the processor comprises a fitting module, and the fitting module identifies the one or more parameters of the data feature by fitting a distribution function to the data feature.

[0065] Example 13: A processing device according to claim 12, wherein the data feature is a peak, and the processor is configured to identify the rising edge of the peak and fit the rising edge of the distribution function to the rising edge of the peak.

[0066] Example 14: A processing device according to claim 12 or 13, wherein the processor is configured to identify a dip in the accumulated signal counts in the dataset and adjust the width of the distribution function based on the position of the dip in the dataset.

[0067] Example 15: A computer program product for estimating object reflectivity in a LIDAR system, the program comprising instructions which, when executed by a computer, cause the computer to carry out the steps of: receiving LIDAR data for a plurality of LIDAR scan cycles; generating a dataset from the LIDAR data by accumulating the recorded return signals over the plurality of scan cycles; identifying a data feature associated with the object in the dataset; identifying one or more parameters of the data feature; and determining an estimated reflectivity of the object based on the one or more parameters.

[0068] The use of “example,” “advantageous,” and grammatically related terms means “serving as an example, instance, or illustration,” and not “preferred” or “advantageous over other examples.” Items represented in the accompanying figures and terms discussed herein may be indicative of one or more items or terms, and thus reference may be made interchangeably to single or plural forms of the items and terms in this written description. The use herein of the word “or” may be considered use of an “inclusive or,” or a term that permits inclusion or application of one or more items that are linked by the word “or” (e.g., a phrase “A or B” may be interpreted as permitting just “A,” as permitting just “B,” or as permitting both “A” and “B”), unless the context clearly dictates otherwise. Also, as used herein, a phrase referring to “at least one of” a list of items refers to any combination of those items, including single members. For instance, “at least one of a, b, or c” can cover a, b, c, a-b, a-c, b-c, and a-b-c, as well as any combination with multiples of the same element (e.g., a-a, a-a-a, a-a-b, a-a-c, a-b-b, a-c-c, b-b, b-b-b, b-b-c, c-c, c-c-c, or any other ordering of a, b, and c).