CONTROL OF A DISPLAY ON AN AUGMENTED REALITY HEAD-UP DISPLAY DEVICE FOR A VEHICLE
20210390926 · 2021-12-16
Inventors
Cpc classification
G08G1/165
PHYSICS
G08G1/166
PHYSICS
G09G2320/0613
PHYSICS
G02B2027/0118
PHYSICS
B60K35/00
PERFORMING OPERATIONS; TRANSPORTING
International classification
B60K35/00
PERFORMING OPERATIONS; TRANSPORTING
Abstract
Controlling a display in an augmented reality head-up display for a vehicle. At least one safety-relevant parameter is determined in a first step. If two or more safety-relevant parameters are determined, one of the determined parameters can optionally be selected as a critical parameter. The intensity of the display element displayed on the augmented reality head-up display is subsequently adjusted on the basis of the at least one safety-relevant parameter. The display element is then output for display on the augmented reality head-up display.
Claims
1-13. (canceled)
14. A method for controlling visual indicia in a vehicle, comprising: operating a display in an augmented reality heads-up display for the vehicle, wherein the display comprises a display element; determining at least one safety-relevant parameter from a sensor system relating to a vehicle driving environment; and adjusting an intensity of the display element within the display, based on the determined at least one safety-relevant parameter
15. The method of claim 14, wherein the at least one safety-relevant parameter comprises at least one of: a distance value representing a distance of the vehicle to a target element, and/or a time-to-collision value of the target element.
16. The method of claim 15, wherein the at least one safety-relevant parameter comprises a plurality of parameters, and further comprising selecting one of the plurality of parameters for adjusting the intensity of the display element within the display.
17 . The method of claim 15, further comprising: determining a size of the display element; and modifying the adjusting of the intensity of the display element within the display, based on the determined size.
18. The method of claim 15, wherein the target element comprises an obstacle and/or an intersection.
19. The method of claim 14, wherein adjusting the intensity of the display element comprises one of: adjusting the intensity between 100% and 0% of a nominal intensity on the basis of the determined at least one safety-relevant parameter, and maintaining a reduced intensity of the display element.
20. The method of claim 14, wherein the display element comprises contour lines, and wherein adjusting the intensity of the display element comprises maintaining the visibility of the contour lines when the intensity of the display element is adjusted.
21. The method of claim 14, wherein the display element comprises a navigation instruction or a warning sign.
22. A device for controlling visual indicia in a vehicle, comprising: a sensor system for generating sensor data; an analysis module, operatively coupled to the sensor system, for determining at least one safety-relevant parameter from the sensor data; an augmented reality heads-up display for the vehicle, wherein the display comprises a display element; and a control module for determining at least one safety-relevant parameter from a sensor system relating to a vehicle driving environment, and adjusting an intensity of the display element within the display, based on the determined at least one safety-relevant parameter
23. The device of claim 22, wherein the at least one safety-relevant parameter comprises at least one of: a distance value representing a distance of the vehicle to a target element, and/or a time-to-collision value of the target element.
24. The device of claim 23, wherein the at least one safety-relevant parameter comprises a plurality of parameters, and further comprising selecting one of the plurality of parameters for adjusting the intensity of the display element within the display.
25. The device of claim 23, further comprising: determining a size of the display element; and modifying the adjusting of the intensity of the display element within the display, based on the determined size.
26. The device of claim 23, wherein the target element comprises an obstacle and/or an intersection.
27. The device of claim 22, wherein adjusting the intensity of the display element comprises one of: adjusting the intensity between 100% and 0% of a nominal intensity on the basis of the determined at least one safety-relevant parameter, and maintaining a reduced intensity of the display element.
28. The device of claim 22, wherein the display element comprises contour lines, and wherein adjusting the intensity of the display element comprises maintaining the visibility of the contour lines when the intensity of the display element is adjusted.
29. The device of claim 22, wherein the display element comprises a navigation instruction or a warning sign.
30. A non-transitory computer-readable medium having stored therein instructions executable by one or more processors of a vehicle to: operate a display in an augmented reality heads-up display for the vehicle, wherein the display comprises a display element; determine at least one safety-relevant parameter from a sensor system relating to a vehicle driving environment, wherein the at least one safety-relevant parameter comprises at least one of: a distance value representing a distance of the vehicle to a target element comprising an obstacle and/or an intersection, and/or a time-to-collision value of the target element; and adjust an intensity of the display element within the display, based on the determined at least one safety-relevant parameter
31. The non-transitory computer-readable medium of claim 30, wherein the at least one safety-relevant parameter comprises a plurality of parameters, and wherein the non-transitory computer-readable medium is further executable to select one of the plurality of parameters for adjusting the intensity of the display element within the display.
32. The non-transitory computer-readable medium of claim 30, wherein the non-transitory computer-readable medium is further executable to: determine a size of the display element; and modify the adjusting of the intensity of the display element within the display, based on the determined size.
33. The non-transitory computer-readable medium of claim 30, wherein the display element comprises contour lines, and wherein adjusting the intensity of the display element comprises maintaining the visibility of the contour lines when the intensity of the display element is adjusted.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
[0015] Further features of the present disclosure shall become clear from the following description and the attached claims, in conjunction with the drawings. Therein:
[0016]
[0017]
[0018]
[0019]
[0020]
[0021]
[0022]
[0023]
[0024]
[0025]
[0026]
DETAILED DESCRIPTION
[0027] For a better understanding of the principles of the present disclosure, embodiments of the present disclosure shall be explained below in reference to the figures. It should be noted that the present disclosure is not limited to the specifically disclosed embodiments, and that the features described in reference thereto can also be combined or modified without abandoning the scope of protection for the present disclosure, as defined in the attached claims.
[0028] In some examples, an occlusion or concealment resulting from display elements may be dynamically reduced according to aspects of the present disclosure. Intensities of the display elements may be reduced on the basis of safety-relevant parameters. The opacity of the display element can be adjusted for this, or a portion of the surface of the display element occupied by image elements can be adjusted. Without this reduction in occlusion, a distant turning arrow with a constant high intensity would become increasingly larger while approaching it, due to the augmented reality presentation, thus leading to a critical concealment. To avoid concealing driver-relevant information, the display elements can be reduced to contour lines. It has been shown in tests, however, that the visibility of these display elements, reduced to contour lines, cannot be ensured in bright ambient light. Furthermore, the size of the display elements can be limited, such that they can only grow to a certain size. This undermines the augmented reality effect, however. Using the solution described herein, the occlusion caused by the enlargement may be compensated for, by reducing the intensity. The salience of display elements is then reduced if the occlusion is critical or problematic. A sufficient salience in relation to the environment is necessary for the display to be discernable. This results in a compromise between discernability and occlusion reduction.
[0029] In some examples, the at least one safety-relevant parameter may be configured using a distance from the vehicle to a target element or a time to a collision with the target element. A small object concealed by display elements, e.g., a child, can be considered as a particularly critical situation. Such a situation occurs most frequently in urban environments. The occlusion by display elements is most critical when they are at their largest, e.g., when the distance between the virtual display element and the ego-vehicle is minimal. To give the driver an opportunity to react to potentially concealed obstacles, the concealment should be reduced within a critical distance. The critical distance may depend on the speed of the ego-vehicle, where the time to a collision is often used as a safety indicator. In urban environments, values of 1.5 seconds for the time to a collision can be regarded as acceptable. In some situations, the time to a collision is not sufficient for this. In slow traffic situations, e.g., during rush hour, this could result in a very late reduction of the occlusion. It therefore makes sense to adjust the intensity on the basis of the distance in these cases.
[0030] In some examples, the size of the display element may be taken into account when adjusting the intensity. By way of example, the intensity adjustment can be limited to display elements of a minimum size, e.g., with respect to height, width or area. Display elements that are not occlusive may be disregarded in this manner. Furthermore, the size of the display element can be taken into account when determining the distance within which the intensity is to be adjusted.
[0031] In some examples, both the distance from the vehicle to a target element and the time to a collision therewith are determined, and a critical parameter may be selected for adjusting the intensity of the display element. The inclusion of the time parameter reliably reduces speed-dependent occlusion. This ensures that there is enough time to react and brake. By way of example, a relatively small display element at an ego-speed of 150 km/h can conceal a distant obstacle due to the contact-analog presentation, such that there is not enough time to react and brake if the solution is based only on the distance, or the size of the display element. The time-dependent solution, on the other hand, ensures that these obstacles are not concealed.
[0032] In some examples, the intensity can be adjusted between 100% and 0% of a nominal intensity, on the basis of the at least one safety-relevant parameter. Depending on the design of the display element, a display element can be composed of numerous parts that have different nominal intensities. By way of example, a virtual sign can be composed of three adjacent direction arrows with intensities of 100%, 70% and 50%. To simplify the adjustment of the intensities, it therefore makes sense to determine respective percentual adjustments of the nominal intensities.
[0033] In some examples, the intensity of the display element is only reduced on the basis of the at least one safety-relevant parameter, but not subsequently increased. In this manner, a fluctuation in intensity is prevented, e.g., when braking abruptly. If the intensity of the display element in a given situation is only affected by the time to a collision, this would result in a fluctuation in cases in which a driver is quickly approaching an intersection, and then decelerates, because the time to the collision then increases again. As a result, the intensity is only affected unidirectionally, e.g., decreased, at least by the time to the collision. If the time to the collision increases after the intensity has been decreased, the intensity remains at the previous level and first decreases again when the time to the collision falls below the previous lower value.
[0034] In some examples, the contour lines of the display element may remain visible. Because the contour lines of the display element are too thin to conceal objects in the environment, they can remain visible without causing problems. This also ensures that the information output by the display elements remains visible and comprehensible.
[0035] In some examples, functions for adjusting the intensity of the display element on the basis of the at least one safety-relevant parameter can be adjusted. The functions can be adjusted on the basis of the size of the display element, e.g., to determine the distance, below which an adjustment of the intensity should take place. These functions can also be adjusted by a vehicle user, who can adjust the presentation of the display elements to his personal preferences. The manufacturer of the vehicle or the augmented reality head-up display can also adjust the functions. They can react in this manner to amended guidelines or results of traffic research.
[0036] In some examples, the display element includes navigation instructions or a warning sign. Because these types of display element are used particularly frequently in contact-analog presentations, use of the solution according to the present disclosure is of particular relevance for these display elements.
[0037] In some examples, the target element may be an obstacle or an intersection. In the case of navigation instructions, the target element is normally an intersection, at which the navigation instruction is placed. Warning signs are usually used for obstacles, e.g., objects, other road users, or road damage.
[0038] display for a vehicle. At least one safety-relevant parameter is determined in a first step 10. The safety-relevant parameter can be, e.g., a distance from the vehicle to a target element, or a time to a collision with the target element. The target element can be an obstacle or an intersection, for example. If two or more safety-relevant parameters are determined 10, one of these parameters can optionally be selected 11 as a critical parameter. An intensity of a display element shown on the augmented reality head-up display, e.g., navigation directions or a warning signal, is subsequently adjusted 12 on the basis of the at least one safety-relevant parameter, in which the size of the display element can also be taken into account. By way of example, the opacity of the display element can be adjusted, or a portion of the display element populated with image elements can be adjusted. The functions for this can be adjusted, e.g., by a user or by a vehicle manufacturer. The intensity can advantageously be adjusted between 100% and 0% of a nominal intensity. Contour lines of the display element preferably remain visible. Advantageously, the intensity may only be reduced, and not subsequently increased in some examples. The display element is then output 13 for display on the augmented reality head-up display.
[0039]
[0040] If the analysis module 22 determines two or more safety-relevant parameters, one of these parameters can optionally be selected as the critical parameter by the analysis module 22. A control module 23 may be configured to adjust the intensity of a display element shown on the augmented reality head-up display, e.g., a navigation instruction or a warning sign, on the basis of the at least one safety-relevant parameter P, in which the size of the display element can also be taken into account. By way of example, the opacity of the display element can be adjusted, or a portion of the display element populated by image elements can be adjusted. The functions for this can be adjusted, e.g., by a user or the vehicle manufacturer. The intensity can advantageously be adjusted between 100% and 0% of a nominal intensity. Contour lines of the display element preferably remain visible. Advantageously, the intensity is only reduced, and not subsequently increased. The display element can then be output for display on the augmented reality head-up display, e.g., in the form of image data or control signals for a control unit 42 in the augmented reality head-up display, via an output 26 in the device 20.
[0041] The analysis module 22 and the control module 22 can be controlled by a control module 24. Settings of the analysis module 22, the control module 23, or the control module 24 can also be modified via a user interface 27. The data received by the device 20 can be stored, as needed, in a memory 25 in the device 20, e.g., for later analysis or use by the components in the device 20. The analysis module 22, control module 23, and the control module 24 can be dedicated hardware, e.g., in the form of integrated circuits. In this example, the device 20 may be configured as an independent component. It could also be integrated in the control unit 42 for the augmented reality head-up display.
[0042]
[0043] The processor 32 can include one or more processor units, such as microprocessors, digital signal processors, or combinations thereof.
[0044] The memories 25, 31 in these embodiments can contain both volatile and non-volatile memory areas, and different memory devices and mediums, such as hard disks, optical memories, or solid state memories.
[0045] Some embodiments of the present disclosure shall be described below in reference to
[0046]
[0047]
[0048] The imaging unit 50 can be an LCD-TFT, for example. The augmented reality head-up display 41 may be configured to be installed in a dashboard in the motor vehicle.
[0049]
[0050]
[0051]
[0052]
[0053] The intensities of the individual triangles may be reduced as a function of both the distance from the ego-vehicle to the target element, such as the position of the virtual sign, as well as the time to a collision with the virtual sign. The adjustment of the intensities can take place, by way of example, according to the following logic:
If (f.sub.intensity(t)<f.sub.intensity(d)) {//the lower value is output target intensity=f.sub.intensity(t); }
otherwise
{target intensity=f.sub.intensity(d):}
if (target intensity<current intensity) {// it is ensured that the intensity decreases current intensity=target intensity:}
In this case, f.sub.intensity(t) describes the intensity as a function of the time to a collision with the target element, and f.sub.intensity(d) describes the intensity as a function of the distance from the ego-vehicle to the target element. The current intensity is the intensity currently used for presenting the individual triangles.
[0054]
[0055]
[0056] With slow speeds, it may be the case that the time to the collision is greater than t.sub.1, resulting in an intensity of f.sub.max, while the distance is already shorter than d.sub.2, resulting in an intensity of 0%. In these cases, the critical parameter must be selected, e.g., the distance in this case. The intensity may then be set to 0% accordingly.
[0057] The outlines or contour lines of the display elements or image elements from which the display element is composed, preferably remain at an opacity of 100% until the entire display element is faded out. By way of example, the entire display element can be faded out over the course of 500 ms, as soon as the display element has reached its maximum size. The height of the display element should not exceed the height of the field of vision. The borderlines surrounding the individual elements in the display elements, e.g., the three arrows in the sign described above, should not exceed the width of the field of vision. Both the distance, at which the maximum size is reached, and the length of the fade-out, can preferably be adjusted.
LIST OF REFERENCE SYMBOLS
[0058] 10 determining safety-relevant parameters [0059] 11 selecting a parameter [0060] 12 adjusting an intensity of a display element [0061] 13 outputting the display element for display [0062] 20 device [0063] 21 input [0064] 22 analysis module [0065] 23 control module [0066] 24 control module [0067] 25 memory [0068] 26 output [0069] 27 user interface [0070] 30 device [0071] 31 memory [0072] 32 processor [0073] 33 input [0074] 34 output [0075] 40 vehicle [0076] 41 augmented reality head-up display [0077] 42 control unit for the augmented reality head-up display [0078] 43 camera [0079] 44 sensor system [0080] 45 navigation system [0081] 46 data transfer unit [0082] 47 assistance system [0083] 48 memory [0084] 49 network [0085] 50 imaging unit [0086] 51 optical module [0087] 52 optical components [0088] 53 projection surface [0089] 60 display element [0090] 62 contour lines [0091] 63 image element [0092] 64 field of vision [0093] d distance [0094] P parameter [0095] t time
REFERENCES
[0096] [1] J. L. Gabbard et al.: “Behind the Glass: Driver Challenges and Opportunities for AR Automotive Applications,” Proceedings of the IEEE, Vol. 102 (2014), pp. 124-136.