METHOD FOR ASSISTING A USER, INCLUDING AN INDICATOR

Abstract

A method for assisting a user in an interaction with an apparatus including at least one automated and/or partially automated and/or manual operating state by way of a control device. A possible change or a compulsory change of operating state of the apparatus is ascertained, an escalation level for the change of operating state of the apparatus is ascertained, the ascertained change of operating state and the escalation level are signaled to the user by a dynamic indicator. A control device, a computer program, a machine-readable storage medium, and an indicator are also described.

Claims

1. A method for assisting a user in an interaction with an apparatus including at least one automated and/or partially automated and/or manual operating state, by way of a control device, the method comprising the following steps: ascertaining a possible or compulsory change of operating state of the apparatus; ascertaining an escalation level for the change of operating state of the apparatus; and signaling the ascertained change of operating state and the escalation level, to the user by a dynamic indicator.

2. The method as recited in claim 1, wherein the ascertained change of operating state and the escalation level are signaled to the user haptically and/or visually and/or acoustically, by the dynamic indicator by way of at least one unit within and/or on the apparatus.

3. The method as recited in claim 1, wherein, to signal the ascertained change of operating state, the dynamic indicator is animated by way of a unit embodied as a lighting system and/or a screen and/or a hologram and/or a portable device and/or a head-up display.

4. The method as recited in claim 1, wherein the apparatus (4) is a vehicle that is operable in an automated and/or partially automated fashion, wherein the ascertained change of operating state and the escalation level are signaled by the dynamic indicator to a driver of the vehicle.

5. A control device configured to assist a user in an interaction with an apparatus including at least one automated and/or partially automated and/or manual operating state, by way of a control device, the control unit configured to: ascertain a possible or compulsory change of operating state of the apparatus; ascertain an escalation level for the change of operating state of the apparatus; and signal the ascertained change of operating state and the escalation level, to the user by a dynamic indicator.

6. A non-transitory machine-readable storage medium on which is stored a computer program for assisting a user in an interaction with an apparatus including at least one automated and/or partially automated and/or manual operating state, by way of a control device, the computer program, when executed by a computer or control device, causing the computer or control device perform the following steps: ascertaining a possible or compulsory change of operating state of the apparatus; ascertaining an escalation level for the change of operating state of the apparatus; and signaling the ascertained change of operating state and the escalation level, to the user by a dynamic indicator.

7. An indicator for signaling a required and/or an optional change of an operating state, the indicator including a head-related responsibility display, a hand-related responsibility display, and a foot-related responsibility display, wherein at least one of the head-related responsibility display, the hand-related responsibility display, and the foot-related responsibility display is configured to signal the required and/or optional change of operating state to a user in such a way that they are distinguishable from each other.

8. The indicator as recited in claim 7, wherein an escalation level for the change of operating state is signalable by the at least one of the head-related responsibility display, the hand-related responsibility display, and the foot-related responsibility display in the form of a color and/or animation.

9. The indicator as recited in claim 7, wherein the indicator is configured to indicate the change of at least one operating state of a vehicle that is operable in an automated and/or partially automated fashion.

10. The indicator as recited in claim 9, wherein: all of the head-related responsibility display, the hand-related responsibility display, and the foot-related responsibility display, are activated in the case of a manual operating state, the foot-related responsibility display is deactivated in the case of a partially automated longitudinal guidance of the vehicle, the hand-related responsibility display is deactivated in the case of a partially automated transverse guidance of the vehicle, and the head-related responsibility display is deactivated in the case of an automated transverse guidance and longitudinal guidance of the vehicle.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0031] FIG. 1 shows a schematic side view of a vehicle arrangement to illustrate a method according to a specific embodiment of the present invention.

[0032] FIG. 2 shows a schematic diagram of an indicator of modular construction according to a specific embodiment of the present invention.

[0033] FIG. 3 shows a schematic flow diagram to illustrate a method according to a first specific embodiment of the present invention.

[0034] FIG. 4 shows a schematic flow diagram to illustrate a method according to a second specific embodiment of the present invention.

[0035] FIG. 5 shows a schematic flow diagram to illustrate a method according to a third specific embodiment of the present invention.

DETAILED DESCRIPTION OF EXAMPLE EMBODIMENTS

[0036] FIG. 1 is a schematic side view of a vehicle arrangement 1 to illustrate a method 2 according to a specific embodiment. The method is described in detail in FIGS. 3, 4, and 5.

[0037] Vehicle arrangement 1 includes an apparatus 4, which is embodied as a vehicle 4 or an exemplary passenger car.

[0038] Vehicle 4 is operable in an assisted, partially automated, highly automated, and/or fully automated or driverless fashion in accordance with the BASt standard. A control device 6 is therefore able to take over the longitudinal guidance and/or the transverse guidance of vehicle 4 in an automated or partially automated fashion. A user 8 of apparatus 4, who is embodied as a driver of vehicle 4 here, is able to interact with control device 6 and vehicle 4 by way of an infotainment system 10. For the sake of simplicity, only a screen of infotainment system 10 is shown. As an alternative or in addition to infotainment system 10, visualization may take place by way of an instrument cluster display and/or a head-up display (not shown).

[0039] Infotainment system 10 here represents a possible embodiment of a unit that is able to signal an ascertained change of operating state of vehicle 4, and an ascertained escalation level, to user 8 haptically, visually, and/or acoustically using a dynamic indicator 12.

[0040] Vehicle 4 may additionally comprise further units, which are embodied as, e.g., lighting elements, speakers, actuators on the pedals or on the steering wheel, windshield wipers, and the like. These elements may be controlled by control device 6 to interact with user 8.

[0041] Furthermore, the at least one unit 10 may be a portable device of user 8, such as e.g. a tablet, smartphone, smartwatch, 3D glasses, or a VR device, and the like.

[0042] Control device 6 may, furthermore, interact with drive components 14 of vehicle 4, or have a data connection thereto, and may also access sensors of vehicle 4 (not shown). This allows control device 6 to ascertain the need for, or an option of, a change of operating state of vehicle 4, and to establish the corresponding urgency or escalation level. For example, on a freeway, the availability of automated steering, during which driver 8 only has to monitor the traffic situation, may be signaled to driver 8 by unit 10. From a manual control, driver 8 may, if desired, access partially automated control of vehicle 4.

[0043] The corresponding signaling takes place by way of an indicator 12, which transmits the optional changes as a suggestion and the required changes of operating state as an instruction to user 8 dynamically, and optionally in animated form.

[0044] FIG. 2 is a schematic diagram of an indicator 12 of modular construction according to one specific embodiment. Indicator 12 may be output graphically, acoustically, and/or haptically, e.g., by infotainment system 10. Infotainment system 10 is used merely by way of example. It is also possible for speakers, actuators in the vehicle interior, instrument cluster displays, head-up displays, vehicle interior lighting, illuminated interior accents, and the like to be used for the output of indicator 12.

[0045] In the exemplary embodiment shown, the output of indicator 12 takes place graphically on a screen of infotainment system 10, and indicates a current operating state. Furthermore, a change of operating state of vehicle 4 may be signaled to driver 8 by indicator 12 in order to illustrate to driver 8 his responsibility and tasks during the operating state.

[0046] Indicator 12 thus serves to signal a required and/or an optional change of an operating state of vehicle 4, and is illustrated on the screen of infotainment system 10 as from a point of view of driver 8. In the exemplary embodiment shown, indicator 12 includes a head-related responsibility display 14, a hand-related responsibility display 16, and a foot-related responsibility display 18. Responsibility displays 14, 16, 18 are designed to signal the required change or the optional change of operating state to driver 8 in such a way that they are distinguishable from each other.

[0047] In contrast to conventional status icons in vehicle 4, which merely inform driver 8 about the driving mode or current operating state, indicator 12, in the form of responsibility displays 14, 16, 18, informs driver 8 directly how to interact appropriately with the automated driving system of vehicle 4. Responsibility displays 14, 16, 18 communicate the role of driver 8 dynamically, and assist with the perception of the operating state of vehicle 4, thus guiding driver 8 toward optimal behavior, and increasing traffic safety.

[0048] Responsibility displays 14, 16, 18 substantially serve as an example of a possible embodiment of indicator 12. The activated responsibility displays are shown by shading.

[0049] If all responsibility displays 14, 16, 18 are activated, a manual operating state is active, or it is necessary to change to a manual operating state of vehicle 4 so that driver 8 takes over control and traffic monitoring completely and without assistance.

[0050] If head-related responsibility display 14 and hand-related responsibility display 16 are activated, and foot-related responsibility display 18 is correspondingly deactivated, acceleration and/or braking may be taken over by vehicle 4, e.g., in the context of an adaptive cruise control.

[0051] If only head-related responsibility display 14 is activated, besides automated braking and accelerating, automated steering also takes place by way of control device 6 or a corresponding control system of vehicle 4. In this case, driver 8 should merely observe the traffic in order to be able to intervene rapidly if necessary. This is signaled intuitively to driver 8 by way of activated head-related responsibility display 14.

[0052] If all responsibility displays 14, 16, 18 are deactivated, a change to a driverless or fully automated operating state may take place. To illustrate this operating state, a schematic driver figure may be shown facing away from the direction of travel in indicator 12 in a fully automated operating state 19 of this type.

[0053] The shadings that illustrate the respective responsibility displays 14, 16, 18 in activated form may be shown in different colors, and/or flashing with varying flash frequencies, to signal an urgency or an escalation level for the change of operating state, or a change of behavior of driver 8. For example, in an emergency situation, the shading may appear in red. In the case of an optional change of operating state, which may be understood merely as a suggestion, a green, white, gray, or blue coloring of the shading is possible.

[0054] The respective responsibility displays 14, 16, 18 may preferably switch on or activate, or switch off or deactivate, dynamically in order to illustrate corresponding changes of operating states of vehicle 4 to driver 8 with appropriate urgency.

[0055] Responsibility displays 14, 16, 18 may be controlled by control device 6 according to the degree of active driving automation or the operating state, and according to the role of driver 8 in the operating state, this being based on the configuration of the system.

[0056] An appropriate two-dimensional or three-dimensional animation of driver 8 may also be integrated into a head-up or holographic display.

[0057] In FIGS. 3 through 5, a method 2 for assisting a user 8 in an interaction with an apparatus 4 including at least one automated and/or partially automated and/or manual operating state is illustrated. Here, the different responsibility displays 14, 16, 18 of indicator 12 shown in FIG. 2 are used by way of example to illustrate method 2. User 8 here is likewise embodied as a driver 8 and apparatus 4 as a vehicle 4.

[0058] FIG. 3 is a schematic flow diagram to illustrate a method 2 according to a first specific embodiment. In this exemplary embodiment, a change between a manual operating state and a partially automated operating state, in which vehicle 4 takes over the steering, is illustrated.

[0059] During a manual operating state 20, driver 8 has sole control of vehicle 4. All responsibility displays 14, 16, 18 of indicator 12 are activated. This is signaled to driver 8 in infotainment system 10.

[0060] Control device 6 is able to communicate with vehicle-based subsystems and thus ascertain a need for a change of operating state of vehicle 4. This may be implemented, e.g., by successfully ascertaining operational boundaries 21. Operational boundaries 21 may be ascertained in the context of ascertaining a possible or compulsory change of operating state. For this purpose, measurement data from sensors of vehicle 4 are received and analyzed. If the requirements for a partially automated operating state are met, the change to this operating state may be signaled 22 to driver 8. By way of example, the availability of the partially automated operating state is signaled here by way of a visual pop-up display, which driver 8 selects 23 by touching a touch-sensitive screen.

[0061] Once the available operating state 23.sup.1 has been selected, the requirements for the operational boundaries of the partially automated operating state are checked 24 by consulting measurement data of vehicle-based sensors 5. .sup.1 [Translator's note: the positioning of the numeral 23 is inconsistent; see German PDF p. 12, lines 23 and 25.]

[0062] If the requirements 24.sup.2 are not met, e.g., because a vehicle 4 is not optimally centered within the driving lane, driver 8 is .sup.2 [Translator's note: the positioning of the numeral 24 is inconsistent; see German PDF p. 12, lines 28, 30, and 33.]requested to align 25 vehicle 4. If driver 8 corrects 26 the alignment of vehicle 4, or if the requirements 24 of the partially automated operating state are met, driver 8 is requested 27 to take his hands off the steering wheel of vehicle 4. This is done by way of indicator 12, which indicates the desired action to driver 8, e.g., by deactivating hand-related responsibility display 16 and foot-related responsibility display 18.

[0063] If driver 8 follows instruction 28 and takes his hands off the steering wheel, a confirmation 29 may then be transmitted to driver 8 by way of infotainment system 10.

[0064] FIG. 4 is a schematic flow diagram to illustrate a method 2 according to a second specific embodiment. Here, an exemplary change of the partially automated operating state from FIG. 3 to manual vehicle control is shown.

[0065] In the exemplary embodiment shown, it is assumed that vehicle 4 is in the partially automated operating state, and driver 8 is monitoring 30 the traffic situation. While the operating state is activated, measurement data from vehicle-based sensors 5, such as, e.g., camera sensors, radar sensors, LIDAR sensors, GNSS sensors, drive monitoring systems, and the like, are received and processed by control device 6.

[0066] In a further step, it is established by control device 6 that the operational boundaries on the stretch of road are exceeded 31, and the partially automated operating state will no longer be available. Driver 8 is informed 32 of this impending change of operating state by infotainment system 10. In a further step 33, it is checked whether driver 8 has positioned his hands on the steering wheel.

[0067] The exceeding of operational boundaries takes place in the context of ascertaining the change of operating state.

[0068] The ascertaining of the change of operating state preferably takes place continuously or at regular time intervals, the time intervals being adjustable according to the situation. For example, the time intervals may be shortened in the event of a high volume of traffic.

[0069] Furthermore, the impending change of operating state is signaled to driver 8 by indicator 12. In addition to head-related responsibility display 14, hand-related responsibility display 16 is activated here. Since the change is not critical at this point in time, control device 6 ascertains a low escalation level, which is illustrated, e.g., by a blue coloring of activated responsibility displays 14, 16.

[0070] If driver 8 has positioned his hands on the steering wheel, the change of operating state occurs, and indicator 12 may confirm 34 the change of operating state by way of an animation. If driver 8 fails to perform this action, a reminder or signaling to driver 8 occurs regarding the required change 35. Indicator 12 may likewise be animated for this purpose. Driver 8 may be allowed 36 a predefined period of, e.g., 10 seconds to perform the action.

[0071] If driver 8 continues to ignore indicator 12, a change in the escalation level takes place, responsibility display 14, 16 of indicator 12 is colored yellow.sup.3. In addition, brakes, horn, lighting elements, and the like may be triggered in order to alert 37 driver 8 to the change. .sup.3 [Translator's note: there may be something missing from the German sentence, probably a conjunction.]

[0072] In a further step 36, driver 8 is again given the opportunity to position his hands on the steering wheel. If driver 8 continues to ignore this opportunity, the classification moves to the highest escalation level, and responsibility displays 14, 16 are colored red in order to illustrate to driver 8 the urgency of the change of operating state. In addition, further units within vehicle 4 may be employed in order to illustrate to driver 8 the urgency of his action. For example, short braking operations may be performed, windshield wipers may be activated, light signals in the vehicle interior may be switched on, or animations may be played by infotainment system 10.

[0073] If driver 8 fails to place his hands on the steering wheel at a final request, control device 6 brings 38 vehicle 4 into the safe state.

[0074] FIG. 5 is a schematic flow diagram illustrating method 2 according to a third specific embodiment. In FIG. 5, an exemplary alternative to the procedure of FIG. 4 is shown, in which a change of operating state from the partially automated operating state of FIG. 3 back to a manual operating state of vehicle 4 is performed. Here, indicator 12 acts as a warning cascade, within which a rapid escalation up to emergency braking is able to take place.

[0075] It is assumed that vehicle 4 is in the partially automated operating state, and that driver 8 is monitoring 40 the traffic situation. While the operating state is activated, measurement data from vehicle-based sensors 5, such as e.g. camera sensors, radar sensors, LIDAR sensors, GNSS sensors, drive monitoring systems, and the like are received and processed by control device 6.

[0076] In a further step, control device 6 establishes that the operational boundaries on the stretch of road are exceeded 41, and that the partially automated operating state will shortly no longer be available. Driver 8 is informed 42 of this impending change of operating state by infotainment system 10. In a further step 43, driver 8 is given the opportunity of positioning his hands on the steering wheel within a period of, e.g., 3 seconds. It is also checked here, e.g., by sensors on the steering wheel, or by optical sensors in the vehicle interior, whether driver 8 has positioned his hands on the steering wheel.

[0077] All responsibility displays 14, 16, 18 of indicator 12 are activated to illustrate the change of operating state of vehicle 4. Because of the driving situation, the change of operating state is urgent or, in the exemplary embodiment shown, has to take place rapidly, and therefore at the end of the time period a high escalation level is ascertained 44 by control device 6.

[0078] Indicator 12 is employed as a wall display. Responsibility displays 14, 16, 18 of indicator 12 in this case are colored red and may be animated such that they flash.

[0079] In a subsequent request or check 45, control device 6 may ascertain that driver 8 has positioned his hands on the steering wheel of vehicle 4. As a result, the operating state 46 changes from the partially automated operating state to the manual operating state of vehicle 4. If driver 8 did not comply with request 45, vehicle 4 would be brought 47 into a safe state by control device 6, or emergency braking would be initiated. The transition to the safe state may comprise, e.g., an automated stop on the shoulder, or an automated approach to the nearest parking lot or rest area.