METHOD AND CONTROL DEVICE FOR OPERATING A HEAD-MOUNTED DISPLAY DEVICE IN A MOTOR VEHICLE

Abstract

A display device that can be worn on the head of a vehicle occupant of a motor vehicle is controlled to display a virtual environment within which the vehicle occupant wearing the display device virtually moves according to a detected movement of the motor vehicle. A potential stopping point of the motor vehicle is determined. A deceleration of the motor vehicle is detected within a given environment of the potential stopping point and the display device is controlled to output a virtual scene staging within the virtual environment which causes a deceleration of the virtual movement of the vehicle occupant within the virtual environment corresponding to the deceleration of the motor vehicle.

Claims

1.-12. (canceled)

13. A method for operating a display device wearable by a vehicle occupant of a motor vehicle, the method comprising: controlling the display device to display a virtual environment within which the vehicle occupant wearing the display device virtually moves according to a detected movement of the motor vehicle; determining a potential stopping point of the motor vehicle; and in response to a deceleration of the motor vehicle being detected within a predetermined environment of the potential stopping point, controlling the display device to output a virtual scene staging within the virtual environment so as to cause a deceleration of the virtual movement of the vehicle occupant within the virtual environment to correspond to the deceleration of the motor vehicle.

14. The method according to claim 13, wherein the potential stopping point of the motor vehicle includes at least one of a traffic light, an intersection, a read junction, a traffic jam, or a location based on a braking of another motor vehicle.

15. The method according to claim 13, determining the potential stopping point of the motor vehicle is based on evaluating at least one of sensor data characterizing an environment of the motor vehicle, digital map data, traffic flow data, car-to-car communication data, or data characterizing a driving behavior of a driver of the motor vehicle.

16. The method according to claim 13, wherein controlling the display device to output the virtual scene staging within the virtual environment is performed in response to the deceleration of the motor vehicle exceeding a predetermined deceleration level.

17. The method according to claim 13, wherein in response to an acceleration of the motor vehicle exceeding a predetermined acceleration after the deceleration of the motor vehicle, controlling the display device to end the output of the virtual scene staging, and controlling the display device to display the virtual environment within which the vehicle occupant virtually moves according to the detected movement of the motor vehicle.

18. The method according to claim 13, wherein in response to no deceleration of the motor vehicle being detected when the motor vehicle passes the potential stopping point, the display device is not controlled to output the virtual scene staging.

19. The method according to claim 13, wherein the virtual environment includes a space scene through which the vehicle occupant wearing the display device flies virtually with a spaceship, and the virtual scene staging includes the spaceship having a defect, and repairing the defect while the motor vehicle is stationary at the potential stopping point.

20. The method according to claim 13, further comprising: from a predetermined approach of the motor vehicle to the predetermined environment, displaying a virtual element within the virtual environment, wherein controlling the display device to output the virtual scene staging within the virtual environment includes using the virtual element to cause the deceleration of the virtual movement of the vehicle occupant within the virtual environment to correspond the deceleration of the motor vehicle.

21. The method according to claim 20, wherein in response to no deceleration of the motor vehicle being detected when the motor vehicle passes the potential stopping point, ending the display of the virtual element within the virtual environment after the motor vehicle passes the stopping point.

22. The method according to claim 20, wherein the virtual environment includes a space scene through which the vehicle occupant wearing the display device flies virtually with a first spaceship, the virtual element includes a second spaceship, and the virtual scene staging includes decelerating the first spaceship according to the deceleration of the motor vehicle by use of a tractor beam of the second spaceship.

23. A control device, comprising: a memory configured to store instructions; and a processor configured to execute the instructions stored in the memory to: control a display device being worn on a head of a vehicle occupant of a motor vehicle to display a virtual environment within which the vehicle occupant wearing the display device virtually moves according to a detected movement of the motor vehicle, determine a potential stopping point of the motor vehicle, and in response to a deceleration of the motor vehicle being detected within a predetermined environment of the potential stopping point, control the display device to output a virtual scene staging within the virtual environment so as to cause a deceleration of the virtual movement of the vehicle occupant within the virtual environment to correspond to the deceleration of the motor vehicle.

24. The control device according to claim 23, wherein the potential stopping point of the motor vehicle includes at least one of a traffic light, an intersection, a read junction, a traffic jam, or a location based on a braking of another motor vehicle.

25. The control device according to claim 23, further comprising a data interface configured to receive information including at least one of sensor data characterizing an environment of the motor vehicle, digital map data, traffic flow data, car-to-car communication data, or data characterizing a driving behavior of a driver of the motor vehicle, wherein the processor is configured to determine the potential stopping point of the motor vehicle based on the information received via the data interface.

26. The control device according to claim 23, wherein the processor is configured to execute the instructions stored in the memory to: control the display device to output the virtual scene staging within the virtual environment in response to the deceleration of the motor vehicle exceeding a predetermined deceleration level, in response to an acceleration of the motor vehicle exceeding a predetermined acceleration after the deceleration of the motor vehicle, control the display device to end the output of the virtual scene staging, and control the display device to display the virtual environment within which the vehicle occupant virtually moves according to the detected movement of the motor vehicle, and in response to no deceleration of the motor vehicle being detected when the motor vehicle passes the potential stopping point, not control the display device to output the virtual scene staging.

27. The control device according to claim 23, wherein the virtual environment includes a virtual scene having a virtual object with which the vehicle occupant wearing the display device virtually interacts, and in response to the deceleration of the motor vehicle being detected within the predetermined environment of the potential stopping point, the virtual object causes the virtual movement of the vehicle occupant to be decelerated.

28. A motor vehicle, comprising: a chassis; and a control device configured to: control a display device being worn on a head of a vehicle occupant of the motor vehicle to display a virtual environment within which the vehicle occupant wearing the display device virtually moves according to a detected movement of the motor vehicle, determine a potential stopping point of the motor vehicle, and in response to a deceleration of the motor vehicle being detected within a predetermined environment of the potential stopping point, control the display device to output a virtual scene staging within the virtual environment so as to cause a deceleration of the virtual movement of the vehicle occupant within the virtual environment to correspond to the deceleration of the motor vehicle.

29. The motor vehicle according to claim 28, wherein the potential stopping point of the motor vehicle includes at least one of a traffic light, an intersection, a read junction, a traffic jam, or a location based on a braking of another motor vehicle.

30. The motor vehicle according to claim 28, wherein the control device is configured to determine the potential stopping point of the motor vehicle based on information including at least one of sensor data characterizing an environment of the motor vehicle, digital map data, traffic flow data, car-to-car communication data, or data characterizing a driving behavior of a driver of the motor vehicle.

31. The motor vehicle according to claim 28, wherein the control device is configured to: control the display device to output the virtual scene staging within the virtual environment in response to the deceleration of the motor vehicle exceeding a predetermined deceleration level, in response to an acceleration of the motor vehicle exceeding a predetermined acceleration after the deceleration of the motor vehicle, control the display device to end the output of the virtual scene staging, and control the display device to display the virtual environment within which the vehicle occupant virtually moves according to the detected movement of the motor vehicle, and in response to no deceleration of the motor vehicle being detected when the motor vehicle passes the potential stopping point, not control the display device to output the virtual scene staging.

32. The motor vehicle according to claim 28, wherein the virtual environment includes a virtual scene having a virtual object with which the vehicle occupant wearing the display device virtually interacts, and in response to the deceleration of the motor vehicle being detected within the predetermined environment of the potential stopping point, the virtual object causes the virtual movement of the vehicle occupant to be decelerated.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0027] These and other aspects and advantages will become more apparent and more readily appreciated from the following description of example embodiments, taken in conjunction with the accompanying drawings, of which:

[0028] FIG. 1 is a schematic representation of a motor vehicle in which an occupant of the vehicle is sitting and is wearing virtual reality glasses, wherein the motor vehicle is approaching a potential stopping point; and

[0029] FIG. 2 is a schematic representation of a virtual environment displayed by the virtual reality glasses.

DETAILED DESCRIPTION

[0030] Reference will now be made in detail to examples which are illustrated in the accompanying drawings. In the drawings, elements having the same function are each denoted by the same reference symbols.

[0031] A motor vehicle 1, in which a vehicle occupant 2 is siting while wearing virtual reality glasses 3, is shown in a schematic representation in FIG. 1. In addition, a control device 4 is shown schematically, which is set up to control the virtual reality glasses 3. For example, the motor vehicle 1 can be an autonomous motor vehicle 1, so that the occupant 2 can easily consume virtual content by use of the virtual reality glasses 3 without awareness of the driving task. It may also be a non-autonomous motor vehicle 1, wherein a driver that is not shown here drives the motor vehicle 1 and the occupant 2 is only a passenger or co-driver. The motor vehicle 1 travels a certain route 5, along which various potential stopping points 6 may be located. These potential stopping points 6 may be, for example, traffic lights, intersections, road junctions, a traffic jam, or even other vehicles which are stopping. In addition, an environment 7 of the only potential stopping point 6 shown here is marked schematically, wherein this environment 7 is discussed in more detail later.

[0032] In FIG. 2, a virtual environment 8 is shown in a very schematic manner, which is displayed by use of the virtual reality glasses 3. While travelling with the motor vehicle 1, the control device 4 controls the virtual reality glasses 3 to display the virtual environment 8, within which the vehicle occupant 2 wearing the virtual reality glasses 3 is moving virtually corresponding to a detected movement of the motor vehicle 1. In the present schematically indicated case, the virtual environment 8 includes a space scene, through which the vehicle occupant 2 wearing the virtual reality glasses 3 is flying virtually with a first spaceship 9. The present representation does not correspond to the virtual perspective of the occupant of the vehicle 2.

[0033] In the present representation, a second spaceship 10 is flying above the first spaceship 9, which decelerates or stops the first spaceship 9 with a tractor beam 11. The second spaceship 10, together with the tractor beam 11, forms a virtual scene staging 12, which causes a deceleration in the virtual movement within the virtual environment 8. This is used to make the resulting virtual deceleration within the virtual environment 8 appear convincing in the case of an actual deceleration process or stopping process at the potential stopping point 6 (see FIG. 1).

[0034] A method for operating the virtual reality glasses 3 is explained in more detail below. While travelling with the motor vehicle 1, the control device 4 continuously receives data which characterize the movement of the motor vehicle 1. Depending on these data characterizing the movement of the motor vehicle, the control device 4 controls the virtual reality glasses 3 to display the virtual environment 8, within which the occupant 2 is moving with the first spaceship 9 according to the detected real movement of the motor vehicle 1. While travelling along the route 5, the control device 4 continuously determines potential stopping points 6 of the motor vehicle, which may be traffic lights, intersections, road junctions, traffic jams or other vehicles which are stopping.

[0035] For example, the control device 4 may receive sensor data characterizing an environment of the motor vehicle 1 from the motor vehicle 1. Based on these sensor data, the control device 4 can predictively detect potential stopping points 6 in the direction of travel. It is also possible for the control device 4 to receive digital map data, so that potential stopping points 6 can be determined based on these data. For example, the control device 4 can also receive traffic flow data from a server which is not shown here or the like in order to detect potential stopping points 6 along the route 5 at an early stage. It is also possible that the control device 4 will receive car-to-car data from other vehicles which are not shown here, for example vehicles ahead, in order to identify a deceleration of or stopping by vehicles ahead and thus potential stopping points 6. If the motor vehicle 1 is being driven manually by a driver that is not shown here, it is also possible that the control device 4 receives data characterizing a driving behavior of the driver in order to determine the potential stopping points 6 based on this.

[0036] Once a deceleration of the motor vehicle 1 is detected within the predetermined environment 7 of the respective determined potential stopping point 6, the control device 4 controls the virtual reality glasses 3 to output the virtual scene staging 12 within the virtual environment 8. Specifically, in the example described in FIG. 2, for example, from a determined approach of the vehicle 1 to the environment 7, the second spaceship 10 appears in the virtual field of view of the occupant 2 of the vehicle 1. If the motor vehicle 1 actually decelerates within the environment 7 when approaching the potential stopping point 6, the control device 4 controls the virtual reality glasses 3 in such a way that in the course of the virtual scene staging 12 the tractor beam 11 is activated within the virtual environment 8 and consequently the virtual movement is decelerated by the tractor beam 11 decelerating the movement of the first spaceship 9 according to the real deceleration of the motor vehicle 1.

[0037] If for example the motor vehicle 1 starts again after a stopping maneuver at the potential stopping point 6, the virtual scene staging 12 is terminated at least to the extent that the tractor beam 11 is terminated and allows the virtual movement of the first spaceship 9 within the virtual environment 8 according to the real movement of the motor vehicle 1. If, on the other hand, the case arises that the motor vehicle 1 approaches the potential stopping point 6 but does not stop there and passes the potential stopping point 6 without deceleration for example, the tractor beam 11 is not activated in the virtual environment 8 in the first place. Subsequently, it may be provided, for example, that the second spaceship 10 which appeared previously disappears from the virtual field of view again and thus from the virtual environment 8 after a certain phase of flight parallel to the first spaceship 9.

[0038] Contrary to the representation in FIG. 2, it is also possible, for example, that the occupant 2 of the vehicle is flying within the virtual environment 8 again within a space scene with a spaceship, wherein however, as the virtual scene staging a defect occurs on the spaceship, which is repaired as long as the vehicle 1 is stationary at the potential stopping point 6. This also allows the occupant 2 of the vehicle, who has put on the virtual reality glasses 3, to be given a particularly convincing explanation within the virtual experience of why he stops within the virtual environment 8.

[0039] In this context, it may also be provided, for example, that a warning light lights up within a cockpit of the spacecraft indicating a potential defect of the spacecraft if there has been a sufficient approach to the potential stopping point 6, for example on entering the environment 7 of the potential stopping point 6. For example, the warning light can first light up green in the cockpit of the spacecraft, wherein this lamp then turns to orange as soon as the motor vehicle 1 enters the environment 7. If the motor vehicle 1 then actually decelerates in order to stop at the potential stopping point 6, this lamp may, for example, turn red and the defect may occur in the form of the virtual scene staging, which is repaired as long as the motor vehicle 1 is at the potential stopping point 6. If, on the other hand, the vehicle simply passes the potential stopping point 6, the lamp previously illuminated orange can turn green again, which makes a trouble-free movement with the spacecraft within the virtual environment 8 appear convincing.

[0040] The example embodiments described in connection with the space scenes or spaceships are to be understood as examples. In principle, a wide variety of virtual scene stagings in the form of different virtual sequences can be displayed by use of the virtual reality glasses 3 in order to correspond to an actual deceleration and stop of the motor vehicle 1 at potential stopping points 6 and thus also a virtual deceleration and stop within the displayed virtual environment 8 to appear convincing. Instead of virtual reality glasses 3, other display devices that can be worn on the head, such as augmented reality glasses or augmented reality contact lenses, can also be used. Overall, the method explained and the control device 4 explained also provide a solution which enables a particularly immersive and compelling virtual experience by use of a display device worn on the head of a vehicle occupant 2.

[0041] A description has been provided with reference to various examples, but it will be understood that variations and modifications can be effected within the spirit and scope of the claims which may include the phrase “at least one of A, B and C” as an alternative expression that means one or more of A, B and C may be used, contrary to the holding in Superguide v. DIRECTV, 358 F3d 870, 69 USPQ2d 1865 (Fed. Cir. 2004).