METHOD FOR OPERATING A HEAD-MOUNTED DISPLAY APPARATUS IN A MOTOR VEHICLE, CONTROL DEVICE, AND HEAD-MOUNTED DISPLAY APPARATUS

Abstract

A control device in a motor vehicle receives a signal from a sensor device which includes at least one image of the current interior situation. The control device generates a superimposition signal containing the at least one image of the current interior situation and transmits the superimposition signal to a display element of a display apparatus. The display apparatus superimposes the at least one image of the current interior situation onto predefined output content output to a user.

Claims

1-10. (canceled)

11. A method for operating a head-mounted display device while the head-mounted display device is located in an interior of a motor vehicle and is configured to output specified output content of an entertainment offering, comprising: receiving, by a controller, a sensor signal from a sensor device, the sensor signal including at least one image of a current interior situation of the motor vehicle; generating, by the controller, a superposition signal instructing superposition of the at least one image of the current interior situation on the specified output content; and transmitting the superposition signal from the controller to a display element of the display device, thereby causing the display device to superimpose the at least one image of the current interior situation on the specified output content.

12. The method as claimed in claim 11, wherein the superposition signal instructs the display device to superimpose the at least one image of the current interior situation in only a partial region of a visual field of a user of the display device.

13. The method as claimed in claim 12, wherein the superposition signal instructs the display device to superimpose the at least one image of the current interior situation in one of a primary visual field and a central visual field.

14. The method as claimed in claim 12, wherein the sensor signal provides at least one camera image of an interior camera of the sensor device.

15. The method as claimed in claim 14, wherein the controller receives the sensor signal from the sensor device of the motor vehicle.

16. The method as claimed in claim 14, wherein the specified output content defines a 360-degree virtual reality.

17. The method as claimed in claim 14, further comprising determining, by the controller, that an occupant is speaking, as a prerequisite for said transmitting of the superposition signal.

18. The method as claimed in claim 17, wherein the occupant determined to be speaking is not the user of the display device.

19. The method as claimed in claim 18, wherein said determining is in dependence on a volume of a voice of the occupant.

20. The method as claimed in claim 17, further comprising determining, by the controller, whether the motor vehicle is approaching a destination, and wherein at least one of said generating and said transmitting of the superposition signal is performed after determining that the motor vehicle approaches the destination.

21. The method as claimed in claim 11, wherein the superposition signal instructs the display device to superimpose the at least one image of the current interior situation in only one of a primary visual field and a central visual field.

22. The method as claimed in claim 11, further comprising determining, by the controller, that an occupant is speaking, as a prerequisite for said transmitting of the superposition signal.

23. The method as claimed in claim 22, wherein the occupant determined to be speaking is not the user of the display device.

24. The method as claimed in claim 23, wherein said determining is in dependence on a volume of a voice of the occupant.

25. A controller in a motor vehicle where a sensor device outputs a sensor signal and a display device displays specified output content of an entertainment offering, said controller comprising: a memory; and a processor, connected to the memory, configured to receive the sensor signal from the sensor device, the sensor signal including at least one image of a current interior situation of the motor vehicle, generate a superposition signal instructing superposition of the at least one image of the current interior situation on the specified output content, and transmit the superposition signal from the controller to the display device, causing the display device to superimpose the at least one image of the current interior situation on the specified output content.

26. The controller as claimed in claim 25, wherein the method further comprises determining that an occupant is speaking, before the superposition signal is transmitted.

27. A motor vehicle, where a display device displays specified output content of an entertainment offering, comprising: a sensor device configured to output a sensor signal including at least one image of a current interior situation of the motor vehicle; and a controller configured to receive the sensor signal from the sensor device, generate a superposition signal instructing superposition of the at least one image of the current interior situation on the specified output content, and transmit the superposition signal from the controller to the display device, causing the display device to superimpose the at least one image of the current interior situation on the specified output content.

28. The motor vehicle as claimed in claim 27, wherein the controller is further configured to determine whether the motor vehicle is approaching a destination, and at least one of generate and transmit the superposition signal after determining that the motor vehicle approaches the destination.

29. A head-mounted display device, operated in a motor vehicle having a sensor device configured to output a sensor signal including at least one image of a current interior situation of the motor vehicle; said head-mounted display device comprising: a display element; and a controller configured to. display specified output content of an entertainment offering on the display element, receive the sensor signal from the sensor device, generate a superposition signal instructing superposition of the at least one image of the current interior situation on the specified output content, and transmit the superposition signal from the controller to the display device, causing the display device to superimpose the at least one image of the current interior situation on the specified output content.

30. The head-mounted display device as claimed in claim 29, wherein the superposition signal instructs the display device to superimpose the at least one image of the current interior situation only in one of a primary visual field and a central visual field, forming less than all of a total visual field of a user of the display device.

Description

[0035] In the figures, identical reference signs in each case refer to elements having the same functions.

[0036] The FIGURE illustrates the principle of the method according to the invention and of the apparatuses according to the invention with reference to a first exemplary embodiment. In this regard, the FIGURE shows a motor vehicle 10, for example a passenger car of a ride-hailing driving service. In the example of the FIGURE, the controller 12 can be, for example, a control device or a control chip of the motor vehicle 10 but can alternatively be installed in the head-mounted display device 14. The head-mounted display device 14 can have for example a screen as the display element (not shown in the FIGURE).

[0037] Communication with the display device 14 takes place via a data communication link 16. For example, if the controller 12 is installed in the display device 14, the data communication link 16 can be a wire-based data communication link, for example a cable or a databus. If the controller 12 is a device component of the motor vehicle 10, the data communication link 16 for communicating between the controller 12 and the display device 14 can likewise be a wire-based data communication link 16, for example a cable, but is preferably a wireless data communication link 16, for example a WLAN, mobile radio, Internet, or Bluetooth connection. The controller 12 of the FIGURE here shows an optional processor device 20 and also an optional data memory 22.

[0038] If the controller 12 is a component of the motor vehicle 10, the data communication link 16 to the sensor device 18, which is illustrated in the FIGURE as an interior camera, can be implemented for example via a databus of the motor vehicle 10, for example a CAN bus.

[0039] In the example of the FIGURE, a user of the exemplary ride-hailing driving service of the motor vehicle 10 can have booked, for example, a journey of ten kilometers or a journey with a driving time of half an hour. The entertainment by means of the head-mounted display device 14 can for example constitute an additional service of this driving service. The user can either have brought along their own display device 14, or the display device 14 can be provided by the driving service. For the journey, for example, an entertainment offering that can describe a trip in a virtual reality can be provided during the journey, in particular a 360-degree virtual reality. The output content can be, for example, a Jurassic Park-based virtual reality, that is to say a form of entertainment that describes a virtual trip through a prehistoric landscape.

[0040] The output of the specified output content can begin for example at the start of the journey. In method step S1, the controller 12 receives at least one sensor signal that ideally describes the interior from a plurality of interior cameras of the motor vehicle 10. Although the FIGURE shows by way of example only one camera of the sensor device 18, it is ideally possible for a plurality of cameras to be arranged such that the interior is filmed, for example, from different perspectives. The images or image series of the individual cameras can then either be described in one sensor signal or in a plurality of sensor signals.

[0041] The superposition signal generated in method step S2 can thus describe a camera image or the camera images of a plurality of interior cameras. After the generated superposition signal has been transmitted (S3) to the display element of the display device 14, the latter is caused to superpose (S4) the corresponding image or the corresponding images preferably into a peripheral visual field of the user. If the controller 12 is a controller 12 of the motor vehicle 10, the generated superposition signal is thus also first transmitted to the display device 14 (S3).

[0042] Owing to the superposed image (or owing to the superposed images or image series), the user is able to see for example other, unfamiliar passengers and the unfamiliar driver. If for example the driver wishes to say something to the user, for example if the driver has a question, the user of the display device 14 sees that the driver is turning toward them. Even if the user of the display device 14 additionally receives through the display device 14 audio content from the virtual reality, for example via earphones, said user can recognize that the driver wishes to speak with them. So if the driver initially receives no reply from the user and decides to shake the shoulder of the user of the display device 14, this will not be unexpected for the user and the user of the display device 14 will not be startled. In addition, the user of the display device 14 is able to react much earlier anyway and can prevent the shaking of the shoulder, for example by taking off the display device 14 in order to speak to the driver.

[0043] In order to avoid constantly drawing the attention of the user of the display device 14 to the superposed image, but to do so only in specific situations, the controller 12 may generate the superposition signal (S2) or transmit it (S3) only if the controller 12 ascertains (S5), for example by means of a microphone (not shown in the FIGURE), that for example the driver has just started to speak. It is optionally possible here for a threshold value for a volume to be additionally stored, to the effect that the image is for example not yet superposed if the driver merely mumbles quietly to themselves.

[0044] Alternatively or additionally, the superposition signal can be generated (S2) or the generated superposition signal can be transmitted (S3) only if the controller 12 ascertains (S6), for example on the basis of GPS data or of navigational data ascertained using a navigation device of the motor vehicle 10, that the motor vehicle 10 is moving for example at a distance of only one kilometer from the destination. If the interior signal is superposed only at that time, the user of the display device 14 will know that they are nearly there and for this reason is better able to understand that any agitation they felt in the interior of the motor vehicle 10 is due to the fact that other, unfamiliar passengers are beginning to gather their belongings.

[0045] Overall, the examples show how the invention can be used to provide a head-mounted display device 14, preferably VR glasses.

[0046] In a further example it is possible, for example when using VR glasses in the motor vehicle 10, to superpose (S4) into the exemplary VR environment, in addition to the main content, a (for example generally small) camera image that can communicate the situation in the vehicle interior and possibly also outside the motor vehicle 10 to the VR user, with the result that the latter will always have an awareness of the current situation in and around the motor vehicle 10.