OPTO-ELECTRONIC DEVICE AND METHOD FOR PROVIDING VISUAL ORIENTATION

Abstract

An opto-electronic device (100) is disclosed. The opto-electronic device (100) comprises at least one sensor (106) configured to obtain head orientation data including information on an orientation of a user's head, a processing unit (112) configured to process the head orientation data so as to obtain self motion data including information on the user's self motion and to provide at least one visual cue (114) based on the obtained self motion data, and a display device (120) configured to display the at least one visual cue (114) at a dis-play (102) at least in a periphery of a visual field of the user. Further, a method for providing visual orientation is disclosed.

Claims

1. An opto-electronic device, comprising at least one sensor configured to obtain head orientation data including information on an orientation of a user's head, a processing unit configured to process the head orientation data so as to obtain self motion data including information on the user's self motion and to provide at least one visual cue based on the obtained self motion data, and a display device configured to display the at least one visual cue at a display at least in a periphery of a visual field of the user.

2. The opto-electronic device according to the preceding claim 1, wherein the sensor includes at least an inertial measurement unit and/or a camera and/or a LIDAR device configured to obtain the head orientation data and head movement data including information on a movement of the user's head.

3. The opto-electronic device according to claim 1, wherein the display device is configured to display the visual cue at the display as a virtually stationary foreground with a predetermined distance to the user.

4. The opto-electronic device according to claim 3, wherein the display device is further configured to provide the visual cue with an optical flow.

5. The opto-electronic device according to claim 4, wherein the display device is configured to vary a size at least of first parts of the visual cue and to fade in or fade out at least of second parts of the visual cue so as to provide the optical flow.

6. The opto-electronic device according to claim 5, wherein the display device is configured to expand the first parts of the visual cue and to fade in the second parts of the visual cue if a distance from the visual cue to the user decreases, and wherein the display device is configured to contract the first parts of the visual cue and to fade out the second parts of the visual cue if a distance from the visual cue to the user increases.

7. The opto-electronic device according to claim 1, wherein the display device is configured to display the visual cue so as to form a contrast relative to a background in the visual field of the user.

8. The opto-electronic device according to claim 1, wherein the visual cue is at least one cue selected from the group consisting of: horizontal lines, vertical lines, a shape of a room, a shape of a tunnel, a dot pattern, a checked pattern and a pattern with depth information.

9. The opto-electronic device according to claim 1, wherein the display device is configured to vary the visual cue based on the obtained self motion data.

10. The opto-electronic device according to claim 1, wherein the display device is configured to limit a shifting of the visual cue to maximum 4°/s.

11. The opto-electronic device according to claim 1, wherein the display device is configured to adjust a resolution of the visual cue based on a distance of the visual cue from a user's fovea.

12. The opto-electronic device according to claim 1, wherein the processing unit is configured to provide a plurality of visual cues based on the obtained self motion data, and the display device is configured to display the plurality of visual cues at the display at least in the periphery of the visual field of the user, wherein the visual cues are identical to one another or are different from one another.

13. The opto-electronic device according to claim 1, wherein the display is a transparent screen or a user's retina.

14. The opto-electronic device according to claim 1, wherein the opto-electronic device is configured to be integrated into or attached to a visual aid or is formed as smart glasses.

15. A method for providing visual orientation, comprising obtaining head orientation data including information on an orientation of a user's head, processing the head orientation data so as to obtain self motion data including information on the user's self motion and providing at least one visual cue based on the obtained self motion data, and displaying the at least one visual cue at a display at least in a periphery of a visual field of the user.

16. The opto-electronic device according to claim 5, wherein the display device is further configured to gradually vary the size of the first parts of the visual cue and to fade in or fade out the second parts of the visual cue so as to provide a gradual optical flow.

17. The opto-electronic device according to claim 1, wherein the visual cue is configured to provide parallax information to the user.

18. The opto-electronic device according to claim 1, wherein the visual cue is a space veridical orientation cue.

19. The opto-electronic device according to claim 11, wherein the display device is configured to increase the resolution of the visual cue based on a decreasing distance of the visual cue from the user's fovea.

20. The opto-electronic device according to claim 12, wherein the display device is configured to display the plurality of visual cues at the display in a subsequent order or at the same time.

Description

SHORT DESCRIPTION OF THE FIGURES

[0135] Further optional features and embodiments will be disclosed in more detail in the subsequent description of embodiments, preferably in conjunction with the dependent claims. Therein, the respective optional features may be realized in an isolated fashion as well as in any arbitrary feasible combination, as the skilled person will realize. The scope of the invention is not restricted by the preferred embodiments. The embodiments are schematically depicted in the Figures. Therein, identical reference numbers in these Figures refer to identical or functionally comparable elements.

[0136] In the Figures:

[0137] FIG. 1 shows a schematic illustration of setup of an opto-electronic device according to an embodiment of the present invention;

[0138] FIG. 2 shows a perspective view of the inertial measurement unit;

[0139] FIG. 3 shows a perspective view of the processing unit;

[0140] FIG. 4 shows an example for visual cues;

[0141] FIG. 5 shows an example for the provision of the optical flow;

[0142] FIG. 6 shows a perspective view of a visual aid;

[0143] FIG. 7 shows a flowchart of a method for providing visual orientation.

DETAILED DESCRIPTION OF THE EMBODIMENTS

[0144] FIG. 1 shows a schematic illustration of setup of an opto-electronic device 100 according to an embodiment of the present invention. Particularly, FIG. 1 shows a block diagram of the components of the opto-electronic device 100. The opto-electronic device 100 is configured to be used together with a display 102. The display 102 is a transparent screen 104. The display 102 is a device separate from the opto-electronic device 100. It is explicitly stated that the display 102 may alternatively be part of the opto-electronic device 100. With other words, the optoelectronic device 100 and the display 102 may be integrated as one unit.

[0145] The opto-electronic device 100 further comprises at least one sensor 106 configured to obtain head orientation data including information on an orientation of a user's head. The sensor 106 includes at least an inertial measurement unit 108.

[0146] FIG. 2 shows a perspective view of the inertial measurement unit 108. As shown in FIG. 2, the inertial measurement unit 108 may be disposed on a printed circuit board 110. The inertial measurement unit 108 is configured to obtain the head orientation data and head movement data including information on a movement of the user's head. Alternatively or in addition, the sensor 106 includes a camera and/or a LIDAR device configured to obtain the head orientation data and head movement data including information on a movement of the user's head.

[0147] The opto-electronic device 100 further comprises a processing unit 112 configured to process the head orientation data so as to obtain self motion data including information on the user's self motion.

[0148] FIG. 3 shows a perspective view of the processing unit 112. As shown in FIG. 3, the processing unit 112 includes a processing device 116 such as a microprocessor and an application-specific integrated circuit (ASIC) 118.

[0149] The processing unit 112 is further configured to provide at least one visual cue 114 based on the obtained self motion data. The visual cue 114 is configured to provide parallax information to the user. The provision of the parallax information may be realized in that the projection of the visual cue 114 onto the display 102 is moved inversely proportional to a self-motion of the user. This may also be realized by two patterns which are moved at different velocities relative to the user. For example, two or more superimposed projections are illustrated at the same time, wherein a first one is rather slow in the background and a rather fast one is in the foreground. Particularly, the visual cue 114 is a space veridical orientation cue. The visual cue 114 is at least one cue selected from the group consisting of: horizontal lines, vertical lines, a shape of a room, a shape of a tunnel, a dot pattern, a checked pattern and a pattern with depth information.

[0150] The opto-electronic device 100 further comprises a display device 120 configured to display the at least one visual cue 114 at the display 102 at least in a periphery of a visual field of the user. The display device 120 may be an output device configured to project the visual cue 114 onto the display 102 such as a projector. Particularly, the display device 120 is configured to display the visual cue 114 at the display 102 as a virtually stationary foreground with a predetermined distance to the user.

[0151] FIG. 4 shows an example for visual cues 114 displayed by the display device 120. The display device 120 is configured to display the visual cue 114 so as to form a contrast relative to a background 122 in the visual field of the user. FIG. 4 shows an example where the background 122 is rather dark and includes only two minor bright spots 124. Such a dark background scene is not suitable to provide visual orientation to the user. As is further shown in FIG. 4, the present invention addresses this issue by displaying the visual cue 114 onto the display 102. Merely as an example, the displayed visual cue 114 includes a plurality of bright dots or spots 126. With other words, the visual cue 114 is a dot pattern, wherein the dots are arranged as an irregular pattern. Needless to say, the dots may be arranged as a regular pattern depending on the respective situation.

[0152] The display device 120 is further configured to provide the visual cue 114 with an optical flow. The display device 120 is configured to vary a size at least of first parts 128 of the visual cue 114 and to fade in or fade out at least of second parts 130 of the visual cue 114 so as to provide the optical flow. Particularly, the display device 120 is configured to gradually vary the size of the first parts 128 of the visual cue 114 and to fade in or fade out the second parts 130 of the visual cue 114 so as to provide a gradual optical flow. For this purpose, the display device 120 is configured to expand the first parts 128 of the visual cue 114 and to fade in the second parts 130 of the visual cue if a distance from the visual cue 114 to the user decreases. Further, the display device 120 is configured to contract the first parts 128 of the visual cue 114 and to fade out the second parts 130 of the visual cue 114 if a distance from the visual cue 114 to the user increases. The first parts 128 and the second parts 130 of the visual cue 114 may be different from one another or may be identical to one another.

[0153] FIG. 5 shows an example for the provision of the optical flow by the display device 120. As shown in FIG. 5, the example for the visual cue 114 comprises a plurality of squares arranged in a regular pattern. Needless to say, the following explanations apply to any other pattern as well unless otherwise stated. FIG. 5 is divided in three portions representing exemplary steps for the provision of the optical flow. The provision of the optical flow will be described with the two steps arranged from the top to the bottom and two steps arranged from the bottom to the top. As shown in the upper portion of FIG. 5, the visual cue 114 comprises first parts 128 and second parts 130, wherein in the upper portion, the first parts 128 are greater than the second parts 130. In order to provide the optical flow, the second parts 130 are expanded and the first parts 128 are faded out in a first step when proceeding towards the middle portion of FIG. 5 as indicated by arrow 132. Subsequently, the first parts 128 indicated by large opaque squares disappear or are contracted and new small squares appear, respectively, when proceeding towards the lower portion of FIG. 5 as indicated by arrow 134. Further, the second parts 130 become bolt or darker.

[0154] As is further shown in the lower portion of FIG. 5, the second parts 130 are greater than the first parts 128. In order to further provide the optical flow, the first parts 128 are expanded such that large opaque squares appear, the small squares disappear and the second parts 130 are faded out when proceeding towards the middle portion of FIG. 5 as indicated by arrow 136. Subsequently, the second parts 130 indicated by large opaque squares disappear or are contracted and new small squares appear when proceeding towards the upper portion of FIG. 5 as indicated by arrow 138. Further, the first parts 128 are faded in or become bolt or darker. Thus, a pattern change for motions perpendicular to the display 102 is provided.

[0155] The display device 120 is further configured to vary the visual cue 114 based on the obtained self motion data. Thus, depending on the kind of detected self motion, the kind of visual cue 114 is illustrated. The display device 120 is configured to limit a shifting of the visual cue 114 to maximum 4°/s. Thus, any shifting or displacement of the illustrated visual cue 114 does not exceed a maximum resolution of a human's eye of 4°/s retinal slip. For higher velocities, a kind of saturation is thereby provided. Further, the display device 120 is configured to adjust a resolution of the visual cue 114 based on a distance of the visual cue 114 from a user's fovea. The display device 120 is configured to increase the resolution of the visual cue based on a decreasing distance of the visual cue from the user's fovea. The processing unit is configured to provide a plurality of visual cues 114 based on the obtained self motion data. The display device 120 is configured to display the plurality of visual cues 114 at the display at least in the periphery of the visual field of the user. The visual cues may be identical to one another or may be different from one another. Particularly, the display device 120 is configured to display the plurality of visual cues 114 at the display 102 in a subsequent order or at the same time.

[0156] FIG. 6 shows a perspective view of a visual aid 140. FIG. 6 shows the opto-electronic device 100 integrated into or attached to the visual aid 140. The visual aid 140 is glasses. In this example, the display 102 is the spectacle lenses of the visual aid 140. The opto-electronic device 100 is integrated into or attached to an upper edge or frame of the spectacle lenses. It has to be noted that the opto-electronic device 100 may be located at the right and/or left upper edge or frame of the spectacle lenses. thus, also more than one opto-electronic device 100 may be present. With this set-up, the display device 120 projects at least one visual cue 114 at least onto the temple pieces of the glasses or an area adjacent or close thereto corresponding a periphery of the user's visual field. In addition, the display device 120 may project visual cues 114 onto the spectacle lenses of the glasses. In latter case, the display 102 is a transparent screen. Alternatively, the display 102 may be a user's retina. Alternatively, the opto-electronic device 100 may be a so-called stand-alone solution comprising the display 102. For example, the opto-electronic device 100 may be formed as smart glasses.

[0157] Translational motions of the user's head in the dimensions of the display 102 are viewed as horizontal or vertical shifts of the displayed pattern of the visual cue 114. Motions of the user's head perpendicular to the display 102 are viewed as contraction or expansion with corresponding fading in and out of the displayed pattern of the visual cue 114. Rotations are displayed as corresponding translations, where e.g. a clockwise rotation of the head will be displayed as backward motion on the left and forward motion of the pattern of the visual cue 114 on the right peripheral display 102.

[0158] Hereinafter, a method for providing visual orientation according to the present invention will be described. FIG. 7 shows a flowchart of the method. The method may be carried out using the opto-electronic device 100 as described above. The method may be computer-implemented. The method comprises a step S10 which includes obtaining head orientation data including information on an orientation of a user's head. Step 10 may be carried out by the inertial measurement unit 108. Subsequently, the method further proceeds to step S12 which includes processing the head orientation data so as to obtain self motion data including information on the user's self motion and providing at least one visual cue 114 based on the obtained self motion data. Step S12 may be carried out by the processing unit 112. Subsequently, the method further proceeds to step S14 which includes displaying the at least one visual cue 114 at a display 102 at least in a periphery of a visual field of the user. The visual cue 114 is displayed as a virtually stationary foreground with a predetermined distance to the user. The visual cue 114 is displayed so as to form a contrast relative to a background in the visual field of the user as explained above with reference to FIG. 4. The visual cue 114 provides parallax information to the user. The visual cue 114 is a space veridical orientation cue. The visual cue 114 and its shape, respectively, may be varied based on the obtained self motion data.

[0159] As described above with reference to FIG. 5, the method may further comprise providing the visual cue 114 with an optical flow. This may be realized by varying a size at least of first parts 128 of the visual cue 114 and fading in or fading out at least of second parts 130 of the visual cue 114 so as to provide the optical flow. Particularly, this may be realized by gradually varying a size at least of first parts 128 of the visual cue 114 and gradually fading in or fading out at least of second parts 130 of the visual cue 114 so as to provide a gradual optical flow. As explained above, the first parts 128 and the second parts 130 of the visual cue 114 may be different from one another or may be identical to one another.

[0160] The method may further comprise limiting a shifting of the visual cue to maximum 4°/s. The method may further comprise adjusting a resolution of the visual cue 114 based on a distance of the visual cue 114 from a user's fovea. Particularly, the resolution of the visual cue 114 may be increased based on a decreasing distance of the visual cue 114 from the user's fovea. The method may further comprise providing a plurality of visual cues 114 based on the obtained self motion data, and displaying the plurality of visual cues 114 at the display 102 at least in the periphery of the visual field of the user. The visual cues 114 may be identical to one another or may be different from one another. The plurality of visual cues 114 may be displayed at the display 102 in a subsequent order or at the same time.

LIST OF REFERENCE NUMBERS

[0161] 100 opto-electronic device [0162] 102 display [0163] 104 transparent screen [0164] 106 sensor [0165] 108 inertial measurement unit [0166] 110 printed circuit board [0167] 112 processing unit [0168] 114 visual cue [0169] 116 processing device [0170] 118 ASIC [0171] 120 display device [0172] 122 background [0173] 124 bright spot [0174] 126 bright spot [0175] 128 first part [0176] 130 second part [0177] 132 arrow [0178] 134 arrow [0179] 136 arrow [0180] 138 arrow [0181] 140 visual aid [0182] S10 obtaining head orientation data [0183] S12 processing the head orientation data [0184] S14 displaying the at least one visual cue