Display devices with reflectors

Abstract

A display device has a display, operable to generate a real image, and an optical system. In the optical system are at least two free-form reflective surfaces, S I and S2. At least one of the reflective surfaces is convex in one direction at substantially all points of its optically active area. Light rays from the display are reflected on SI before they are reflected on S2. The reflective surfaces SI and S2 are arranged to generate a virtual image from the real image on the display, by projecting light from the display to an eye position. The field of view occupied by the virtual image as seen from the eye position is greater than 50 degrees in at least one direction, preferably the direction linking the two eyes of an intended user.

Claims

1. A display device comprising: a single display, operable to generate a real image; and an imaging optical system, comprising at least first and second free-form reflective surfaces, such that at least one of the first and second reflective surfaces is convex in at least one direction at substantially all points; wherein the first and second reflective surfaces are arranged to generate a virtual image from the real image on the single display, by projecting light from the single display to a position of a single eye; wherein the first and second reflective surfaces are configured to provide only one reflection each; wherein the field of view occupied by the virtual image as seen from the eye position is greater than 50 degrees in at least one direction; wherein the imaging optical system is arranged to produce the virtual image that when projected by a normal human eye onto the retina of said eye when said eye is at a position of an imaginary sphere, the resolution of the part of said virtual image to be projected onto a 1.5 mm fovea when the pupil of the eye is within a pupil range has a higher resolution than the resolution of the same part of said virtual image when said pupil is at a different position within the pupil range such that said part of said virtual image is to be projected outside the fovea, said resolution difference being adapted to the visual resolution curve in the periphery of a normal human eye.

2. The display device of claim 1, wherein the at least one of the first and second reflective surfaces that is convex in at least one direction at substantially all optically active points is the first reflective surface, wherein the second reflective surface is concave at substantially all optically active points, and wherein the light rays from the display are reflected on the first reflective surface before they are reflected on the second reflective surface.

3. The display device of claim 2, wherein the first and second reflective surfaces are arranged to generate a virtual image from the real image on the display, by projecting light without further reflections from the single display to an eye position.

4. The display device of claim 1, further comprising a lens in the optical path between the single display and the eye position, and wherein the field of view in the at least one direction is more than 80 degrees.

5. The display device of claim 4, wherein the lens is between the single display and the first reflective surface.

6. The display device of claim 4, wherein the lens is between the first reflective surface and the second reflective surface.

7. The display device of claim 1, wherein the second reflective surface is semitransparent and is on a transparent substrate, permitting a direct view of an external environment from the eye position.

8. The display device of claim 7, wherein the second reflective surface is on a surface of the transparent substrate nearer to the eye position or is adjacent to an additional transparent substrate further from the eye than the second reflective surface, and wherein a frontward entrance surface of said transparent substrate or said additional transparent substrate, through which light from the external environment enters the optical system, is so formed that light rays from the external environment exiting the optical system to the eye position exits the optical system substantially parallel to directions in which the respective rays entered the optical system.

9. The display device of claim 7, wherein the single display is offset laterally from a direct line of view of the eye.

10. The display device of claim 1, wherein the optical system is configured such that when placed at a distance between 5 and 40 mm from an anticipated forward periphery of an eye position defined by an imaginary 13 mm radius sphere, the optical system subtending a solid angle from a closest point horizontally on the imaginary sphere comprising a cone with 40 degrees whole angle, wherein the single display is on a side of the optical system remote from the imaginary sphere, at a distance from the optical system of no more than 40 mm.

11. A headgear comprising the display device of claim 1, with a mount for positioning the display device on a human head with the eye position of the display device coinciding with an eye of the human.

12. The headgear of claim 11, wherein said at least one direction in which the field of view occupied by the virtual image as seen from the eye position is greater than 50 degrees is parallel to a direction joining the eyes of a person wearing the headgear.

13. The headgear of claim 11, further comprising a second display device, mounted with an eye position of the second display device coinciding with a second eye of the human.

14. The display device of claim 1, wherein the reflective surfaces are mirrored.

Description

BRIEF DESCRIPTION OF DRAWINGS

(1) The above and other aspects, features and advantages will be apparent from the following more particular description of certain embodiments, presented in conjunction with the following drawings. In the drawings:

(2) FIG. 1 shows a Head Mounted Display (HMD) with a free-form wedge-shaped prism designed by Morishima et al. in 1995 (prior art).

(3) FIG. 2 shows a mobile camera optical lens system designed by Ho Sik You et al., in 2010 (prior art).

(4) FIG. 3 shows a two-mirror free-form HMD for Virtual Reality.

(5) FIG. 4 shows a two-mirror free-form HMD with an additional lens providing a larger field of view.

(6) FIG. 5 shows a two-mirror free-form HMD with an additional lens between the display and the mirrors.

(7) FIG. 6 shows a two-mirror free-form HMD with an additional lens between the mirrors.

(8) FIG. 7 shows a configuration for Virtual Reality (VR) applications with four free-form optical surfaces.

(9) FIG. 8 shows an HMD with a free-form wedge-shaped prism lens with an additional free-form lens providing a larger field of view.

(10) FIG. 9 shows a two-mirror free-form HMD for Augmented Reality (AR) applications.

(11) FIG. 10 shows a two-mirror free-form HMD for AR with an additional lens.

(12) FIG. 11 shows an embodiment for AR with four free-form optical surfaces.

(13) FIG. 12 shows an HMD for AR with a free-form wedge-shaped prism and with an additional free-form lens.

DETAILED DESCRIPTION

(14) The embodiments to be described here are designed for HMD devices, suitable for augmented reality (AR) or virtual reality (VR) applications. These embodiments aim to provide simultaneously a wide field of view, high resolution, low weight, and small volume.

(15) For an effective immersive experience, this wide field of view must to be provided independently of the eye pupil orientation relative to the head. This approach considers the pupil range as a design parameter. The maximum pupil range is the region of the eye sphere formed by the union of all physically accessible pupil positions for an average human. It is then a spherical shell in good approximation. The boundary of the maximum pupil range is approximately an ellipse with angular horizontal semi-axis of 60 degs and vertical semi-axis of 45 degs relative to the front direction, subtended at the center of rotation of the eye. However, for a practical immersive design, an elliptical cone of semi-axis in the 15 to 30 degrees range can be considered sufficient for the pupil range definition.

(16) Human vision resolution peaks on the part of the scene imaged at the fovea (which is about 1.5 mm in diameter) and decreases rapidly away from that part. Therefore, the angular resolution of a typical human eye is a decreasing function of the peripheral angle (according to J. J. Kerr, Visual resolution in the periphery, Perception & Psychophysics, Vol. 9 (3), 1971). Since the human eye resolution is much coarser in peripheral vision than close to the gazing direction, the embodiments in this specification have been designed to match the imaging quality so that the ipixels of the virtual screen are no finer than strictly needed (because the eye will not appreciate further increase in fineness).

(17) Embodiments shown here consist of:

(18) A display whose surface coincides preferably with a plane or a cylinder, and which is composed by a multiplicity of physical pixels called object pixels or opixels.

(19) An optical system, which can contain various numbers of refractive/reflective surfaces, providing a virtual image composed by pixels on a virtual screen, called ipixels. The virtual screen is preferably spherical, lying at a certain distance from the eye; and the virtual image is defined by a mapping from opixels to ipixels.

(20) FIG. 3 shows a two-mirror embodiment together with user's eye 314 and nose 315 shown for orientation purpose. This design is denoted an XX (which means that the lens consists of two surfaces, both being reflective) design. Rays 305, 306, 307 and 308 emitted by the digital display 301 undergo 2 reflections: first on freeform mirror surface 302 and then on the see-through free-form mirror 303 to be directed to the eye 314. After 2 reflections, rays 306 and 308 are directed towards the eye sphere center 310. The rays 305 and 307 define the vertical FoV of this mirror device as these are edge rays of the eye input pupil center 309 when eye rests looking forward and these rays come from the edges of the display 301. The embodiment generates a virtual image from the real image that appears on the display. The distortions of the virtual image, if any, are corrected electronically by imparting a contrary distortion to the real image on display 301.

(21) FIG. 3 shows an example of an XX design with focal length f=52 mm, and 35 of vertical FoV and 60 of horizontal FoV. FIG. 3 left shows a vertical cross-section at the x=0 plane. Surfaces are described with the equation z=P.sub.m(x,y):

(22) $\mathrm{Pm}\left(x,y\right)=\underset{i=0}{\overset{\frac{m}{2}}{.Math.}}\underset{j=0}{\overset{m}{.Math.}}{c}_{2i,j}{P}_{2i}\left(\frac{x-\frac{x_{\max }+x_{\min }}{2}}{x_{\max }}\right)P_i\left(\frac{y-\frac{y_{\max }+y_{\min }}{2}}{y_{\max }}\right),$

(23) where Pm(x,y) is the 10.sup.th order polynomial, i.e. m=10, c.sub.2i,j are surface coefficients listed in Table 1 below, and P.sub.2i((x(x.sub.max+x.sub.min)/2)/x.sub.max) and P.sub.j((y(y.sub.max+y.sub.min)/2)/y.sub.max) are Legendre-polynomials that are orthogonal inside the rectangle x.sub.min<x<x.sub.max, and y.sub.min<y<y.sub.max. All surfaces are symmetric respect to the plane x=0 (the plane of the drawing shown in FIG. 3) so Legendre polynomial P.sub.2i ((x(x.sub.max+x.sub.min)/2)/x.sub.max) has only pair order monomials. The Legendre polynomial P.sub.n(x) can be expressed as:

(24) $P_n\left(x\right)=\frac{1}{2^n}\underset{k=0}{\overset{n}{.Math.}}{\left(\begin{array}{c}n\\ k\end{array}\right)}^2{\left(x-1\right)}^{n-k}{\left(x+1\right)}^k=2^n\underset{k=0}{\overset{n}{.Math.}}{x}^k\left(\begin{array}{c}n\\ k\end{array}\right)\left(\frac{n+k-1}{\begin{array}{c}2\\ n\end{array}}\right)$
Where the latter expresses the Legendre polynomials by simple monomials and involves the multiplicative formula of the binomial coefficient, and where

(25) $\left(\begin{array}{c}n\\ k\end{array}\right)=\frac{n!}{k!\left(n-k\right)!}.$

(26) The origin of the global coordinate system (x,y,z)=(0, 0, 0) is placed in the eye center 310. The local coordinate system for a display 311, with 2.5 (63.5 mm) diagonal and aspect ratio 16:9, has coordinates (x,y,z)=(0, 32.6943, 44.2473) and it is rotated 20 around the x axis in negative (clockwise) direction with respect to the global coordinate system. The local coordinate system 312 for mirror 302 has its origin at (x,y,z)=(0, 23.7419, 34.8724) and is rotated 11.6039 around the x axis in positive (counterclockwise) direction respect to the global coordinate system. The local coordinate system 313 for mirror 303 has its origin at (x,y,z)=(0, 0, 45), and is rotated 33.4493 around the x axis in positive direction respect to the global coordinate system. Coordinates are given in mm. Coefficients of all surfaces' polynomials are listed in Table 1. The first four rows are x.sub.min, x.sub.max, y.sub.min and y.sub.max that describe rectangular area between x.sub.min and x.sub.max in x-direction, and y.sub.min and y.sub.max in y-direction where every Legendre polynomial P.sub.m(x,y) is orthogonal. The subsequent rows of Table 1 are the coefficients of 10.sup.th order Legendre polynomial P.sub.m(x,y) for each surface we have designed. Mirror-1 in Table 1 corresponds to the mirror 302 in FIG. 3. Mirror-2 in Table 1 corresponds to the mirror 303 in FIG. 3.

(27) TABLE-US-00002 TABLE 1 Parameter Mirror-1 Mirror-2 x.sub.min 18 27 x.sub.max 18 27 y.sub.min 16 16 y.sub.max 16 16 c.sub.0,0 1.42507 2.59056 c.sub.0,1 0.0451835 0.143174 c.sub.0,2 0.72009794 1.06388171 c.sub.0,3 0.01547969 0.01484046 c.sub.0,4 0.00206765 0.00346054 c.sub.0,5 0.00028542 0.0012447 c.sub.0,6 0.00064372 0.00035631 c.sub.0,7 0.00029827 0.00175909 c.sub.0,8 0.00076968 0.0005269 c.sub.0,9 0.00289666 0.00057807 c.sub.0,10 0.0009011 0.00041103 c.sub.2,0 2.17547149 4.19354056 c.sub.2,1 0.04495347 0.20853189 c.sub.2,2 0.0315604 0.07086542 c.sub.2,3 0.02546096 0.00760281 c.sub.2,4 8.14E05 0.00087688 c.sub.2,5 0.00194202 0.00111976 c.sub.2,6 0.00036391 0.00406823 c.sub.2,7 0.00263027 0.00127324 c.sub.2,8 0.00388143 0.00244588 c.sub.2,9 0 0 c.sub.2,10 0 0 c.sub.4,0 0.01848644 0.0570722 c.sub.4,1 0.02417861 0.04399492 c.sub.4,2 0.01349926 0.01382028 c.sub.4,3 0.00473022 0.0034179 c.sub.4,4 0.00384481 0.00095022 c.sub.4,5 0.00295132 0.00035308 c.sub.4,6 0.00010286 0.00039971 c.sub.4,7 0 0 c.sub.4,8 0 0 c.sub.4,9 0 0 c.sub.4,10 0 0 c.sub.6,0 0.00470142 0.01049631 c.sub.6,1 0.00667805 0.01978286 c.sub.6,2 0.03062507 0.01824176 c.sub.6,3 0.00213372 0.00212772 c.sub.6,4 0.00256455 0.00131309 c.sub.6,5 0 0 c.sub.6,6 0 0 c.sub.6,7 0 0 c.sub.6,8 0 0 c.sub.6,9 0 0 c.sub.6,10 0 0 c.sub.8,0 0.00682394 0.0082152 c.sub.8,1 0.01123838 0.00529108 c.sub.8,2 0.00589758 0.01197266 c.sub.8,3 0 0 c.sub.8,4 0 0 c.sub.8,5 0 0 c.sub.8,6 0 0 c.sub.8,7 0 0 c.sub.8,8 0 0 c.sub.8,9 0 0 c.sub.8,10 0 0 c.sub.10,0 0.00240235 0.0005887 c.sub.10,1 0 0 c.sub.10,2 0 0 c.sub.10,3 0 0 c.sub.10,4 0 0 c.sub.10,5 0 0 c.sub.10,6 0 0 c.sub.10,7 0 0 c.sub.10,8 0 0 c.sub.10,9 0 0 c.sub.10,10 0 0

(28) Reversed rays for different ipixels are traced from a notional eye pupil (displaced to the center 310 of the eye 314) towards the display 301, impinging on different opixels. This eye pupil allows us to simulate the performance of the ipixels when they are gazed while the eye is rotated, which are the ipixels whose performance should be the best. Raytrace simulation results indicate that the average rms spot diameter on the display 301 for the ipixels inside the gazed region of the virtual screen when they are gazed is about 70 microns. However, if the reversed rays from different ipixels are traced from the eye pupil at 310 when the eye is gazing frontwards, the image quality for these rays can be progressively relaxed when the peripheral angle increases, as permitted by the decreasing human eye resolution. As an example, the rms spot diameter of the reversed rays impinging on the display for an ipixel at a peripheral angle of 12 is about 160 microns, much higher than the value of the rms spot diameter for that ipixel when it is gazed, which is 56 microns. Since the focal length is about 52 mm, the forward ray-trace gives the result that the angular rms spot diameter of that ipixel seen through the pupil gazing frontwards is 10.3 arcmin. This is not perceived as blurred by the human eye, because it is smaller than 12 arcmin, which is what can be resolved by the human eye at 12 degrees peripheral angle (according to J. J. Kerr, Visual resolution in the periphery, Perception & Psychophysics, Vol. 9 (3), 1971)

(29) The field of view of the design shown in FIG. 3 can be increased by putting a lens 404 between the second mirror 403 and the eye 400, as shown in FIG. 4. The rays 405, and 406, emitted by the display 401 undergo 2 reflections: first on freeform mirror surface 402 then on see-through free-form mirror 403. After the 2 reflections, these rays are refracted twice at the lens 404 being directed towards the eye center 408. The field of view is increased due to additional collimation at the lens 404, thus for the same distance between the mirror surfaces and the eye, a bigger angular spread of rays can be directed from the digital display to the eye. With this configuration, we have a monocular vertical and horizontal FoV of 70 degrees and 100 degrees, respectively, with a 16:9 display with a diagonal of 2.1, 55 mm. As an alternative embodiment, the lens 404 can be designed with low optical power (closer to a plano parallel plate) with the purpose of correcting non-chromatic aberrations.

(30) The position of the lens can be changed along the ray trajectory. FIG. 5 shows an embodiment with a lens 504 placed between a digital display 501 and first mirrored surface 502. Rays 505 and 506, emitted by the display 501, undergo two refractions at the lens 504, and then 2 reflections: a first reflection on the free-form mirror surface 502 and a second reflection on the see-through freeform mirror 503. After the 4 deflections, these rays are directed towards the eye center 508. In this case the field of view is not increased compared with the design presented in FIG. 3. However, since the lens provides more degrees of freedom in the optical design (compared to the design from FIG. 3), the geometrical aberrations can be reduced.

(31) Another embodiment is presented in FIG. 6. The lens 604 is placed between two mirrored surfaces 602 and 603. The rays 605 and 606 coming from digital display 601 first reflect on the mirrored surface 602, then refract twice at the lens 604, and finally reflect at the see-through freeform mirror 603. After the 4 deflections, these rays are directed towards the eye center 608. Here, as in FIG. 5, only the aberrations are reduced while the field of view is the same as in FIG. 3.

(32) FIG. 7 shows a configuration consisting essentially of a tilted display 701 and a lens 714. All the optical surfaces are surfaces of the lens 714, as explained in more detail below. The user's nose 713 and eye 712 are shown for orientation. With this configuration, we have monocular vertical and horizontal fields of view of 70 degrees and 100 degrees, respectively, with a display diagonal of 2.1 (55 mm). This configuration contains 4 freeform optical surfaces, 2 refractive and 2 reflective. Rays 706, 707, 708 and 709 emitted by tilted display 701 suffer 4 deflections in total: first they refract on entrance surface 702, then reflect on surface 703, remaining inside lens 714, reflect on surface 704, remaining inside lens 714, refract on exit surface 705, and finally reach the eye 712. The rays 706 and 708 are directed towards the eye sphere center 711. The rays 706 and 709 define the vertical FoV of this mirror device as these are edge rays of the eye pupil center 710 when eye rests looking forward.

(33) Another design is presented in FIG. 8. This design has two solid lenses separated by a narrow air-gap 805. The rays emitted by a digital display 802 are first refracted by entrance surface 803 of the first lens, then some rays are reflected by mirrored surface 804 and other rays are reflected by TIR thanks to the air-gap (or gap filled with low-refractive-index material such as fluoropolymer FEP) 805. All the rays are then reflected by mirrored surface 806. Afterwards, the rays go across the low-index-gap without experiencing major changes in their directions, and finally are refracted by exit surface 807 towards the eye 801. The ray 810 impinges on the mirrored part of surface 804, while the rays 808 and 809 are reflected by TIR by the air-gap 805. Compared to the prior art in FIG. 1, the field of view is increased due to additional collimation at surface 807, thus for the same distance between the optical system and the eye, a bigger angular spread (defined by the angle between rays 810 and 809) of rays can be directed from the digital display to the eye. Surface 804, which looks concave in FIG. 8, is convex at substantially all of its points in the direction perpendicular to the drawing.

(34) Embodiments shown up to here are designed for virtual reality applications. From this point forward we show embodiments to be used for augmented reality (AR) applications allowing the user to see the surrounding reality through the optics.

(35) The design shown in FIG. 3 can be easily adapted for AR application if mirror 303 is a see-through mirror, in which the substrate of mirror 303 is transparent and the reflective coating is partially transmissive. This is shown in FIG. 9, where the surface 914 of mirror 903 nearer to the eye 911 is preferably the one that is mirror coated. If the substrate is thin (<1 mm), then the distance between the surfaces 914 and 913 can just be a constant value, and any distortion of the see-through live view will be minimal. If a fine correction of the see-though quality is desired, particularly for thicker substrates, the frontward surface 913 can be designed with the condition that rays 907, 908 coming from the environment exit surface 914 in same direction they had before impinging on surface 913. The embodiments presented in FIG. 5 and FIG. 6 can be adapted for AR in the same way as shown in FIG. 9. Rays 905, 906, coming from display 901 via first mirror 902 behave in the same way as the corresponding rays in FIG. 3. The nose 912 is shown for orientation. 910 is the center of the eye 911, and 909 is the pupil range.

(36) The design shown in FIG. 4 can be adapted for AR as shown in FIG. 10. The substrate of mirror 1003 is transparent and the reflective coating 1011, which is on the surface nearer to the eye 1000, is partially transmissive. The rays 1005 and 1006 coming from the display 1001 are reflected on mirror 1002 and reflective surface 1011, then refracted twice by lens 1004 being directed towards the eye center 1009. The substrate of mirror 1003 is a lens, and the frontward surface 1010 is designed with the condition that rays 1007, 1008 coming from the environment and refracted at surfaces 1010 and 1011 and also after two refractions at lens 1004 come to the eye in a direction parallel to the direction they have before impinging on 1010. In FIG. 10 the substrate of mirror 1003 is a biconcave lens, but that is not required in the general case.

(37) The embodiment shown in FIG. 11 is an adaptation of the embodiment in FIG. 7 to Augmented Reality applications. FIG. 11 shows a tilted display 1101, a lens 1114, and an additional piece of transparent material 1116. The user's nose 1115 and the eye 1112 are shown for orientation. The rays 1107 and 1108 emitted by the display 1101 suffer 4 deflections in total: first they refract at entrance surface 1102, then reflect on back surface 1103, reflect on semi-transparent surface 1104, which is at the boundary between the main lens 1114 and the additional piece 1116, refract at exit surface 1105, and finally reach the center 1113 of eye 1112. Thus a virtual image of a real image from the digital display 1101 is formed. On the other hand, rays 1110 and 1111 coming from the environment are first refracted at entrance surface 1109 of piece 1116, then refracted twice at the connection between piece 1116 and lens 1114. Since the two pieces of transparent material 1114, 1116 are very close to each other and the surfaces are parallel, this double refraction does not change the ray path considerably. The rays 1110, 1111 are then finally refracted at exit surface 1105, being redirected towards the eye. The entrance surface 1109 is designed so that after these refractions the rays 1110, 1111 preserve their directions, so the user is able to see through the optical system. As the result the user perceives both a virtual image of the digital display 1101 and the environment that surrounds the user.

(38) The embodiment shown in FIG. 12 is the adaptation of the embodiment in FIG. 8 to AR. Rays 1208 and 1209 coming from a tilted display 1202 have the same trajectories as in FIG. 8. The rays 1208, 1209 are refracted at entrance surface 1203, reflected on back surface 1204 (partly at a metallic reflector and partly by TIR at low-index gap 1206), reflected on semitransparent surface 1205, pass across low-index gap 1206, are then refracted at exit surface 1207, and finally reach the eye 1201. An additional piece 1214 of transparent material (lens) is placed with its eyeward surface on the back of semitransparent mirror 1205. The frontward entrance surface of additional piece 1214 is designed with the condition that rays 1210, 1211 coming from the environment reach the eye in direction parallel to the one they have before impinging on 1213. These rays are refracted at entrance surface 1213, pass through semitransparent mirror 1205, pass across thin air gap 1206, and are finally refracted at exit surface 1207.

(39) Although specific embodiments have been described, the preceding description of presently contemplated modes of practicing the invention is not to be taken in a limiting sense, but is made merely for the purpose of describing certain general principles of the invention. Variations are possible from the specific embodiments described. For example, the patents and applications cross-referenced above describe systems and methods that may advantageously be combined with the teachings of the present application. Although specific embodiments have been described, the skilled person will understand how features of different embodiments may be combined.

(40) The full scope of the invention should be determined with reference to the claims, and features of any two or more of the claims may be combined.