DISPLAY DEVICE WITH DARK RING ILLUMINATION OF LENSLET ARRAYS FOR VR AND AR
20230386373 · 2023-11-30
Inventors
Cpc classification
G09G3/002
PHYSICS
International classification
Abstract
A display device including a display panel to generate a real image, and an optical system. The optical system includes a plurality of lenslets, each having one cluster of object pixels, where the assignation of object pixels to clusters may change periodically in time intervals. The cluster emits light towards its corresponding lenslet and the emission is such that no light is sent to neighbor lenslets to avoid cross-talk between channels. Each channel projects a partial virtual image into the eye. The combination of all partial virtual images creates a virtual image. In a preferred embodiment, the partial images of neighbor channels are interlaced, which allows for a higher resolution. Additionally, each channel may be devoted to a single color, avoiding color filters and allowing for a higher efficiency.
Claims
1. A display device comprising: a panel, operable to generate a real image comprising a plurality of object pixels; and an optical system, comprising a plurality of lenslets; the panel and the optical system both arranged in a plurality of channels, each channel comprising a lenslet and a cluster of object pixels; wherein the assignation of object pixels to clusters may change in time intervals; wherein each object pixel of a cluster projects a corresponding ray pencil from the channel lenslet towards an imaginary sphere at an eye position; said sphere being an approximation of the eyeball sphere and being in a fixed location relative to a user's skull; wherein said ray pencils of each channel are configured to generate a partial virtual image from a real image of its corresponding cluster, and wherein the partial virtual images of the channels combine to form a virtual image to be visualized through a pupil of an eye during use; and wherein the average illuminance produced by each cluster on the output pupil of the lenslet associated to this cluster is at least 10 times greater than the average illuminance generated by this cluster on the output pupil of at least one of any other lenslet.
2. A display device of claim 1, wherein the average illuminance produced by at least one cluster on the output pupil of the lenslet associated to this cluster is at least 10 times greater than the average illuminance generated by this cluster on the output pupil of a set of lenslets surrounding the lenslet associated to that cluster.
3. A display device of claim 2, wherein said set of lenslets include the lenslets adjacent to the lenslet associated to the cluster.
4. A display device of claim 1, wherein at least two of the lenslets cannot be made to coincide by a simple translation rigid motion.
5. A display device of claim 1, wherein adjacent lenslets project light of different primary colors.
6. A display device of claim 5, wherein the different colors are produced by color filters.
7. A display device of claim 1, wherein at least one lenslet has a pancake optical configuration.
8. A display device of claim 1, wherein waists of said pencils of adjacent lenslets are interlaced at a waist surface.
9. A display device of claim 1, wherein foveal rays are a subset of rays emanating from the lenslets during use that reach the eye and whose straight prolongation is away from the imaginary sphere center a distance smaller than a value between 2 and 4 mm; and wherein the image quality of the virtual image formed by the foveal rays is greater than the image quality of the virtual image formed by non-foveal rays emanating from the lenslets during use.
10. A display device of claim 1, wherein each lenslet produces a ray pencil from each object pixel of its corresponding cluster, said pencils having corresponding waists laying close to a waist surface.
11. A display device of claim 1, wherein the ray pencils are activated to make the accommodation pixels lay close to a waist surface.
12. A display device of claim 1, further comprising a backlight to illuminate the panel.
13. A display device of claim 1, further comprising a backlight to illuminate the panel, wherein the backlight comprises a plurality of microlenslets and light emitters.
14. A display device of claim 1, wherein a set of o-pixels is turned off along the cluster's peripheries.
15. A display device of claim 1, wherein the panel is transmissive and it further comprises a backlight to illuminate the panel, wherein the backlight comprises a plurality of microlenslets and light emitters; wherein the state of a light emitter may change between active and inactive in time intervals; wherein at a given instant a fraction of the light emitters are inactive wherein the emitters are in an off state; wherein each channel further comprises a plurality of microlenslets and active light emitter pairs; wherein the object pixels of the cluster are grouped in microclusters, each one associated to a corresponding microlenslet of the channel; wherein the assignation of microlenslets and active light emitter to channels may change in time intervals; and wherein each active light emitter illuminates the channel's lenslet output pupil through its corresponding microlenslet and the lenslet, producing an image of the light emitter on the output pupil of the lenslet.
16. A display device of claim 15, wherein the images of a light emitter through two adjacent microlenslets is formed on the output pupil of two non-adjacent lenslets whose centers are separated by a distance at least twice the minimum diameter of the output pupil of the lenslets.
17. A display device of claim 15, wherein adjacent light emitters produce different primary colors.
18. A display device of claim 15, wherein the light emitters are light emitting diodes.
19. A display device of claim 15, wherein some active light emitters are dimmed according to the brightness of the image to be displayed on the microcluster associated to the emitter.
20. A display device of claim 19, wherein at least one microcluster contains an object pixel with a transmission greater than 90% of its maximum transmission.
21. A display device of claim 15, wherein the light emitters are pixels of a second transmissive panel back illuminated by a lightguide.
22. A display device of claim 21, wherein the lightguide is fed sequentially by different primary colors.
23. A display device of claim 15, wherein each light emitter further comprises a collimator.
24. A display device of claim 15, wherein the lenslets are configured in a locally-squared array.
25. A display device of claim 15, wherein the lenslets are configured in a locally-hexagonal array.
26. A display device of claim 15, wherein the fraction of active emitters is less than 50%.
27. A display device of claim 15, wherein the number of microlenslets belonging to a channel is greater than 20.
28. A display device of claim 1, wherein the optical system further comprises at least a conforming lens along the ray path from the panel to the eye.
29. A display device of claim 28, wherein the conforming lens has a pancake optical configuration.
30. A display device of claim 1, wherein there are more green color ray pencils than blue color ray pencils.
31. A display device of claim 1, wherein the intersection of each ray pencil with the eye pupil plane fully lays inside the eye pupil.
32. A display device of claim 1, wherein the intersection of each ray pencil with the eye pupil plane fully lays inside a static eye pupil position.
33. A display device of claim 1, further comprising a driver operative to drive and assign the object pixels to the channel clusters.
34. A display device of claim 1, further comprising a pupil tracker and a driver operative to dynamically drive and assign the object pixels to the channel clusters.
35. A display device of claim 15, further comprising a pupil tracker and a driver operative to dynamically drive and assign the object pixels and light emitters to the channel clusters.
36. A display device of claim 28, wherein said conforming lens has at least one surface with slope discontinuities.
37. A display device of claim 1, wherein the display device includes two or more panels per eye.
38. A display device of claim 1, further comprising a second display device, a mount to position the first and second display devices relative to one another such that their respective lenslets project the light towards two eyes of a human being, and a driver operative to cause the display devices to display objects such that the two virtual images from the two display devices combine to form a single image when viewed by a human observer.
39. A display device of claim 15, wherein the object pixels close to a border of the cluster are dark.
40. A display device of claim 33, wherein the display driver drives more power to the object pixels whose corresponding pencils enter partially the eye pupil to compensate for flux lost by vignetting.
41. A display device of claim 1, further comprising a mask to block the undesired light from the lenslet exit apertures.
Description
BRIEF DESCRIPTION OF DRAWINGS
[0064] The above and other aspects, features and advantages of the present invention will be apparent from the following more particular description thereof, presented in conjunction with the following drawings wherein:
[0065]
[0066]
[0067]
[0068]
[0069]
[0070]
[0071]
[0072]
[0073]
[0074]
[0075]
[0076]
DETAILED DESCRIPTION
[0077] A better understanding of the features and advantages of the present invention will be obtained by reference to the following detailed description of the invention and accompanying drawings, which set forth illustrative embodiments in which the principles of the invention are utilized.
[0078] The embodiments in the present invention consist on an display device comprising one or more displays per eye, operable to generate a real image comprising a plurality of object pixels (or opixels for short); and an optical system, comprising a plurality of lenslets, each one having associated at a given instant a cluster of object pixels. Each lenslet produces a ray pencil from an object pixel of its corresponding cluster. We shall call ray pencil (or just pencil) to the set of straight lines that contain segments coincident with ray trajectories illuminating the eye, such that these rays carry the same information at any instant. The same information means the same (or similar) luminance, color and any other variable that modulates the light and can be detected by the human eye. In general, the color of the rays of the pencil is constant with time while the luminance changes with time. This luminance and color are a property of the pencil. The pencil must intersect the pupil range to be viewable at some of the allowable positions of the pupil. When the light of a pencil is the only one entering the eye's pupil, the eye accommodates at a point near the location of the pencil's waist if it is being gazed and if the waist is far enough from the eye. The rays of a pencil are represented, in general, by a simply connected region of the phase space. The set of straight lines forming the pencil usually has a small angular dispersion and a small spatial dispersion at its waist. A straight line determined by a point of the central region of the pencil's phase space representation at the waist is usually chosen as representative of the pencil. This straight line is called central ray of the pencil. The waist of a pencil may be substantially smaller than 1 mm.sup.2 and its maximum angular divergence may be below ±10 mrad, a combination which may be close to the diffraction limit. The pencils intercept the eye sphere inside the pupil range in a well-designed system. The light of a single o-pixel lights up several pencils of different lenslets, in general, but only one or none of these pencils may reach the eye's retina, otherwise there is undesirable cross-talk between lenslets. The o-pixel to lenslet cluster assignation may be dynamic because it may depend on the eye pupil position.
[0079] The waist of a pencil is the minimum RMS region of a plane intersecting all the rays of the pencil. This flat region is in general normal to the pencil's central ray. In some embodiments the waists of some or all of the pencils can be grouped by its proximity to certain surfaces. These surfaces are called waist surfaces. Sometimes planes can approximate these surfaces. These planes are preferably normal to the frontward direction.
[0080]
[0081]
[0082] Different lenslets and their clusters have a similar behavior. As another example, light emitted from the points of cluster 210 cross the corresponding lens 209 and segment 211. Light crossing all lenslets or array 214 will enter the eye pupil 212 making it visible to the eye 213.
[0083]
[0084]
[0085]
[0086] In this configuration, if emitter 507 is on, microlenslet 505 will emit light towards lenslet 508 through LCD panel 501. However, if emitter 509 is off, lenslet 505 will not emit light towards lenslet 510 through LCD panel 501 and no crosstalk is generated. Accordingly, if emitter 511 is off, microlenslet 505 will not emit light towards lenslet 512 through LCD panel 501 and no crosstalk is generated.
[0087] Using this embodiment, it is then possible to turn on a given emitter in panel 502 such that a given microlenslet in panel 503 will illuminate a given lenslet in array 506. However, by turning off the emitters next to said emitter in panel 502, said microlenslet will not emit light to the neighbors of said lenslet, avoiding crosstalk.
[0088] A given lenslet 508 is associated with a cluster 513 because both belong to the same channel. One may then select the microlenslets under said cluster and turn on only the emitters such that said microlenslets illuminate lenslet 508.
[0089]
[0090]
[0091]
[0092]
[0093]
[0094]
[0095] Also shown are light emitters 1104 coupled to nonimaging collimators 1105. Said nonimaging collimators widen the apparent size of said light emitters as seen from the microlens array 1106. Example of such collimators may be Compound Parabolic Concentrators (CPCs) or aspheric lenses.
[0096] Without loss of generality consider next the case in which the interlacing factor k=2 will have 4 families of lenslets interlaced and with a square subpixel panel configuration. (which could be Red, Green, Blue and White if the white light emitter is more efficient than the Green, or alternatively Red, Green, Blue and Yellow is a wider color gamut is desired). Any skill in the art can be extrapolate this description to other interlacing factors, as k=2.sup.1/2, 3.sup.1/2, 7.sup.1/2, 3, etc. As described in PCT11, a squarish lenslet array configuration is the suitable one for this interlacing k=2 factor, so four families on lenslets, each one producing the full virtual image, but their pixels being projected to the eye interlaced. “Squarish” or stands for a general case in which the lenslets distribution is not perfectly allocated in a square grid, but is locally squared, either because the channel designs are done to produce variable cluster sizes, or because one or more conforming lenses are used in the system.
[0097] Consider the canonical simplification to illustrate the invention in which a square array of lenslets is used, whose pitch is d, located at a distance to the eye pupil ER (which stands for eye relief) and that when the eye rotates the pupil approximately shifts laterally, perpendicular to the z axis. To avoid the resolution of the virtual image be limited by diffraction, the size of the lenslet output pupils should be larger than, let say, 0.75 mm, so the lenslets pitch d, will not be smaller that that value. A minimum design value should be around d=0.8 mm, since for λ=589.3 nm, the Rayleigh criterion states that the resolvable pixel will be 0.61λ/d=0.045 mrad=0.0257 deg, that is, 1/0.0257≈40 ppd.
[0098] An underfilling strategy with k=2 requires that each minilens produces a virtual image with size in its diagonal cross section given by:
where α.sub.n+1 and α.sub.n are the extreme diagonal fields of channel n, and are the conjugates of the diagonal corners of the clusters. Assuming the waist plane is for simplicity of this explanation is located at infinity and a tangent law mapping in this example, we get that:
where F is the lenslets focal length and c is the cluster side. The clusters associated to each lenslet are preferably assigned so their size is proportional to the solid angle subtended by their output pupil from the center of the eye pupil. In this canonical example, with the clusters are squares with side:
[0099] Combining Equations 1, 2 and 3, we can solve for F and c to find:
F=ER [Equation 4]
c=2d=1.6 mm [Equation 5]
[0100] Notice this focal length is very long compared to the underfilling strategy disclosed in PCI 11, in which the illumination is not confined in the channels, since the equivalent canonical example in that case gets:
[0101] If the comparison between both systems is done with the same eye relief ER and the same circular FOV=90 deg, the present invention requires the use of a larger display due to the larger focal length of the lenslets. As an example, for ER=15 mm, the present invention (in this canonical example) has F=15 mm and requires a 3.34 inch diagonal panel, while PCT1.1 invention has F=5.72 mm and uses a 2.31 inch diagonal panel. If both panel have the same total pixel count of 4.5 k×4.5 k, the former opixel pitch will be 13.33 microns, while the latter has 9.21 microns. Since the resolution at the virtual image is given by:
where op is the panel opixel pitch, the present invention (in this canonical example) will provide a resolution of 39.3 ppd (matching the diffraction limit above), while PCT11 invention obtains only 21.7 ppd, that is, nearly a half.
[0102] If the comparison is done, instead of with the same ER, with the same panel with 3.34 inch diagonal and the same circular FOV=90 deg FOV, then the PCT11 invention with have an ER=21.7 mm, F=8.28 mm, but will provide a very similar resolution (22.3 ppd).
[0103] The union of all ray pencil prints UPP of the channels at the pupil plane is equal for all channels and is the same for the canonical example of this invention and the equivalent canonical examples of the PCT11 invention. It is given by a square of diagonal centered on the eye pupil:
UPP diagonal<3d√{square root over (2)} [Equation 8]
[0104] Therefore, for no vignetting to be produced by the eye pupil, the minimum size D of the user eye pupil should be bigger than that UPP. For d=0.8 mm, D≥3.4 mm (for smaller pupil, some vignetting occurs, that can be corrected by software).
[0105] Regarding the backlight design, it is formed by microlenslets imaging the plane where an array of light emitters is placed on the output pupil of the lenslets. This type of illumination is known as Köhler integration. Each light emitters position is configured together with the microlenslets and lenslets positions, so the following conditions are fulfilled: [0106] Condition 1. For any lenslet and any microlenslet that belongs to the lenslet channel, there is a light emitter that may illuminate the lenslet output pupil, the microlenslet producing an image of the light emitter on the output pupil of one lenslet and laying inside it, preferably filling it, and not invading the adjacent ones. [0107] Condition 2. The images of a light emitter through two adjacent microlenslets is formed on two non-adjacent lenslets whose centers are separated by a distance preferably at least three times the lenslet pitch.
[0108] At a given instant, a microlenslet is associated to a single channel and a single light emitter is associated to that microlenslet, and it will be addressed to illuminate its associated channel lenslet through the microlenslet (eventually this addressing may be such that no flux is produced if local dimming is being used). Due to Condition 2, the adjacent lenslets will not be illuminated by this microlenslet, typically two or more coronas of lenslets around the associated one. At another instant, the microlens may become associated to a different channel, and according to Condition 1, this can be done by activating a different light emitter to be associated to said microlenslet to illuminate the lenslet associated to that different channel.
[0109] In a preferred embodiment, the light emitters are all white and the panel is LCD type with color filters on each opixel. Such white emitters may be generated by a second, lower resolution LCD without color filters whose pixels become the emitters when they allow polarized light to cross through. This light is preferably generated by several R, G, and B LEDs that feed a lightguide whose purpose is to spread evenly the light through the LCD as in a conventional LCD display.
[0110] In a different embodiment the color filter may be allocated on the minilenses instead of being on the panel o-pixels. Then, all the pencils of each channel will have the same color. This means that the different families of minilenses will generate the whole image, but of different colors, and their superposion generates the final image.
[0111] In another preferred embodiment, the light emitters themselves emit a primary color, and again each family of channels of the interlacing is associated to one color. In this embodiment no color filters are used, resulting in a higher efficiency configuration. In the previous canonical example, a k=2 design in square configuration will have 4 families of lenslets interlaced, and each one can be associated to one of the four colors R, G, B, W. If this embodiment, one lenslet belongs to a given family, it will have a fixed colour too, and in this case a lenslet family can be named by its color. Assume the previous canonical example in a square grid where square clusters contain N×N microlenslets. The light emitters array will be in a four color square grid analogous to the one of the lenslets, so along a line there will be light emitters of two colors, for instance . . . RGRGRG . . . while along a contiguous line will be the other two colors . . . BWBWBW . . . . If a microlenslet belong, for instants, a red channel, its corresponding active light emitter will be red, and so will the active light emitter of an adjacent microlenslet of the same cluster be.
[0112] Let M−1 be the number of light emitters which are inactive between those two. According to Condition 2, M is preferable greater than 3. Therefore, if we denote with lower case the inactive light emitters and upper case the active ones, we could have M=4 which will mean that, for instance in red cluster we would have . . . RgrgRgrgR . . . and . . . bwbwbw . . . . In the adjacent green cluster the active pixels will be . . . rGrgrGrgr . . . and . . . bwbwbw . . . . In the transition between these red and green clusters we will preferably have . . . RgrgRgrGrgrG . . . M could also be another greater even number (for instance, 6), but the larger M the smaller size of light emitters is needed.
[0113] Following
[0114] Using that F=ER (Eq. 4), we can solve Eq. 9 to get:
For d=0.8 mm, ER=15 mm, N=10 and M=4, we get p.sub.μ=160 microns, p.sub.LE=42.1 microns, S=790 microns and f=750 microns.
[0115] The light from each active light emitter associated to a channel will reach the eye pupil through lenslets of other channels outside a circle on the eye pupil plane concentric with the eye whose diameter fulfills:
[0116] Using again that F=ER (Eq. 4) and that c=2d (Eq. 5), we obtain:
D.sub.max=(4M−3)d−2p.sub.μ [Equation 12]
That is, D.sub.max=10.8 mm, which is much larger than the maximum eye pupil diameter of the users that is expected for in operation (5-7 mm).
[0117] The calculation done so far is valid for any position of the eye pupil perpendicular to the z axis, provided that there is an eye pupil tracker and a panel driver that drives the panel opixels and light emitters to modify the clusters accordingly. If the center of the eye pupil shifts, the centers of the clusters should shift by the same amount on the panel plane in this canonical example, because the F=ER (Eq. 4). The shift of cluster centers is discretized to the values p.sub.μ, since they are composed by N×N microlenslets. This implies that the print diagonal of Eq. 8 should be enlarged in practice by 2√{square root over (2)}p.sub.μ (which is 0.45 mm in this example). Nevertheless, if the lenslet design includes dark corridors (set of o-pixels turned off along the cluster's peripheries), the shift of cluster centers is discretized to the values op, i.e., the o-pixel pitch, so the enlargement of the UPP diagonal is negligible.
[0118] Another preferred embodiment is the hexagonal configuration, which is suitable for panels with hexagonal pixel structures (called RGB-delta type) interlacing factors k=3.sup.1/2 and k=7.sup.1/2. This configuration produces a UPP on the pupil plane which is closer to a circle, better fitting the eye pupil shape. In the k=3.sup.1/2 case, the cluster contours are preferable arranged with a. 90 deg rotation with respect to the lenslet contours, to produce the tiling of the partial virtual images. The hexagonal contour of the clusters can be properly defined with the panel o-pixels.
[0119]
[0120] By combining the present invention with the inventions disclosed in PCT2 and PCT8 a further increase in resolution and/or field view can be achieved if the switching time of the panel allows a time multiplexing scheme.
[0121] Moreover, color sequential techniques can also be applied to the present invention. For example, the light emitters can be made of an LCD arrangement such that the LCD pixels become the emitters when they allow polarized light to cross through. This light is generated by several R, G, and B LEDs, which feed a lightguide whose purpose is to spread evenly the light through the LCD as in a conventional LCD display. The color sequential scheme is achieved by sequentially switching the R, G and B LEDs feeding the light guide. Note that in this case, at any instant all the openings and consequently all the channels and all the pencils are fed with the same light color. This is important for the interlacing design (see Definitions above) because now two pencils forming the same accommodation pixel have the same color and consequently the eye only perceives an added brightness for this accommodation pixel but not a color combination. In this situation, interlacing can still improve the resolution if the panel fill factor is low enough (ideally 25% or less) so there are opaque regions of areas similar or greater than the lit ones.
[0122] The use of the terms “a” and “an” and “the” and similar referents in the context of describing the invention (especially in the context of the following claims) are to be construed to cover both the singular and the plural, unless otherwise indicated herein or clearly contradicted by context. The terms “comprising,” “having,” “including,” and “containing” are to be construed as open-ended terms (i.e., meaning “including, but not limited to,”) unless otherwise noted. The term “connected” is to be construed as partly or wholly contained within, attached to, or joined together, even if there is something intervening.
[0123] The recitation of ranges of values herein are merely intended to serve as a shorthand method of referring individually to each separate value falling within the range, unless otherwise indicated herein, and each separate value is incorporated into the specification as if it were individually recited herein.
[0124] All methods described herein can be performed in any suitable order unless otherwise indicated herein or otherwise clearly contradicted by context. The use of any and all examples, or exemplary language (e.g., “such as”) provided herein, is intended merely to better illuminate embodiments of the invention and does not impose a limitation on the scope of the invention unless otherwise claimed. The various embodiments and elements can be interchanged or combined in any suitable manner as necessary.
[0125] The use of directions, such as forward, rearward, top and bottom, upper and lower are with reference to the embodiments shown in the drawings and, thus, should not be taken as restrictive. Reversing or flipping the embodiments in the drawings would, of course, result in consistent reversal or flipping of the terminology.
[0126] No language in the specification should be construed as indicating any non-claimed element as essential to the practice of the invention.
[0127] It will be apparent to those skilled in the art that various modifications and variations can be made to the present invention without departing from the spirit and scope of the invention. There is no intention to limit the invention to the specific form or forms disclosed, but on the contrary, the intention is to cover all modifications, alternative constructions, and equivalents falling within the spirit and scope of the invention, as defined in the appended claims. Thus, it is intended that the present invention cover the modifications and variations of this invention provided they come within the scope of the appended claims and their equivalent.