Method for computing holograms for holographic reconstruction of two-dimensional and/or three-dimensional scenes
11460809 · 2022-10-04
Assignee
Inventors
Cpc classification
G03H2001/0825
PHYSICS
G03H2210/441
PHYSICS
G03H1/26
PHYSICS
G03H1/0808
PHYSICS
International classification
G03H1/26
PHYSICS
Abstract
The invention relates to methods for computing holograms for holographic reconstruction of two-dimensional and/or three-dimensional scenes in a display apparatus, wherein a scene for reconstruction is broken down into object points and the object points are encoded as sub-holograms into at least one spatial light modulation device of the display apparatus. A reconstructed scene is viewed from a region of visibility. At least one virtual plane of the at least one spatial light modulation device is stipulated on the basis of a real plane of the spatial light modulation device. A computation of sub-holograms is performed in the at least one virtual plane of the at least one spatial light modulation device.
Claims
1. A method for calculating holograms for the holographic reconstruction of two-dimensional and/or three-dimensional scenes in a display apparatus, comprising decomposing a scene to be reconstructed into object points and encoding the object points as sub-holograms into at least one spatial light modulator device of the display apparatus, where a reconstructed scene is observed from a visibility region; determining at least one virtual plane of the at least one spatial light modulator device based on a real or physical plane of the spatial light modulator device, carrying out a calculation of sub-holograms in the at least one virtual plane of the at least one spatial light modulator device, and depending on the position of all object points of the scene to be reconstructed, the distance of the at least one virtual plane of the spatial light modulator device to the visibility region is selected such that the average size, preferably averaged over all object points, in form of a number of modulation elements of the sub-holograms to be calculated for the scene to be reconstructed takes a minimum value.
2. The method according to claim 1, wherein the object points of the scene to be reconstructed are calculated as sub-holograms in the at least one virtual plane of the at least one spatial light modulator device, the calculated sub-holograms are transformed from the at least one virtual plane of the at least one spatial light modulator device into the visibility region by means of an integral transformation and are added up there, and the summed sub-holograms are transformed from the visibility region into the physical plane of the at least one spatial light modulator device by means of a further integral transformation and are written in as a hologram.
3. The method according to claim 1, wherein the object points of the scene to be reconstructed are calculated as sub-holograms in the at least one virtual plane of the at least one spatial light modulator device and the sub-holograms are added up in the at least one virtual plane, the summed sub-holograms of the at least one virtual plane of the at least one spatial light modulator device are transformed into the visibility region by means of an integral transformation and are transformed from the visibility region into the physical plane of the at least one spatial light modulator device by means of a further integral transformation and are written in as a hologram.
4. The method according to claim 3, wherein when at least two virtual planes of the at least one spatial light modulator device are determined, one of these virtual planes is each assigned to the object points of the scene to be reconstructed and the object points are calculated as sub-holograms in the virtual plane assigned to them and the sub-holograms of the object points assigned to the virtual plane are added up in each virtual plane, the summed sub-holograms are transformed from the at least two virtual planes into the visibility region by means of an integral transformation, the transforms of the at least two virtual planes are added up in the visibility region, and the total result of the summed transforms is transformed from the visibility region into the physical plane of the at least one spatial light modulator device by means of a further integral transformation and is written in as a hologram.
5. The method according to claim 3, wherein when at least two virtual planes of the at least one spatial light modulator device are determined, one of these virtual planes is each assigned to the object points of the scene to be reconstructed and the object points are calculated as sub-holograms in the virtual plane assigned to them and the sub-holograms of the object points assigned to the virtual plane are added up in each virtual plane, the summed sub-holograms are transformed from the at least two virtual planes into the visibility region by means of an integral transformation, for each of the at least two virtual planes the corresponding transform is transformed from the visibility region into the real or physical plane of the at least one spatial light modulator device by means of a further integral transformation, the transforms of the at least two virtual planes are added up in the physical plane of the at least one spatial light modulator device, and the total result of the summed transforms is written in as a hologram.
6. The method according to claim 1, wherein the transformation of the sub-holograms or the summed sub-holograms from a virtual plane into the visibility region or the further integral transformation from the visibility region into the physical plane of the spatial light modulator device is carried out by means of a one-dimensional integral transformation in the case of a single-parallax encoding or by means of a two-dimensional integral transformation in the case of a single-parallax encoding or a full-parallax encoding.
7. The method according to claim 1, wherein the position of the at least one virtual plane of the spatial light modulator device is selected within the depth range of the scene to be reconstructed, where the depth range of the scene to be reconstructed is delimited by the object point having the smallest distance to the visibility region and the object point having the greatest distance to the visibility region.
8. The method according to claim 1, wherein with respect to the depth range of the scene to be reconstructed, the position of the at least one virtual plane of the spatial light modulator device is selected such that the visible resolution during the reconstruction of the scene does not fall below a given value.
9. The method according to claim 1, wherein the scene to be reconstructed is decomposed into at least two depth range sections, where one virtual plane of the spatial light modulator device is each assigned to the at least two depth range sections and the sub-holograms in the virtual plane assigned to the depth range section are calculated for all object points located within a depth range section.
10. The method according to claim 9, wherein one virtual plane of the virtual planes of the spatial light modulator device, which are assigned to the at least two depth range sections, coincides with the physical plane of the spatial light modulator device.
11. The method according to claim 10, wherein the adding up of the sub-holograms calculated for the physical plane of the spatial light modulator device is carried out either in the physical plane of the spatial light modulator device or in the visibility region.
12. The method according to claim 1, wherein the individual object points of the scene to be reconstructed are encoded in form of sub-holograms on the physical plane of the spatial light modulator device as a lens element having different focal lengths in the horizontal direction and the vertical direction.
13. The method according to claim 1, wherein the calculation of the sub-holograms in the at least one virtual plane of the spatial light modulator device is carried out by means of a two-dimensional full-parallax encoding or by means of a one-dimensional single-parallax encoding.
14. The method according to claim 1, wherein depending on the depth range of the scene to be reconstructed, the number of the virtual planes of the spatial light modulator device as well as the distance of said virtual planes of the spatial light modulator device to the visibility region are selected such that the size in form of a number of modulation elements of the sub-holograms to be calculated for the scene to be reconstructed does not exceed a given value.
15. The method according to claim 1, wherein the number of the virtual planes of the spatial light modulator device is limited to a given value.
16. The method according to claim 1, wherein the position of the at least one virtual plane of the spatial light modulator device is adjusted to the new depth range when there is a change in time of the depth range of the scene to be reconstructed.
17. The method according to claim 1, wherein when tracking the visibility region in the axial direction and/or lateral direction according to a detected position of an observer of the reconstructed scene, the number and position of the at least one virtual plane of the spatial light modulator device are adjusted to the new position of the visibility region.
18. The method according to claim 1, wherein the deviation of the calculated hologram of the scene to be reconstructed by means of the at least one virtual plane of the spatial light modulator device from a hologram calculated directly in the physical plane of the spatial light modulator device for the same scene does not exceed a defined criterion.
19. The method according to claim 1, wherein when using a display apparatus for two-dimensional and/or three-dimensional scenes in which an image plane of the at least one spatial light modulator device is generated, which acts as a physical plane of the spatial light modulator device, at least one virtual plane of the at least one spatial light modulator device is determined depending on the content of a scene to be reconstructed, and the image plane of the spatial light modulator device is shifted such that said image plane coincides with the virtual plane of the spatial light modulator device, the object points of the scene to be reconstructed are calculated as sub-holograms and added up and are written in as a hologram.
20. A display apparatus, in particular a holographic display apparatus for the representation of two-dimensional and/or three-dimensional scenes, comprising a spatial light modulator device, where the light modulator device is suitable for carrying out the method according to claim 1.
21. The display apparatus according to claim 20, wherein the display apparatus comprises an image plane of the spatial light modulator device and an imaging system, where the imaging system is designed such that the position of the image plane is variable.
22. The display apparatus according to claim 21, wherein the imaging system comprises at least one element having a variable focal length.
23. The display apparatus according to claim 20, wherein the display apparatus is designed as a head-mounted display.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
(1) The Figures Show:
(2)
(3)
(4)
(5)
(6)
(7)
(8)
(9)
DETAILED DESCRIPTION OF THE INVENTION
(10) It should be mentioned briefly that the same elements/components have the same reference numerals.
(11) In
(12) In this example, the display apparatus comprises a light source LQ, a field lens FL and a spatial light modulator device 1. The light source LQ and the field lens FL are arranged in such a way that, where no hologram is written in into the spatial light modulator device 1, the light emitting by the light source LQ is focused by means of the field lens FL into an observer plane 2 at a distance z.sub.lm from the spatial light modulator device 1. If a scene to be reconstructed (not shown here) is decomposed into object points in a suitable way and the object points are encoded into the spatial light modulator device 1 of the display apparatus as sub-holograms, a visibility region 3 or a virtual viewing window is generated at said distance z.sub.lm from the spatial light modulator device 1.
(13) In the display apparatus shown, the distance of the field lens FL of the spatial light modulator device 1 is small, that is, significantly smaller than the distance z.sub.lm of the spatial light modulator device 1 to the visibility region 3.
(14)
(15) The display apparatus shown in
(16) Compared to
(17) In
(18) Using a field lens FL, which is arranged very close to the real image rB of the spatial light modulator device 1, furthermore, light of a light source LQ is focused into an observer plane 2.
(19) Then, the decomposing of a scene to be reconstructed into object points and the encoding of the object points into the spatial light modulator device 1 of the display apparatus as sub-holograms can be carried out in such a way as if there were indeed a materially available spatial light modulator device at the location of the real image rB of the spatial light modulator device 1.
(20)
(21) In this exemplary embodiment of a display apparatus, a spatial light modulator device 1 is located at a position such that the distance between the spatial light modulator device 1 and the field lens FL is similar in size to the distance between the field lens FL and the observer plane 2.
(22) The distance between the field lens FL and the observer plane 2 corresponds to, for example, the focal length of the field lens FL. The distance between the spatial light modulator device 1 and the field lens FL is approximately as large as but smaller than the focal length of the field lens.
(23) In this case, a usually enlarged virtual image vB of the spatial light modulator device 1 is generated.
(24) In turn, the decomposing of a scene to be reconstructed into object points and the encoding of the object points into the spatial light modulator device 1 of the display apparatus as sub-holograms can then be carried out as if there were indeed a materially available spatial light modulator device 1 at the location of the virtual image vB of the spatial light modulator device 1.
(25) Such a display apparatus can, for example, be designed as a head-mounted display, where a field lens having a small focal length and a spatial light modulator device small in size are located near the eyes of an observer—for example, used as glasses or as a hocular.
(26) The term ‘physical plane of the spatial light modulator device’ as used in the description of the invention should be a generic one comprising also image planes of a spatial light modulator device in a display apparatus as illustrated, for example, by
(27)
(28) Thus, further devices or elements provided in the display apparatus will not be described herein, unless pointed out accordingly.
(29) The spatial light modulator device 1, also referred to as SLM (spatial light modulator), according to
(30) If, however, a comlex value of a hologram is not written in in a single modulation element (pixel) of an amplitude+phase light modulator device but is written in instead using several modulation elements (pixels) of an only phase- or only amplitude-modulating light modulator device, the aforementioned terms ‘pixel pitch’ and ‘number of modulation elements (pixels)’ are to be understood so that a modulation element and a pixel pitch each refers to several modulation elements (pixels) of such a light modulator device into which, in total, one complex value is written in.
(31) The spatial light modulator device 1 can be designed as an amplitude+phase light modulator device in form of a sandwich array. The amplitude light modulator device has, for example, a smaller distance (typically <2 mm) from the phase light modulator device.
(32) In this case, the term ‘physical plane of the spatial light modulator device’ is used preferably such that a central plane is chosen in the sandwich array.
(33) As shown in
(34) The spatial light modulator device 1 shown in
(35) Here, in absolute terms, small sub-holograms S1 and S2 are generated on the spatial light modulator device according to
(36)
(37) For calculating sub-holograms, the method according to the invention introduces a virtual plane of the spatial light modulator device, which is located at a physical location, in order to minimize the computational effort. In this way, it is possible to obtain fast and exact computational results, irrespective of the size and distance of the spatial light modulator device from the observer plane.
(38) Irrespective of the actual configuration of the holographic display apparatus, in particular of the position of the spatial light modulator device and of the optical system, first, a sub-hologram is calculated in such a way as if the virtual plane of the spatial light modulator device according to
(39) Regarding
(40) For the virtual plane of the spatial light modulator device 1, 10, a position is chosen advantageously with regard to the scene 4 to be reconstructed, which is within the scene 4 to be reconstructed, so that, watched by an observer in the observer plane 2, one part of the object points P1 to P.sub.n, seen in the direction of propagation of light emitted by an illumination device of the display apparatus, is in front of and an other part of the object points P1 to P.sub.n is behind the virtual plane of the spatial light modulator device 1, 10, where the object points that are closest to the observer plane 2 should be located not more than half the virtual observer's distance in front of the virtual plane of the spatial light modulator device 1, 10.
(41) However, the method is not limited to a certain definition of the position of the virtual plane of the spatial light modulator device.
(42) The position of the virtual plane of the spatial light modulator device may move virtually, dynamically with the content of the scene to be displayed, to be able to calculate, for example, scenes with a variable depth range in an optimal way.
(43) The virtual size of the modulation elements 7, 70 of the spatial light modulator device 1, 10 is calculated using the intercept theorem from the ratio of the observer's distances of the virtual plane and the physical plane of the spatial light modulator device multiplied by the pixel pitch of the spatial light modulator device.
(44) For the virtual plane of the spatial light modulator device 1, 10, a sub-hologram or a hologram is calculated in such a way as would be required in the prior art, for example, according to WO 2008/025839 A1, if a spatial light modulator device were provided in this plane. This can be carried out such that for each object point P1 to P.sub.n a sub-hologram is calculated in form of a lens function, and the individual sub-holograms will then be added up coherently.
(45) Subsequently, a one-time computational (two-dimensional) integral transformation, for example, a Fresnel transformation, of the entire hologram is carried out, which corresponds to the optical propagation of light of the virtual plane of the spatial light modulator device to the visibility region 3. Then, a further computational (two-dimensional) integral transformation is carried out, here for example, also a Fresnel transformation, which corresponds to the propagation of light from the visibility region 3 back to the physical plane of the spatial light modulator device 1, 10. An entire hologram is thus obtained, corresponding at least approximately to the hologram that, according to the prior art, is obtained by directly calculating the hologram in the physical plane of the spatial light modulator device 1, 10. The resulting hologram may then be written in or encoded into the spatial light modulator device 1, 10.
(46) Compared to the conventional calculation, the method according to
(47) The following embodiments according to
(48)
(49) The one-dimensional sub-hologram S1 calculated in the virtual plane 100′ of the spatial light modulator device 100 is then transformed by means of an integral transformation, for example, by means of a Fresnel transformation or a Fourier transformation, into the visibility region 3 and is then transformed from there into the physical plane 100 of the spatial light modulator device 100, where it acts as a sub-hologram extended into two dimensions. However, the focal lengths of the lens function of the sub-hologram are not identical in both directions on the physical plane 100 of the spatial light modulator device 100 (orthogonal to each other). In one direction, for example, the encoding direction of the sub-hologram on the virtual plane 100′ of the spatial light modulator device 100, the focal point of the lens function is located in the plane of the object point; in the direction perpendicular to it, the focal point of the lens function is located on the virtual plane 100′ of the spatial light modulator device 100.
(50) This variant of the method can be used in a particularly advantageous way for a spatial light modulator device that is located at a smaller distance to the observer plane 2 and in which the virtual plane of the spatial light modulator device has a greater distance to the observer plane 2 than the physical plane of the spatial light modulator device according to
(51)
(52) Thus,
(53) According to
(54) In a single-parallax calculation, the segmentation into virtual planes of the spatial light modulator device and the subdivision of the scene 4 to be reconstructed into several depth range sections TA1, TA2 and TA3 are carried out such that an observer can see the reconstructed two-dimensional and/or three-dimensional scene with no or with only a minor loss in resolution.
(55) In a full-parallax calculation, the segmentation into virtual planes of the spatial light modulator device and the subdivision of the depth range of the scene 4 to be reconstructed into sections are carried out such that a maximum size of the sub-holograms is not exceeded, as explained in detail before the description of the figures.
(56) Subsequently, the calculated sub-holograms S1, S2 and S3 or S1 to S.sub.n of the depth range sections TA1, TA2 and TA3 are transformed into the observer plane 2 or into the visibility region 3, and are added up there. Then, the summed sub-holograms or the entire hologram are transformed into the physical plane of the spatial light modulator device 100.
(57) As shown in
(58) In a graphic representation,
(59) An object point of a scene to be reconstructed, which is 2 m away from an observer, generates, for example, a sub-hologram with a size of approximately 200×1 modulation elements (pixels) in a single-parallax encoding or a sub-hologram with a size of approximately 200×200 modulation elements in a full-parallax encoding on a spatial light modulator device (SLM), which is 0.7 m away from the visibility region in the observer plane. On a spatial light modulator device (SLM), which, however, is 2.2 m away from the visibility region in the observer plane, the extent of the sub-hologram of the same object point is only few modulation elements (pixels) in size, for example <10×1 modulation elements in a single-parallax encoding or <10×10 modulation elements in a full-parallax encoding. The contrary applies when object points are located at a closer distance to an observer. On a spatial light modulator device at a close distance, said object points are small in their extent or size, but on a spatial light modulator device at a far distance, they are larger in their extent.
(60) The dashed lines in the diagram according to
(61)
(62) In
(63) Furthermore, light beams emanating from a modulator element (pixel) of the spatial light modulator device 110 and directed toward the center and the edges of a viewing window 3 are illustrated. The light beams are deflected by the field lens FL in such a way as if they emanated from or were emitted by an enlarged image 110B of the spatial light modulator device 110 at a greater distance. An imaging system or a focusing system AS which is, advantageously, configured to have a variable focal length herein, is provided between the spatial light modulator device 110 and the field lens FL. The focal length of the imaging system AS can thus be varied in a controllable way. Of course, the imaging system AS can also be arranged at another suitable position in the display apparatus.
(64) If the field lens FL is a lens system comprising several lenses, the imaging system AS having a variable focal length can also be a part of the field lens FL or the field lens FL itself can be designed as an imaging system or a focusing system AS having a variable focal length. The imaging system or the focusing system AS having a variable focal length can, for example, also be designed as a system of several lenses in which the distance of the individual lenses to each other can be varied, whereby the focal length of the entire system is changed.
(65) Furthermore,
(66) For calculating the holograms for an HMD or hocular, a virtual plane 110′ of the spatial light modulator device 110 is determined based on the content of the scene 4 to be reconstructed and is set in the display apparatus at a position in such a way that a size suitable for the calculation of the sub-holograms results or is achieved. For example, as described above, the virtual plane 110′ can be set such that either the maximum size or the average size of the sub-holograms is minimized. According to
(67) In
(68) It is also possible to generate and determine not only one virtual plane of the spatial light modulator device but a multitude thereof, i.e., at least two virtual planes. The image of the spatial light modulator device can then be shifted into a suitable virtual plane, where the calculating and encoding of the sub-holograms into the spatial light modulator device can be carried out according to one of the many methods according to the invention described above. In the case of at least two virtual planes, it is furthermore possible to shift the image of the spatial light modulator device sequentially first into one virtual plane and then into another virtual plane.
(69) Particularly for dynamic hologram content, in which the depth range of the preferably three-dimensional scene to be reconstructed changes, the shifting of the image 110B of the spatial light modulator device 110 into the respective virtual plane 110′ of the spatial light modulator device 110 may be adjusted to the respective depth range of the scene of the frame to be displayed.
(70) By subdividing the scene to be reconstructed and displayed into depth range sections and using virtual planes of the spatial light modulator device, the computational effort required for calculating the sub-holograms is significantly reduced and thus there is a significant time improvement when displaying reconstructed scenes.
(71) In contrast, there is an additional effort involved in transforming the sub-hologram data from several virtual planes of the spatial light modulator device into the visibility region, adding it up there and subsequently transforming it into the physical plane of the spatial light modulator device. The total computational effort is reduced by the use of virtual planes of the spatial light modulator device and by positioning said virtual planes in a suitable way.
(72) In this way, the advantages of the calculation methods with regard to the integral transformations and the direct calculation of sub-holograms may be combined.
(73) The invention is not limited to the exemplary embodiments disclosed herein, and it can be used to minimize the computational effort for computer-generated holograms.
(74) In conclusion, it shall explicitly be pointed out that the exemplary embodiments described above serve only to describe the teaching claimed, but do not limit it to the exemplary embodiments.