HOLOGRAPHIC DISPLAY AND METHOD FOR GENERATING A HOLOGRAM

Abstract

A holographic display includes a light source configured to emit incoherent light, a spatial light modulator (SLM) disposed at a Fourier plane of the light source and configured to modulate the incoherent light and emit the modulated incoherent light as diffused light, and at least one lens disposed downstream of the spatial light modulator and configured to focus the modulated incoherent light, and the spatial light modulator receives an RGB-D image signal and converts the incoherent light array incident from the light source into a first light, which is a coherently-reconstructed incoherent sum (CRIS) for implementing a 3D hologram.

Claims

1. A holographic display comprising: a light source configured to emit incoherent light; a spatial light modulator (SLM) disposed at a Fourier plane of the light source and configured to modulate the incoherent light and emit the modulated incoherent light as diffused light; and at least one lens disposed downstream of the spatial light modulator and configured to focus the modulated incoherent light, wherein the spatial light modulator receives an RGB-D image signal and converts the incoherent light array incident from the light source into a first light, which is a coherently-reconstructed incoherent sum (CRIS) for implementing a 3D hologram.

2. The holographic display of claim 1, wherein the lens is arranged to direct the first light, for generating a 3D hologram, into a pupil located at the Fourier plane of the spatial light modulator.

3. The holographic display of claim 1, wherein the first light is expressed as: $I_{\mathrm{CRIS}}\left(\stackrel{.fwdarw.}{r},z\right)=\frac{1}{}\underset{{\stackrel{\text{?}}{k}}_g}{.Math.}{.Math.{}_A\stackrel{\_}{U}\left(\stackrel{.fwdarw.}{k}+{\stackrel{.fwdarw.}{k}}_g\right)e^{\mathrm{iz}\sqrt{{\left(2/\right)}^2-{.Math.\stackrel{.fwdarw.}{k}.Math.}^2}}{e}^{i\stackrel{.fwdarw.}{k}.Math.\stackrel{.fwdarw.}{r}}d\stackrel{.fwdarw.}{k}.Math.}^2$ $\text{?}\text{indicates text missing or illegible when filed}$ where ICRIS({right arrow over (r)},z) is the intensity of the first light, z is the distance along the longitudinal direction, N is a normalization constant, {right arrow over (r)} is the distance vector from the center of the spatial light modulator, U({right arrow over (r)}) is the Fourier transform function of the modulation signal of the spatial light modulator, {right arrow over (k)} is the radial wavevector before modulation, {right arrow over (k)}.sub.g=(k.sub.x, k.sub.y) is the modulated radial wavevector, and A is the area of the pupil of the Fourier plane (pupil plane), wherein the first light is generated by deriving a Fourier transform function U({right arrow over (k)}) that satisfies $I_{\mathrm{CRIS}}\left(\stackrel{.fwdarw.}{r},z\right)=I_{\mathrm{target}}$ where Itarget is the intensity of the second light, which is the incoherent sum of the target light.

4. The holographic display of claim 1, wherein the first light exhibits translational invariance, wherein the intensity of the first light remains constant regardless of the positional shift of the pupil.

5. The holographic display of claim 4, wherein the translational invariance is expressed by introducing a positional vector of the pupil with respect to the center of the eye box (px, py, pz), wherein $I_{\mathrm{CRIS}}\left(\stackrel{.fwdarw.}{r},z,p_x,p_y,p_z\right)=I_{\mathrm{CRIS}}\left(\stackrel{.fwdarw.}{r},z,0,0,0\right)$ where I.sub.CRIS({right arrow over (r)}, z) is the intensity of the first light, and z is the distance along the longitudinal direction, and wherein the first light is implemented by deriving a Fourier transform function U({right arrow over (k)}) that satisfies $I_{\mathrm{CRIS}}\left(\stackrel{.fwdarw.}{r},z\right)=I_{\mathrm{target}}$ where Itarget is the intensity of the second light, which is the incoherent sum of the target light.

6. A method for generating a hologram using the holographic display of claim 1, comprising: a first step of performing an optical propagation simulation to propagate, over a predetermined distance, first light that is a coherently-reconstructed incoherent sum (CRIS) generated by the spatial light modulator upon receiving light from the light source, wherein an arbitrary RGB-D image signal for implementing the hologram is input to the spatial light modulator; a second step of simulating the angles of light incident within the pupil and calculating the average intensity of the first light incident at each of the simulated angles; and a third step of optimizing the arbitrary pattern by comparing the average intensity of the first light with the intensity of second light as the target light.

7. The method for generating a hologram of claim 6, wherein the first light is expressed as: $I_{\mathrm{CRIS}}\left(\stackrel{.fwdarw.}{r},z\right)=\frac{1}{}\underset{{\stackrel{\text{?}}{k}}_g}{.Math.}{.Math.{}_A\stackrel{\_}{U}\left(\stackrel{.fwdarw.}{k}+{\stackrel{.fwdarw.}{k}}_g\right)e^{\mathrm{iz}\sqrt{{\left(2/\right)}^2-{.Math.\stackrel{.fwdarw.}{k}.Math.}^2}}{e}^{i\stackrel{.fwdarw.}{k}.Math.\stackrel{.fwdarw.}{r}}d\stackrel{.fwdarw.}{k}.Math.}^2$ $\text{?}\text{indicates text missing or illegible when filed}$ where I.sub.CRIS({right arrow over (r)}, z) is the intensity of the first light, zzz is the distance along the longitudinal direction, NNN is a normalization constant, {right arrow over (r)} is the distance vector from the center of the spatial light modulator, U({right arrow over (k)}) is the Fourier transform function of the modulation signal of the spatial light modulator, {right arrow over (k)}.sub.g is the radial wavevector before modulation, {right arrow over (k)}.sub.g=(k.sub.x, k.sub.y) is the modulated radial wavevector, and A is the area of the pupil in the Fourier plane, and wherein the first light is generated by deriving a Fourier transform function U({right arrow over (k)}) that satisfies $\mathrm{ICRIS}\left(\stackrel{.fwdarw.}{r},z\right)=\mathrm{Itarget}$ where Itarget is the intensity of the second light, which is the incoherent sum of the target light.

8. The method for generating a hologram of claim 6, wherein the first light exhibits translational invariance, whereby the intensity of the first light remains constant regardless of the positional shift of the pupil.

9. The method for generating a hologram of claim 8, wherein the translational invariance of the first light is expressed by introducing a positional vector of the pupil with respect to the center of the eye box (px,py,pz), wherein $I_{\mathrm{CRIS}}\left(\stackrel{.fwdarw.}{r},z,p_x,p_y,p_z\right)=I_{\mathrm{CRIS}}\left(\stackrel{.fwdarw.}{r},z,0,0,0\right)$ where I.sub.CRIS({right arrow over (r)}, z) is the intensity of the first light, and z is the distance along the longitudinal direction, and wherein the first light is implemented by deriving a Fourier transform function U({right arrow over (k)}) that satisfies $I_{\mathrm{CRIS}}\left(\stackrel{.fwdarw.}{r},z\right)=I_{\mathrm{target}}$ where Itarget is the intensity of the second light, which is the incoherent sum of the target light.

10. The method for generating a hologram of claim 6, wherein, in the third step, the first light formed by the spatial light modulator based on the optimized arbitrary pattern is incident on a pupil positioned at the Fourier plane of the spatial light modulator via a lens.

Description

BRIEF DESCRIPTION OF THE FIGURES

[0024] Embodiments will be described in more detail with regard to the figures, wherein like reference numerals refer to like parts throughout the various figures unless otherwise specified, and wherein:

[0025] FIGS. 1A to 1E are diagrams illustrating examples of CRIS implementations.

[0026] FIGS. 2A to 2F are diagrams illustrating the numerical reconstruction of CRIS, and FIG. 3 depicts various reconstructed images of CRIS.

[0027] FIGS. 3A to 3F are a diagram depicting various reconstructed images of CRIS.

[0028] FIGS. 4A to 4E are diagrams illustrating the optical reconstruction of CRIS. FIGS. 5A to 5F illustrate 3D reconstructions at various different positions of pupil.

[0029] FIGS. 6A to 6D illustrate the schematic arrangement of the spatial light modulator, lens, and pupil for translational invariance of the pupil and effective eye box calculation.

[0030] FIGS. 7A to 7C are diagrams illustrating the dependence on pupil area and computation time.

[0031] FIGS. 8A to 8C illustrate the eye box calculation process of holograms in prior art.

[0032] FIGS. 9A and 9B are graphs showing relative PSNR as a function of NoIA (Number of Incident Angles).

[0033] FIG. 10 is a graph illustrating PSNR according to wavelength differences. FIGS. 11A and 11B are diagrams showing the numerically reconstructed intensity of the first light, including additional losses.

[0034] FIG. 12 is a flowchart of a hologram generation method according to another embodiment of the present invention.

DETAILED DESCRIPTION OF THE DISCLOSURE

[0035] The embodiments of the present invention disclosed in this specification include specific structural or functional descriptions that are illustrated solely for the purpose of explaining the embodiments of the invention. The embodiments of the invention may be implemented in various forms and are not limited to the examples described in this specification.

[0036] The embodiments of the present invention are subject to various modifications and may take multiple forms, and thus the embodiments are illustrated in the drawings and described in detail in the specification. However, these are not intended to limit the embodiments of the invention to specific forms of disclosure, but to include modifications, equivalents, or substitutions that fall within the spirit and scope of the invention.

[0037] The terms first, second, and the like may be used to describe various components but should not be understood to limit the components by these terms. These terms are only used to distinguish one component from another. For example, within the scope of the invention, a first component may be referred to as a second component and vice versa.

[0038] When a component is described as being connected to or coupled to another component, it may be directly connected or coupled, or there may be intermediate components. In contrast, if a component is described as being directly connected to or directly coupled to another component, it should be understood that no intermediate components are present. Similarly, terms describing the relationships between components, such as between, immediately between, or directly adjacent to, should be interpreted in the same manner.

[0039] The terms used in this specification are intended to describe particular embodiments and are not intended to limit the invention. Singular expressions include plural forms unless the context clearly indicates otherwise. Terms such as comprises or has specify the presence of stated features, numbers, steps, operations, components, parts, or their combinations, and do not exclude the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts, or their combinations.

[0040] Unless otherwise defined, all terms used herein, including technical and scientific terms, have the same meaning as commonly understood by those skilled in the art. Terms defined in dictionaries should be interpreted as having meanings consistent with the context of the relevant technology and should not be construed in an overly idealized or formal sense unless expressly defined in this specification.

[0041] The holographic display and hologram generation method of the present invention scatter light at various angles using an incoherent light source. Conventional displays faced limitations in scattering light at various angles due to a lack of sufficient pixels.

[0042] The holographic display and hologram generation method of the present invention implement incoherent light by combining lasers with optical diffusers or using LEDs.

[0043] Moreover, the holographic display and hologram generation method of the present invention address the difficulty of accurately implementing holograms when scattering light in all directions. To overcome this, the method confines the operation within experimentally known pupil size ranges, performs hologram computation based on the size of the pupil, and ensures that spatially incoherent light completely fills the pupil to generate the hologram.

[0044] By applying a spatially incoherent light source rather than temporally incoherent light, the eye box is expanded.

[0045] Additionally, the holographic display and hologram generation method of the present invention involve displaying arbitrary patterns on a spatial light modulator (SLM) for hologram implementation, performing an optical propagation simulation to propagate light over a certain distance, simulating the angles of light entering the pupil, calculating the average intensity of the light entering at those angles, and optimizing the arbitrary pattern by comparing the calculated average intensity with the target intensity.

[0046] In addition, the holographic display and hologram generation method of the present invention implement a consistent 3D holographic image regardless of the viewing position by applying the first light, which is a Coherently Reconstructed Incoherent Sum (CRIS) expressed by the following Equation 1.

[00002]$\begin{array}{cc}{I}_{\mathrm{CRIS}}\left(\stackrel{.fwdarw.}{r},z\right)=\frac{1}{}\underset{{\tilde{k}}_g}{.Math.}{.Math.{}_{}\stackrel{\_}{U}\left(\stackrel{.fwdarw.}{k}+{\stackrel{.fwdarw.}{k}}_g\right)e^{\mathrm{iz}\sqrt{{\left(2/\right)}^2-{.Math.\stackrel{.fwdarw.}{k}.Math.}^2}}{e}^{i\stackrel{.fwdarw.}{k}.Math.\stackrel{.fwdarw.}{r}}d\stackrel{.fwdarw.}{k}.Math.}^2& \left[\mathrm{Equation}1\right]\end{array}$

[0047] Here, I.sub.CRIS({right arrow over (r)}, z) represents the intensity of the first light, z is the distance along the longitudinal direction, N is a normalization constant, {right arrow over (r)} is the distance vector from the center of the spatial light modulator, U({right arrow over (k)}) is the Fourier transform function of the modulation signal of the spatial light modulator 20, {right arrow over (r)} is the radial wavevector before modulation, {right arrow over (k)}.sub.g=(kx, ky) is the modulated radial wavevector, and A is the area of the pupil in the Fourier plane (pupil plane, 41).

[0048] The first light may be generated by deriving a Fourier transform function U({right arrow over (k)}) that satisfies I.sub.CRIS({right arrow over (r)}, z)=Itarget, where Itarget is the intensity of the second light, which is the incoherent sum of the target light.

[0049] By introducing the positional vector of the pupil 40 with respect to the center of the eye box px, py, pz a translational invariance equation as shown in [Equation 2] may be derived.

[00003]$\begin{array}{cc}{I}_{\mathrm{CRIS}}\left(\stackrel{.fwdarw.}{r},z\right)=\mathrm{Itarget}\left(\mathrm{the}\mathrm{intensity}\mathrm{of}\mathrm{the}\mathrm{second}\mathrm{light},\mathrm{which}\mathrm{is}\mathrm{the}\mathrm{incoherent}\mathrm{sum}\mathrm{of}\mathrm{the}\mathrm{target}\mathrm{light}\right)& \left[\mathrm{Equation}2\right]\end{array}$

[0050] The following describes the embodiments in detail with reference to the accompanying drawings. However, the scope of the patent application is not limited or restricted by these embodiments. The same reference numerals in the drawings denote the same elements.

[0051] FIGS. 1A to 1E are diagrams illustrating examples of CRIS implementations.

[0052] In FIGS. 1A to 1E, reference numeral 11 represents light sources from different positions, 11a indicates incoherent light, 11b represents the sum of incoherent light reconstructed as coherent light (first light), and 11C shows light focused by lens 30 and incident on pupil 40.

[0053] FIG. 1A illustrates a point-shaped eye box limited by the space-bandwidth product of the spatial light modulator 20 (Spatial Light Modulator, SLM).

[0054] FIG. 1B shows the configuration of a holographic display 1 according to an embodiment of the present invention for implementing CRIS (Coherently-reconstructed incoherent sum). To achieve the sum of incoherent light propagated coherently in a wavefield, the spatial light modulator 20 must be located at the Fourier plane of the light source, and the pupil 40 should be near the Fourier plane of the spatial light modulator 20.

[0055] FIG. 1C represents a subset of incoherent light passing through pupil 40. To numerically simulate the reconstructed intensity, only the light passing through pupil 40 among numerous illuminations needs to be considered.

[0056] FIG. 1D depicts the paths of two selected subsets of incoherent light for different pupil 40 positions.

[0057] FIG. 1E illustrates the structure of CRISNet, which takes an RGB-D image as input and outputs a modulation for CRIS.

[0058] When incoherent light is directed onto the spatial light modulator 20, the modulated light spreads at a wide angle. The expanded incident angle enables a field of view to be secured from all regions, overcoming the diffraction angle limitations of the SLM in conventional holographic displays.

[0059] However, incoherent light generally does not create interference, making it challenging to reconstruct arbitrary 3D scenes. To utilize the interfering characteristics, the spatial light modulator 20 is positioned at the Fourier plane of the light source, ensuring that light from a single point of the light source forms a plane wave with a specific incident angle on the spatial light modulator 20 plane. With consideration of only specific incident angles, spatial coherence exists, and light modulated by the spatial light modulator 20 may reconstruct arbitrary 3D wavefields. Conversely, light with different incident angles does not interfere due to discrepancies between the light waves.

[0060] When multiple incident lights modulated by the spatial light modulator 20 pass through another lens 30, the light is Fourier-transformed by lens 30 at a specific plane (FIG. 1B).

[0061] For collimated incident light using a radial wavevector (kgx, kgy) the light is focused at the point (fk.sub.gx/2, fk.sub.gy, 2) on the Fourier plane, where A is the wavelength of the light, and f is the focal length of lens 30. By placing pupil 40 on the Fourier plane, only a portion of the incident light, specifically the part concentrated within the region, is allowed. Therefore, only a subset of the incident light is imaged through the aperture of pupil 40, and the total intensity may be expressed by Equation 1 (FIG. 1C).

[00004]$\begin{array}{cc}{I}_{\mathrm{CRIS}}\left(\stackrel{.fwdarw.}{r},z\right)=\frac{1}{}\underset{{\tilde{k}}_g}{.Math.}{.Math.{}_{}\stackrel{\_}{U}\left(\stackrel{.fwdarw.}{k}+{\stackrel{.fwdarw.}{k}}_g\right)e^{\mathrm{iz}\sqrt{{\left(2/\right)}^2-{.Math.\stackrel{.fwdarw.}{k}.Math.}^2}}{e}^{i\stackrel{.fwdarw.}{k}.Math.\stackrel{.fwdarw.}{r}}d\stackrel{.fwdarw.}{k}.Math.}^2& \left[\mathrm{Equation}1\right]\end{array}$

[0062] Here, I.sub.CRIS({right arrow over (r)}, z) represents the intensity of the first light, z is the distance along the longitudinal direction, N is a normalization constant, {right arrow over (r)} is the distance vector from the center of the spatial light modulator, U({right arrow over (k)}) is the Fourier transform function of the modulation signal of the spatial light modulator 20, {right arrow over (k)} is the radial wavevector before modulation, {right arrow over (k)}.sub.g=(kx, ky) is the modulated radial wavevector, and A is the area of the pupil in the Fourier plane (pupil plane 41).

[0063] The first light may be generated by deriving a Fourier transform function U({right arrow over (k)}) that satisfies I.sub.CRIS({right arrow over (r)}, z)=I.sub.target, where I.sub.target is the intensity of the second light, which is the incoherent sum of the target light, as expressed in Equation 1.

[0064] By introducing the positional vector of pupil 40 with respect to the center of the eye box (px, py, pz), a translational invariance equation as shown in [Equation 2] may be derived.

[00005]$\begin{array}{cc}{I}_{\mathrm{CRIS}}\left(\stackrel{.fwdarw.}{r},z\right)=I_{\mathrm{target}}\left(\mathrm{the}\mathrm{intensity}\mathrm{of}\mathrm{the}\mathrm{second}\mathrm{light},\mathrm{which}\mathrm{is}\mathrm{the}\mathrm{incoherent}\mathrm{sum}\mathrm{of}\mathrm{the}\mathrm{target}\mathrm{light}\right)& \left[\mathrm{Equation}2\right]\end{array}$

[0065] Since Equation 2 is valid under the Fresnel approximation, the reconstructed intensity of the first light remains unchanged even if pupil 40 moves along any axis within the region where the Fresnel approximation holds. In other words, by finding an appropriate Fourier transform function U({right arrow over (k)}) that satisfies I.sub.CRIS({right arrow over (r)}, z)=I.sub.target, a wavefield may be displayed to reconstruct a 3D scene viewable from any position. This translational invariance may be understood as a cancellation between the grating phase of the incident angle and the shift in the Fourier domain.

[0066] When pupil 40 moves in the Fourier plane, the specific incident angle of light passing through the center of pupil 40 may be identified (FIG. 1D). Under the Fresnel approximation, the light has the same wavefield as light with an incident angle of 0, except for the oscillatory phase induced by the incident angle.

[0067] The same principle may be applied to off-axis illumination. Consequently, the sum of incoherent light remains identical to the case where pupil 40 is at the center. The translational invariance of CRIS may also be applied to movement along the z-axis.

[0068] Gradient descent may be used to find a wavefield that satisfies Equation 1. For an arbitrary initial wavefield, the right-hand side of Equation 1 is computed, and the difference between the sum of mismatched components and the target intensity is calculated. The wavefield is then updated to minimize the difference, and the process may be repeated until a predefined number of iterations is reached.

[0069] To numerically simulate the propagation of incoherent light, coherent propagation is calculated for 400 different incident angles, and all the coherent propagation results are incoherently summed. Consequently, synthesizing a single hologram requires repeating the 400 coherent propagations hundreds of times, making the algorithm impractical for real-time applications.

[0070] To overcome this limitation, a neural network (CRISNet) was trained, consisting of residual blocks with 33 convolutional layers (FIG. 1E). To train multi-depth holograms, incoherent propagation was numerically calculated for each depth and compared with depth-dependent target images.

[0071] Depth-dependent target images may be synthesized by multiplying an all-in-focus image with a mask where only the pixels corresponding to each specific depth are non-zero.

[0072] As a result, the total loss function may be expressed by Equation 3:

[00006]$\begin{array}{cc}=\underset{n}{.Math.}M\left(\stackrel{.fwdarw.}{r},d_n\right)\left[I_{\mathrm{CRIS}}\left(\stackrel{.fwdarw.}{r},d_n\right)-I_{\mathrm{target}}\left(\stackrel{.fwdarw.}{r}\right)\right]& \left[\mathrm{Equation}3\right]\end{array}$

[0073] Here, d.sub.n is the distance between the n-th layer and the spatial light modulator 20, M({right arrow over (r)}, d.sub.n) is the mask for the pixels of the n-th layer, {right arrow over (r)} is the distance vector from the center of the spatial light modulator, and I.sub.target({right arrow over (r)}) is the target intensity of the first light.

[0074] FIGS. 2A to 2F are diagrams illustrating the numerical reconstruction of CRIS, and FIG. 3 depicts various reconstructed images of CRIS.

[0075] FIGS. 2A and 2B show 3D scenes of numerically reconstructed CRIS. The moon is located on the front plane, while the Earth is on the plane of the spatial light modulator 20. When the focus is on the front, the moon is reconstructed clearly (FIG. 2A), and when the focus is on the spatial light modulator 20 plane, the Earth is reconstructed clearly (FIG. 2B).

[0076] FIG. 2C illustrates a numerically reconstructed stationary 2D scene to evaluate image quality metrics. The PSNR (Peak Signal-to-Noise Ratio) in dB is shown in the upper right corner, and the inset represents an enlarged image.

[0077] FIG. 2D shows enlarged images of numerically reconstructed intensities as a function of pupil 40 position. The pupil 40 positions for each image correspond to (0, L.sub.e{square root over (2)}), (L.sub.e/{square root over (2)}, 0), and (L.sub.e/{square root over (2)}, L.sub.e/{square root over (2)})

[0078] FIG. 2E depicts the average PSNR as a function of pupil 40 position. The positions are normalized by the eye box length Le.

[0079] FIG. 2F is a graph showing the average PSNR and SSIM (Structural Similarity Index Measure) as a function of the pupil 40 position along the x-axis. The dashed line (L1) represents the 3 dB line of the maximum PSNR.

[0080] By synthesizing CRIS, a scene may be reconstructed where the moon is in the foreground and the Earth is on the spatial light modulator 20 plane. The numerical reconstruction demonstrates that when the focus is on the foreground, the moon remains sharp while the Earth appears blurry, indicating 3D reconstruction (FIG. 2A). In contrast, when the focus is on the spatial light modulator 20 plane, the Earth becomes sharp (FIG. 2B).

[0081] FIGS. 3A to 3F show images from the Flickr2K dataset that are numerically reconstructed (FIGS. 3A, 3B, 3C) and optically reconstructed (FIGS. 3D, 3E, 3F) when the eye position is at the center of the eye box. The PSNR (Peak Signal-to-Noise Ratio) in dB is displayed at the top right corner, and the insets represent enlarged images.

[0082] Considering that most image quality metrics are designed for 2D images, the DIV2K validation dataset, a 2D image dataset, was used to quantitatively analyze the image quality of CRIS.

[0083] To simulate floating objects, it was assumed that the image was positioned 1.5 diopters away from the spatial light modulator 20 plane. Here, 1.5 diopters correspond to the average arm's length. The average PSNR of the numerically reconstructed floating images is 29 dB at the center of the eye box, and one of the images with an enlarged inset may be found in FIG. 2C.

[0084] FIGS. 3A to 3F illustrate additional intensities of numerically reconstructed images for various target images

[0085] To demonstrate the translational invariance of CRIS, the same image may be numerically reconstructed at different positions within the eye box (FIG. 2D). Theoretically, the half-length of the eye box along the x-axis and y-axis is given by Le=(8f.sup.2/D).sup., where D is the diopter, and f is the focal length. Each point in FIG. 2D corresponds to (0, L.sub.e/{square root over (2)}), (L.sub.e/{square root over (2)}, 0), and (L.sub.e/{square root over (2)}, L.sub.e/{square root over (2)}).

[0086] To visualize the image quality distribution, CRIS was synthesized for all images in the dataset, and the PSNR values calculated through numerical reconstruction were averaged for each pupil 40 position.

[0087] FIG. 2E shows the average PSNR as a function of the x and y positions in units of Le. Since translational invariance does not extend beyond the valid range of the Fresnel approximation, the average PSNR decreases near the distance Le, confirming the validity of the theoretical results. The cross-sectional graph also confirmed that the full width at half maximum (FWHM) of the PSNR is (FIG. 2F).

[0088] The length of the eye box does not depend on the pixel pitch of the spatial light modulator 20. Therefore, increasing the focal length of lens 30 allows for the use of a spatial light modulator 20 with a larger physical size, enabling the eye box to be expanded while maintaining the field of view.

[0089] Additionally, as the diopter of the hologram increases, the length of the eye box decreases. However, doubling the diopter reduces the eye box length by approximately 15%, which is inversely proportional to the fourth root of the diopter. Under typical conditions, the eye box length of a glass-type display ranges from 10 to 15 mm, while that of a flat-panel display ranges from 50 to 100 mm.

[0090] FIGS. 4A to 4E are diagrams illustrating the optical reconstruction of CRIS.

[0091] FIG. 4A shows the experimental setup. Collimated RGB laser beams are scattered by a diffuser and then collimated by another lens 30. An amplitude-only spatial light modulator 20 with a polarizing beam splitter cube (PBS) is used to reconstruct the hologram. A camera with a lens 30 and an aperture captures the optically reconstructed hologram as its position is adjusted along the x, y, and z directions.

[0092] Optically reconstructed images of the moon and Earth CRIS are displayed when the camera focus is on the foreground (FIG. 4B) and on the spatial light modulator 20 plane (FIG. 4C).

[0093] FIG. 4D shows an optically reconstructed 2D scene floating 1.5 diopters above the spatial light modulator 20 plane. The PSNR in dB is displayed in the top right corner, and the inset represents an enlarged image.

[0094] FIG. 4E depicts enlarged images of optically reconstructed intensities as a function of the pupil 40 position. The axes indicate the positions normalized by the length of the unexpanded eye box.

[0095] FIGS. 5A to 5F illustrate 3D reconstructions at various different positions of pupil 40.

[0096] The 3D scenes are optically reconstructed for various pupil 40 positions when the camera focus is on the foreground (FIGS. 5A, 5B, 5C) and on the spatial light modulator 20 plane (FIGS. 5D, 5E, 5F). Each pupil 40 position in FIGS. 5A, 5B, and 5C corresponds to points A, B, and C in FIG. 3F.

[0097] To confirm the translational invariance of CRIS, the optical reconstruction of the hologram was performed (FIG. 4). While spatial coherence must be suppressed to implement CRIS, it is ideal to maintain temporal coherence for high image quality.

[0098] To create an ideal light source for CRIS, red, blue, and green lasers were combined into a single-mode optical fiber, and a rotating diffuser was used to suppress spatial coherence. Despite using temporally coherent light in the experiments, it was speculated that image quality could be maintained even without temporal coherence.

[0099] An amplitude-only spatial light modulator 20 was used to optically reconstruct CRIS, while CRIS was synthesized to be represented only in amplitude. Unlike conventional holograms, the Fourier plane is filled with light from various incident angles, making it impossible to use a Fourier filter. Thus, all images reconstructed and benchmarked in numerical reconstructions use an amplitude-only spatial light modulator 20.

[0100] Using the same 3D scene presented in the numerical reconstruction, the optical reconstruction showed a sharp moon and a blurry Earth when the camera focus was on the foreground (FIG. 4B). When the camera focus was on the spatial light modulator 20 plane, the Earth became sharp, indicating successful 3D reconstruction (FIG. 4C).

[0101] The suppressed spatial coherence prevents speckle noise caused by laser interference and diffraction patterns from dust, which are major noise sources in conventional holograms, thereby providing high image quality. Additional 3D scenes for various pupil 40 positions are shown in FIG. 5.

[0102] To evaluate image quality metrics based on aperture position, 2D floating images were optically reconstructed, and the PSNR was evaluated as the aperture position varied. FIG. 4E shows an optically reconstructed image with an enlarged inset when the aperture is at the center of the eye box. Even when the aperture moved beyond the boundaries of the unexpanded eye box, the image quality remained similar (FIG. 4F).

[0103] In addition, unlike conventional holograms, the movement of the aperture along the z-axis did not affect image quality or the field of view. Theoretically, the eye box volume is 121253 mm.sup.3, which corresponds to 1,000 times the volume of the unexpanded eye box under the experimental conditions.

[0104] However, due to the limited number of apertures in the setup, the light source area that the optical system may accommodate is restricted. Experimentally, the measured eye box volume was 4.54.512 mm.sup.3, corresponding to a 32-fold expansion.

[0105] The following provides the mathematical basis for the equations used in the present invention.

Translational Invariance Property Under the Fresnel Approximation

[0106] According to the Angular Spectrum Method (ASM), the Fourier-transformed wavefield, U, may be expressed as Equation 4.

[00007]$\begin{array}{cc}\stackrel{\_}{U}\left(k_x,k_y,z\right)=\stackrel{\_}{U}\left(k_x,k_y,0\right)e^{\mathrm{iz}\sqrt{k^2-k_x^2-k_y^2}}\stackrel{\_}{U}\left(k_x,k_y,0\right)e^{\mathrm{ikz}\left(1-\frac{k_x^2+k_y^2}{2k^2}\right)}& \left[\mathrm{Equation}4\right]\end{array}$

[0107] As a result, the propagated wavefield U (x, y, z) may be expressed as Equation 5.

[00008]$\begin{array}{cc}U\left(x,y,z\right)=\stackrel{\_}{U}\left(k_x,k_y,0\right)e^{\mathrm{ikz}\left(1-\frac{k_x^2+k_y^2}{2k^2}\right)}{e}^{{\mathrm{ik}}_{x}x+{\mathrm{ik}}_{y}y}{\mathrm{dk}}_x{\mathrm{dk}}_y& \left[\mathrm{Equation}5\right]\end{array}$

[0108] Since the pupil 40 of the eye functions like an aperture, only the limited region A should be integrated into Equation 5 to numerically reconstruct the visible intensity. By considering spatially incoherent light as the incoherent sum of numerous coherent lights, Equation 5 may be used to represent a hologram reconstructed by coherent light.

[0109] When considering coherent light with different incident angles, the grating phase e.sup.ik.sup.gx.sup.xik.sup.gy.sup.y must be applied to the wavefield before propagation U (x, y, 0). Utilizing the properties of the Fourier transform, changes to (k.sub.x, k.sub.y, 0) changes to (k.sub.x+k.sub.gx, k.sub.y+k.sub.gy, 0) under the grating phase.

[0110] By integrating holograms reconstructed with coherent light at different incident angles, the sum of incoherent light may be expressed as Equation 6.

[00009]$\begin{array}{cc}I\left(x,y,z,p_x,p_y\right)=\frac{1}{}\underset{k_{\mathrm{gx}},k_{\mathrm{gy}}}{.Math.}{.Math.{}_{\left(p_x,p_y\right)}\stackrel{\_}{U}\left(k_x+k_{\mathrm{gx}},k_y+k_{\mathrm{gy}},0\right)e^{\mathrm{ikz}\left(1-\frac{k_x^2+k_y^2}{2k^2}\right)}{e}^{{\mathrm{ik}}_{x}x+{\mathrm{ik}}_{y}y}{\mathrm{dk}}_x{\mathrm{dk}}_y.Math.}^2& \left[\mathrm{Equation}6\right]\end{array}$

[0111] Here, A(p.sub.x, p.sub.y) represents the area corresponding to the position of the pupil. The pupil 40 is defined as a circle with a center at (px, py) and a radius pr.

[0112] To explicitly represent the dependency of the integration area on the pupil 40 position, kx and ky are shifted, and Equation 6 is rewritten.

[0113] Considering that f is the focal length of lens 30 used to perform the spatial Fourier transform, the shifted wavevectors may be defined as k.sub.xk.sub.x+k.sub.px, k.sub.yk.sub.x+k.sub.py,, k.sub.pxk.sub.px/f, k.sub.pyk.sub.py/f

[0114] The transformed equation is provided as Equation 7.

[00010]$\begin{array}{cc}I\left(x,y,z,p_x,p_y\right)=\frac{1}{}\underset{k_{\mathrm{gx}},k_{\mathrm{gy}}}{.Math.}{.Math.{}_{\left(0,0\right)}\stackrel{\_}{U}\left(k_x^{}+k_{\mathrm{px}}+k_{\mathrm{gx}},k_x^{}+k_{\mathrm{py}}+k_{\mathrm{gy}},0\right)e^{\mathrm{ikz}\left(1-\frac{{\left(k_x^{}+k_{\mathrm{px}}\right)}^2+{\left(k_x^{}+k_{\mathrm{py}}\right)}^2}{2k^2}\right)}{e}^{i\left(k_x^{}+k_{\mathrm{px}}\right)x+i\left(k_x^{}+k_{\mathrm{py}}\right)y}{\mathrm{dk}}_x^{}{\mathrm{dk}}_y^{}.Math.}^2& \left[\mathrm{Equation}7\right]\end{array}$

[0115] Since the range of k.sub.gx(k.sub.gy) is from negative infinity to positive infinity, k.sub.gx(k.sub.gy) may be arbitrarily shifted. Thus, k.sub.gx(k.sub.gy) may be replaced with k.sub.gx k.sub.px(k.sub.gyk.sub.py).

[0116] By substituting k.sub.gx(k.sub.gy) and performing simple calculations, the result becomes Equation 8.

[00011]$\begin{array}{cc}I\left(x,y,z,p_x,p_y\right)=\frac{1}{}\underset{k_{\mathrm{gx}},k_{\mathrm{gy}}}{.Math.}{.Math.{}_{\left(0,0\right)}\stackrel{\_}{U}\left(k_x^{}+k_{\mathrm{gx}},k_x^{}+k_{\mathrm{gy}},0\right)e^{\mathrm{ikz}\left(1-\frac{k_x^2+k_y^2}{2k^2}\right)}{e}^{{\mathrm{ik}}_x^{}\left(x-\frac{{\mathrm{xk}}_{\mathrm{px}}}{k}\right)+{\mathrm{ik}}_y^{}\left(y-\frac{{\mathrm{xk}}_{\mathrm{py}}}{k}\right)}{\mathrm{dk}}_x^{}{\mathrm{dk}}_y^{}.Math.}^2& \left[\mathrm{Equation}8\right]\end{array}$

[0117] During the calculation process,

[00012]$e^{-\mathrm{ikz}\left(\frac{k_{\mathrm{px}}^2}{2k^2}+\frac{k_{\mathrm{py}}^2}{2k^2}\right)}{e}^{{\mathrm{ik}}_{\mathrm{px}}x+{\mathrm{ik}}_{\mathrm{py}}y}$

is eliminated. Since it does not include k.sub.x or k.sub.y the pure phase term may be removed during the computation of the squared magnitude.

[0118] To clarify the result, x(y) is replaced with

[00013]$\begin{array}{cc}x+\frac{{\mathrm{zk}}_{\mathrm{px}}}{k}\left(y+\frac{{\mathrm{zk}}_{\mathrm{py}}}{k}\right).& \left[\mathrm{Equation}9\right]\end{array}$$I\left(x+\frac{p_x}{f}z,y+\frac{p_y}{f}z,z,p_x,p_y\right)=\frac{1}{}\underset{k_{\mathrm{gx}},k_{\mathrm{gy}}}{.Math.}{.Math.{}_{\left(0,0\right)}\stackrel{\_}{U}\left(k_x^{}+k_{\mathrm{gx}},k_x^{}+k_{\mathrm{gy}},0\right)e^{\mathrm{ikz}\left(1-\frac{k_x^2+k_y^2}{2k^2}\right)}{e}^{{\mathrm{ik}}_x^{}x+{\mathrm{ik}}_y^{}y}{\mathrm{dk}}_x^{}{\mathrm{dk}}_y^{}.Math.}^2=I\left(x,y,z,0,0\right)$

[0119] On the right-hand side of Equation 9, I(x, y, z, 0, 0) does not depend on px and py, whereas the left-hand term does. Therefore, Equation 9 may be transformed into the form of Equation 10.

[00014]$\begin{array}{cc}I\left(x,y,z,p_x,p_y\right)=I\left(x-\frac{p_x}{f}z,y-\frac{p_y}{f}z,z,0,0\right)& \left[\mathrm{Equation}10\right]\end{array}$

[0120] FIGS. 6A to 6D illustrate the schematic arrangement of the spatial light modulator 20, lens 30, and pupil 40 for translational invariance of the pupil and effective eye box calculation.

[0121] FIG. 6A shows the coordinates for pupil 40 movement along the x-axis and y-axis. For the pupil position (px, py), the virtual image of the reconstructed hologram remains in the same location relative to the eye.

[0122] FIG. 6B shows the coordinates for pupil 40 movement along the z-axis. The pupil 40 moves by pz along the z-axis in the Fourier plane, and the coordinates used in the calculation are indicated at the top of the dotted line.

[0123] As shown in FIG. 6C, for pupil 40 movement in the radial direction, the radial wavevector of light passing through pupil 40 may be expressed as a function of the pupil 40 position.

[0124] As shown in FIG. 6D, when pupil 40 moves along the z-axis, the radial wavevector of light passing through pupil 40 may be expressed as a function of both the pupil 40 position and the wavefield position.

[0125] The reconstructed intensity shifts by

[00015]$\left(\frac{p_x}{f}z,\frac{p_y}{f}z\right)$

according to Equation 10; however, the intensity perceived by the eye does not shift because the eye itself also moves. This condition may be easily understood by visualizing the optically reconstructed virtual image created by lens 30 (FIG. 6A).

[0126] In other words, the observed intensity of the hologram reconstructed by spatially incoherent light does not depend on the position of pupil 40, allowing the hologram to be viewed from any position.

Generalization of Translational Invariance Along the Z-Axis

[0127] To describe pupil 40 movement along the z-axis, the complete optical propagation from the spatial light modulator 20 to the retina must be calculated.

[0128] Propagation may be calculated using the following procedure (FIG. 6B). [0129] 1. Propagation over a distance of f [0130] 2. Phase change by the lens 30 with a focal length of f [0131] 3. Propagation over a distance of f+pz [0132] 4. Phase change by the eye lens 30 with a focal length of f [0133] 5. Propagation over a distance of f

[0134] After a simple calculation based on the Fresnel approximation, the wavefield at the retina may be given as Equation 11.

[00016]$\begin{array}{cc}U\left(x,x,f+f+p_z+f^{}\right)=\frac{1}{{}_0}{e}^{-{\mathrm{ik}}_{x}x\left(\frac{f}{f^{}}\right)-{\mathrm{ik}}_{y}y\left(\frac{f}{f^{}}\right)}\stackrel{\_}{U}\left(k_x,k_y,0\right){\mathrm{dk}}_x{\mathrm{dk}}_y& \left[\mathrm{Equation}11\right]\end{array}$

[0135] Here, N.sub.0 is the normalization constant.

[0136] The right-hand side of Equation 11 remains unaffected by the movement of pupil 40 along the z-axis (pz), resulting in translationally invariant intensity formed on the retina. The only change is the magnification determined by the focal length ratio f/f.

[0137] In summary, by combining the results of the previous subsection with Equation 11, the translational invariance property may be expressed as Equation 12.

[00017]$\begin{array}{cc}I\left(x,y,z,p_x,p_y,p_z\right)=I\left(x,y,z,0,0,0\right)& \left[\mathrm{Equation}12\right]\end{array}$

3D Eye Box Size

[0138] According to Equation 12, the reconstructed intensity does not vary with the position of pupil 40. However, the fundamental limitation of CRIS lies in the Fresnel approximation, and the size of the eye box is constrained by the valid region of the Fresnel approximation.

[0139] Considering a plane wave with a radial wavevector (kx, ky) originating from the spatial light modulator 20, the Fresnel approximation condition may be expressed as Equation 13.

[00018]$\begin{array}{cc}\frac{d}{8}{\left(\frac{k_r}{k}\right)}^{4}1& \left[\mathrm{Equation}13\right]\end{array}$

[0140] Here,

[00019]$k_r=\sqrt{k_x^2+k_y^2},$

d is the propagation distance, is the wavelength of the light, and k is the magnitude of the wavevector. More specifically, the expansion of the CRIS eye box arises from the cancellation of grating phases at the incident angle due to the shift in the Fourier domain. However, when the incident angle becomes large, the approximation

[00020]$d\sqrt{k^2-k_r^2}\mathrm{kd}\left(1-\frac{k_r^2}{2k^2}\right)$

breaks down, and the cancellation becomes incomplete.

[0141] A plane wave with a radial wavevector (kx, ky) is focused at the point (fk.sub.x/k, fk.sub.y/k) on the Fourier plane, and the light is filtered by the pupil 40 (FIG. 6C).

[0142] Assuming that the radius of pupil 40 is smaller than the distance between the optical axis and the center of pupil 40, all radial wavevectors of plane wave light passing through pupil 40 may be approximated as (kx, ky).

[0143] The magnitude of the radial wavevector of light passing through pupil 40 may be expressed as Equation 14.

[00021]$\begin{array}{cc}\frac{k_r}{k}=\frac{1}{f}\sqrt{{p_x}^2+{p_y}^2}& \left[\mathrm{Equation}14\right]\end{array}$

[0144] Here, the position of pupil 40 is set as (px, py, 0).

[0145] When pupil 40 moves along the z-axis, kr depends on the distance r from the center of the spatial light modulator 20. As shown in FIG. 6D, the magnitude of the wavevector may be calculated as

[00022]$\frac{k_r}{k}=\frac{{\mathrm{rp}}_z}{f^2},$

and the maximum wavevector magnitude occurs at the edge of the field of view (FoV).

[0146] Rewriting in terms of the FoV, the maximum wavevector magnitude may be expressed as Equation 15.

[00023]$\begin{array}{cc}\frac{k_{\mathrm{rMAX}}}{k}\frac{1}{f}{p}_z\frac{\mathrm{FoV}}{2}& \left[\mathrm{Equation}15\right]\end{array}$

[0147] By combining Equations 14 and 15, the relationship of the eye box may be expressed as Equation 16.

[00024]$\begin{array}{cc}\frac{k_{\mathrm{rMAX}}}{k}\frac{1}{f}\sqrt{{p_x}^2+{p_y}^2+{p_z^2\left(\frac{\mathrm{FoV}}{2}\right)}^2}& \left[\mathrm{Equation}16\right]\end{array}$

[0148] By applying Equations 13 and 16, the eye box may be expressed as Equation 17.

[00025]$\begin{array}{cc}\frac{1}{f}\sqrt{{p_x}^2+{p_y}^2+{p_z^2\left(\frac{\mathrm{FoV}}{2}\right)}^2}<{a_0\left(\frac{}{d}\right)}^{1/4}{a_0\left(\frac{}{{\mathrm{Df}}^2}\right)}^{1/4}& \left[\mathrm{Equation}17\right]\end{array}$

[0149] Here, a.sub.0=8.sup., Dd/f.sup.2 represents the diopter depth. The condition of very small may be adjusted to small based on (kr/k).sup.4.

[0150] From Equation 17, it may be understood that the CRIS eye box is an ellipsoid, and its volume varies depending on the depth of the hologram.

[0151] By restricting pupil 40 movement along the x-axis (e.g., py=0, pz=0), the length of the eye box along the x-axis may be calculated as

[00026]$.Math.p_x.Math./f<{a_0\left(\frac{}{{\mathrm{Df}}^2}\right)}^{1/4}.$

[0152] Considering the parameters used in the numerical reconstruction, the angular eye box is 0.28 radians, which is nearly identical to the numerically calculated value of 0.29 radians. This discrepancy may be explained by the fact that the eye box criterion in the numerical reconstruction is based on the FWHM (Full Width at Half Maximum) of image quality, whereas the criterion in Equation 17 is based on the initial point of image quality degradation.

[0153] For a typical glass-type display (e.g., focal length 25 mm, wavelength 515 nm, 2 diopters), the eye box length along the x-axis and y-axis is 12 mm.

[0154] For a much larger flat-panel display (e.g., focal length 1000 mm, wavelength 515 nm, 3 diopters), the eye box length along the x-axis and y-axis is 68.5 mm, and the eye box length along the z-axis may be (2/FoV)68.5 mm.

[0155] The eye box does not depend on the pixel pitch or the number of pixels in the display.

Experimental Setup

[0156] Lasers with wavelengths of 638 nm, 515 nm, and 450 nm were combined using a single-mode optical fiber, and the lasers were spatially decorrelated by a rotating diffuser. The spatially incoherent light collimated by lens 30 was illuminated through a PBS (polarizing beam splitter), and only modulated light passed through the PBS.

[0157] An amplitude-only modulator (IRIS-F55, MAY Inc.) with a resolution of 19201080 and a pixel pitch of 6.3 m was used to modulate the light. However, for optical reconstruction, only the 10801080 area was used to minimize noise caused by the limited aperture size. The lens 30 array formed the Fourier plane of the hologram, and an aperture mounted on a motorized stage was configured to move near the Fourier plane. As a result, the aperture mimicked the movement of a human iris.

[0158] The calibration of the spatial light modulator 20 was performed by assigning a single value to the entire modulator and measuring the modulated intensity. After measuring the intensity, a fit function of output-to-input was applied to the input values to correct the modulation.

[0159] The LCOS modulation of the wavefield varies depending on the incident angle, but this difference is negligible if the angular difference is less than 10 degrees. To extend the field of view beyond 10 degrees, a digital micromirror device may be adopted to minimize modulation dependence on the incident angle.

Dependence on Pupil Area

[0160] FIGS. 7A to 7C are diagrams illustrating the dependence on pupil 40 area and computation time.

[0161] FIG. 7A is a graph showing the average PSNR as a function of the relative pupil 40 area. The PSNR was evaluated using the DIV2K validation dataset. The dashed line (L2) represents the 3 dB line from the maximum PSNR.

[0162] FIG. 7B shows images of CRIS numerically reconstructed as a function of the relative pupil 40 area. The large image represents CRIS reconstructed at a relative pupil 40 area of 1.3, corresponding to point F in FIG. 7A. The smaller images represent enlarged views corresponding to the labeled points in FIG. 7A. In the enlarged images, a decline in image quality was observed, but the difference was not noticeable in the overall images.

[0163] FIG. 7C is a graph representing computation time. In FIG. 7C, ITER refers to traditional iterative methods using stochastic gradient descent, Raw represents CRIS synthesis without optimization, and OPT represents CRIS synthesis with applied optimization.

[0164] During CRIS synthesis, only the propagated light passing through pupil 40 area AAA contributes to the reconstruction. Changes in the pupil 40 area resulted in variations in reconstructed intensity, leading to a decline in image quality (FIG. 7A).

[0165] Since the pupil 40 area varies with brightness, it is possible to compensate for the pupil 40 area used in synthesis by detecting ambient brightness. However, even under fixed ambient brightness, emotional arousal may cause up to 20% changes in the individual's pupil 40 area.

[0166] Numerical simulations showed that a 20% change in the pupil 40 area could reduce the PSNR by up to 2 dB, which was insignificant in most cases due to the transient nature of the change. Estimating a 3 dB reduction in PSNR, the FWHM of the relative pupil 40 area was approximately 51%. Furthermore, since the added and subtracted light is incoherent, patterns such as interference and speckle were not observed in reconstructions with mismatched pupil 40 areas (FIG. 7B).

CRISNet Training

[0167] Real-time synthesis of CRIS was achieved using CRISNet, adopting unsupervised learning for optimal image quality.

[0168] The primary computational challenge involved calculating coherent propagation for various incident angles using an extended Fourier domain to accommodate off-axis propagation. By optimizing numerical reconstruction, such as randomly selecting incident angles between 100 and 300 instead of covering 400, and utilizing cached tensors, computation time was reduced to 14.7%.

[0169] CRISNet was trained with an optimized reconstruction method, further reducing computation time by a factor of 1/40,000. It operated at 57 Hz on an NVIDIA RTX4090 without image quality degradation.

[0170] The network was trained using the ADAM optimizer with a learning rate of 0.0001 over 100 epochs. The DIV2K training dataset [10] was used to train CRISNet.

[0171] FIGS. 8A to 8C illustrate the eye box calculation process of holograms in prior art.

[0172] FIG. 8A shows the Fourier-transformed hologram and the intensity of the first light at pupil 40. When the shifted distance of pupil 40 is similar to its radius, the hologram is partially blocked.

[0173] FIG. 8B depicts the PSNR as a function of pupil 40 position. The diameter of pupil 40 was set to 2 mm. The hologram was synthesized using the DIV2K validation dataset with ASM and replayed with an ideal composite spatial light modulator 20.

[0174] FIG. 8C illustrates a hologram blocked due to movement along the z-axis.

[0175] There are two methods for estimating the eye box size of conventional holograms. The first method uses the diffraction limit, while the second method involves numerically calculating image quality, similar to CRIS.

[0176] The eye box size based on the diffraction limit is given by f/Xx, where f is the focal length of the lens, is the wavelength, and x is the pixel pitch.

[0177] For each wavelength used in the experiment, the radial eye box sizes are 1.5 mm, 1.7 mm, and 2.1 mm. Since the overall eye box is limited by the smallest size, the radial length of the eye box is 1.5 mm.

[0178] Meanwhile, the eye function estimated based on image quality may be calculated by assuming the pupil is located on the Fourier plane (FIG. 1A).

[0179] In a spatially coherent light source, when the pupil moves away from the center, high-frequency components are partially blocked by the pupil. These blocked components result in a decline in image quality, and the FWHM (Full Width at Half Maximum) of image quality may be determined at the point where the average PSNR decreases by 3 dB (see FIG. 1B).

[0180] Numerical calculations show that the radial eye box length is approximately 1.2 mm.

[0181] The length of the eye box along the z-axis may be calculated by considering the blocked region when the pupil moves along the z-axis (FIG. 1C). The half-length of the eye box along the z-axis is defined as (2/FoV)pr, where pr is the radial eye box length.

[0182] The eye box size estimated based on image quality may also be used for comparison with CRIS.

Generalization of Translational Invariance Along the z-Axis

[0183] By assuming the Fourier-transformed wavefield on the plane of the spatial light modulator 20 as (k.sub.x,k.sub.y, 0), the wavefield at the retina may be expressed as Equation 18.

[00027]$\begin{array}{cc}U(x\text{?}=\frac{H_o\left(2f+p\text{?}+f^{}\right)}{\left(2\right)\text{?}}e\text{?}e\text{?}e\text{?}e\text{?}e\text{?}U\left(k_z{k}_y,0\right)e\text{?}{\mathrm{dk}}_z{\mathrm{dk}}_{y}e\text{?}\mathrm{dzdy}e\text{?}{\mathrm{dk}}_z^{}\mathrm{dk}\text{?}e\text{?}\mathrm{dk}\text{?}\mathrm{dk}\text{?},& \left[\mathrm{Equation}18\right]\end{array}$$\text{?}\text{indicates text missing or illegible when filed}$

[0184] Here, H.sub.0(d)=e.sup.ikd, and the coordinates follow the notation shown in FIG. 6. Equation 18 is derived under the assumption that the wavefield undergoes the following steps: [0185] 1. Propagation over a distance f [0186] 2. Phase change by the lens

[00028]$\left(e^{-\mathrm{ik}\frac{x^2+y^2}{2f}}\right)$ [0187] 3. Propagation over a distance f+pz [0188] 4. Phase change by the eye lens

[00029]$\left(e^{-\mathrm{ik}\frac{x^2+y^2}{2f^{}}}\right)$ [0189] 5. Propagation over a distance f

[0190] During these steps, Fourier transforms and inverse Fourier transforms are applied to the wavefield.

[0191] To simplify the equation, the following Equation 19 may be used.

[00030]$\begin{array}{cc}\begin{array}{c}{e}^{i\text{?}\left(x^2+y^2\right)e^{\text{?}}}\mathrm{dxdy}=e^{i\text{?}\left(x^2+\frac{\text{?}}{\text{?}}x+{\left(\frac{\text{?}}{\text{?}}\right)}^{\text{?}}+y^2+\frac{\text{?}}{\text{?}}y+{\left(\frac{\text{?}}{\text{?}}\right)}^{\text{?}}\right)}{e}^{-i\text{?}\left({\left(\frac{\text{?}}{\text{?}}\right)}^{\text{?}}+{\left(\frac{\text{?}}{\text{?}}\right)}^{\text{?}}\right)}\mathrm{dxdy}\\ =e^{-i\frac{\text{?}}{\text{?}}}{e}^{i\text{?}\left(x^2+y^2\right)}\mathrm{dxdy}C\left(\right)e^{-i\frac{\text{?}}{\text{?}}}\end{array}& \left[\mathrm{Equation}19\right]\end{array}$$\text{?}\text{indicates text missing or illegible when filed}$

[0192] By doubling Equation 19, Equation 18 is transformed into Equation 20.

[00031]$\begin{array}{cc}U\left(x^{},y^{},f+f+p_z+f^{}\right)=\frac{H_0\left(2f+p_{\text{?}}+f^{}\right)}{{\left(2\right)}^{\text{?}}}{e}^{-\text{?}\frac{\text{?}}{\text{?}}}{e}^{-\text{?}\frac{\text{?}}{\text{?}}}{e}^{-\text{?}\frac{\text{?}}{\text{?}}}{e}^{-\text{?}\frac{\text{?}}{\text{?}}}{e}^{-\text{?}\frac{\text{?}}{\text{?}}}{e}^{\text{?}}{e}^{\text{?}}{e}^{\text{?}}{e}^{\text{?}}{e}^{{\mathrm{ik}}_x^{}{x}^{}+{\mathrm{ik}}_y^{}{y}^{}}\stackrel{\text{?}}{U}\left(k_x,k_y,0\right){\mathrm{dk}}_x{\mathrm{dk}}_y{\mathrm{dk}}_x^{}{\mathrm{dk}}_y^{}{\mathrm{dk}}_x^{}{\mathrm{dk}}_y^{}{\mathrm{dxdydx}}^{}{\mathrm{dy}}^{}=\frac{H_o\left(2f+p_{\text{?}}+f^{}\right)}{\text{?}}{e}^{\text{?}\frac{\text{?}}{\text{?}}}{e}^{\text{?}\frac{\text{?}}{\text{?}}}{e}^{\text{?}\frac{\text{?}}{\text{?}}}{e}^{\text{?}\frac{\text{?}}{\text{?}}}C\left(-\frac{k}{2f}\right)e^{i\frac{\text{?}}{\text{?}}}{e}^{\text{?}}{e}^{\text{?}}{e}^{\text{?}}\stackrel{\text{?}}{U}\left(k_x,k_y,0\right){\mathrm{dk}}_x{\mathrm{dk}}_y{\mathrm{dk}}_x^{}{\mathrm{dk}}_y^{}{\mathrm{dk}}_x^{}{\mathrm{dk}}_y^{}{\mathrm{dxdydx}}^{}{\mathrm{dy}}^{}=\frac{H_o\left(2f+p_{\text{?}}+f^{}\right)}{{\left(2\right)}^{\text{?}}}{e}^{\text{?}\frac{\text{?}}{\text{?}}}{e}^{\text{?}\frac{\text{?}}{\text{?}}}{e}^{\text{?}\frac{\text{?}}{\text{?}}}{e}^{\text{?}}{e}^{\text{?}}C\left(-\frac{k}{2f}\right)e^{i\frac{\text{?}}{\text{?}}}C\left(-\frac{k}{2f^{}}\right)e^{i\frac{\text{?}}{\text{?}}}\stackrel{\text{?}}{U}\left(k_x,k_y,0\right){\mathrm{dk}}_x{\mathrm{dk}}_y{\mathrm{dk}}_x^{}{\mathrm{dk}}_y^{}{\mathrm{dk}}_x^{}{\mathrm{dk}}_y^{}.& \left[\mathrm{Equation}20\right]\end{array}$$\text{?}\text{indicates text missing or illegible when filed}$

[0193] In Equation 20, since

[00032]$k_x^{}$

and are linear, the integral may be replaced with a -function.

[0194] That is,

[00033]$e^{{\mathrm{ik}}_x^{}{x}^{}+{\mathrm{ik}}_y^{}{y}^{}}{e}^{i\frac{-2k_x^{}{k}_x^{}}{2}}{\mathrm{dk}}_x^{}{\mathrm{dk}}_y^{}={\left(2\right)}^2\left(x^{}-k_x^{}{f}^{}/k\right)\left(y^{}-k_y^{}{f}^{}/k\right).$

[0195] Accordingly, Equation 20 is simplified into Equation 21.

[00034]$\begin{array}{cc}U\left(x^{},y^{},f+f+p_z+f^{}\right)=\frac{H_0\left(2f+p_z+f^{}\right)}{{\left(2\right)}^3}C\left(-\frac{k}{2f}\right)C\left(-\frac{k}{2f^{}}\right)e^{-i\left(f+p_z\right)\frac{{\left(\frac{x^{}k}{f^{}}\right)}^2+{\left(\frac{y^{}k}{f^{}}\right)}^2}{2k}}{e}^{-\mathrm{if}\frac{k_{\text{?}}^{\text{?}}+k_{\text{?}}^{\text{?}}}{2k}}{e}^{\frac{{i\left(k_x-\left(\frac{x^{}k}{f^{}}\right)\right)}^2+{i\left(k_y-\left(\frac{y^{}k}{f^{}}\right)\right)}^2}{2k/f}}{e}^{i\frac{{\left(\frac{x^{}k}{f^{}}\right)}^2+{\left(\frac{y^{}k}{f^{}}\right)}^2}{2k/f^{}}}\stackrel{\text{?}}{U}\left(k_x,k_y,0\right){\mathrm{dk}}_x{\mathrm{dk}}_y=\frac{H_0\left(2f+p_z+f^{}\right)}{{\left(2\right)}^3}C\left(-\frac{k}{2f}\right)C\left(-\frac{k}{2f^{}}\right)e^{-i\left(f+{\text{?}}_{\text{?}}\right)\frac{{\left(\frac{\text{?}}{\text{?}}\right)}^2+){\left(\frac{\text{?}}{\text{?}}\right)}^2}{\text{?}}}{e}^{i\frac{{\left(\frac{\text{?}}{\text{?}}\right)}^2+{\left(\frac{\text{?}}{\text{?}}\right)}^2}{\text{?}}}{e}^{i\frac{{\left(\frac{\text{?}}{\text{?}}\right)}^2+{\left(\frac{\text{?}}{\text{?}}\right)}^2}{\text{?}}}{e}^{-{\mathrm{ik}}_x{x}^{}\left(\frac{1}{f^{}}\right)-{\mathrm{ik}}_y{y}^{}\left(\frac{1}{f^{}}\right)}\stackrel{\text{?}}{U}\left(k_x,k_y,0\right){\mathrm{dk}}_x{\mathrm{dk}}_y=\frac{1}{{}_0}{e}^{-{\mathrm{ik}}_x{x}^{}\left(\frac{1}{f^{}}\right)-{\mathrm{ik}}_y{y}^{}\left(\frac{1}{f^{}}\right)}\stackrel{\text{?}}{U}\left(k_x,k_y,0\right){\mathrm{dk}}_x{\mathrm{dk}}_y& \left[\mathrm{Equation}21\right]\end{array}$$\text{?}\text{indicates text missing or illegible when filed}$

[0196] Here, N.sub.0 includes all phase factors that are ignored during the intensity calculation. The last line of Equation 21 is identical to Equation 11.

Number of Incident Angles

[0197] FIGS. 9A and 9B are graphs showing relative PSNR as a function of NoIA (Number of Incident Angles).

[0198] FIG. 9A shows the relative PSNR of numerical reconstruction as a function of NoIA per axis. The dashed line represents values smoothed using the adjacent averaging method. FIG. 9B depicts the relative PSNR of synthesized CRIS as a function of the minimum NoIA per axis.

[0199] To numerically simulate the propagation of incoherent light, coherent propagation is calculated for all incident angles, and all coherent propagation results are added incoherently. Ideally, simulating incoherent light numerically requires an infinite number of incident angles (NoIA). However, 400 NoIA produces results similar to much larger numbers, making 400 the standard NoIA (FIG. 9A).

[0200] More specifically, the 2D domain of NoIA is divided into a regular grid, ensuring that each axis has the same number of divisions. To reduce the optimization time for CRIS, NoIA was selected randomly.

[0201] First, to maximize the effect of selecting different numbers, fractional values were chosen based on the difference between the minimum and maximum ranges. Among them, 2 and 3 appeared insufficient, and 5 was avoided because it is a multiple of 10. Thus, the difference between the minimum and maximum was set to 7. Subsequently, to find the optimal minimum value, CRIS was synthesized using NoIA ranges from x to x+7, where xxx could range from 5 to 14 (FIG. 9B). To analyze the synthesized CRIS based on NoIA ranges, the PSNR of the synthesized CRIS was evaluated using 400 NoIA. To achieve sufficient speed while providing reasonable PSNR, x=10 was selected.

[0202] FIG. 10 is a graph showing PSNR as a function of wavelength differences.

[0203] The wavelength difference represents the gap between the wavelength used in synthesis and the wavelength used in reconstruction for all three colors. The red dashed line represents values smoothed using the adjacent averaging method.

Wavelength Dependence of CRIS

[0204] In CRIS reconstruction, temporal coherence is maintained, but CRIS may still be successfully reconstructed even in cases of temporal incoherence, provided the spectral bandwidth is not excessively wide. FIG. 10 shows the average PSNR as a function of the wavelength difference between the wavelength used in synthesis and the wavelength used in reconstruction.

[0205] When the wavelength difference is as large as 100 nm, the PSNR degradation is only about 0.5 dB. Therefore, using LED light sources minimizes degradation in the reconstructed image quality.

[0206] FIGS. 11A and 11B are diagrams showing the numerically reconstructed intensity of the first light, including additional losses.

[0207] FIG. 11A depicts the numerically reconstructed intensity focused on the spatial light modulator 20 plane without L DB, while FIG. 11B shows the numerically reconstructed intensity focused on the spatial light modulator 20 plane using L DB.

Additional Loss to Enhance Defocus Blur

[0208] To demonstrate improvements in defocus blur, Equation 22 was adopted as the defocus blur loss function.

[00035]$\begin{array}{cc}{}_{\mathrm{DB}}=-.Math.\frac{I\left(x,y\right)}{I\left(x,y\right)+1}.Math.+.Math.I_G\left(x,y;\right).Math.& \left[\mathrm{Equation}22\right]\end{array}$

[0209] Here, I(x,y) represents the intensity of the numerically reconstructed CRIS on the spatial light modulator 20 plane, IG(x, y, ) is the Gaussian blur intensity with a sigma value , and (.Math.) is a user-selected constant.

[0210] The first term of Equation 22 enhances speckles, similar to phase-only holograms. Simultaneously, the second term reduces the overall gradient by excluding speckle patterns through Gaussian blur, thereby improving defocus blur.

[0211] FIG. 11 shows reconstructed images with defocus blur loss adopted at =1, =1, and =0.005. The demonstrated loss function is an example of an additional loss function used to enhance defocus blur.

[0212] The present invention may also be configured to achieve natural defocus blur.

[0213] FIG. 12 is a flowchart of a hologram generation method according to another embodiment of the present invention.

[0214] As shown in FIG. 12, the hologram generation method of the present embodiment, implemented through the holographic display 1 described in FIGS. 1 to 8, may include: a first step (S10) of performing a light propagation simulation, a second step (S20) of calculating the average intensity of light incident into the pupil, a third step (S30) of optimizing an arbitrary pattern by comparing the average intensity of light with the target intensity, and a fourth step (S40) of implementing a consistent 3D holographic image regardless of the viewing position.

[0215] The first step (S10) of performing the light propagation simulation may involve inputting arbitrary RGB-D video signals into the spatial light modulator 20, which receives light from the light source 10, and performing a light propagation simulation where the first light, which is the Coherently-reconstructed Incoherent Sum (CRIS) generated as coherent light by the spatial light modulator 20, propagates over a certain distance.

[0216] The second step (S20) of calculating the average intensity of light incident into the pupil 40 may involve simulating the angles of light entering pupil 40 and calculating the average intensity of the first light incident at those angles.

[0217] The third step (S30) of optimizing an arbitrary pattern by comparing the average intensity of light with the target intensity may involve optimizing the arbitrary pattern by comparing the average intensity of the first light with the intensity of the target second light.

[0218] The first light may be generated based on Equation 1.

[0219] The fourth step (S40) of implementing a consistent 3D holographic image regardless of the viewing position may involve ensuring translational invariance so that the intensity of the first light remains the same regardless of the movement of the pupil's position.

[0220] The translational invariance of the first light may be expressed by introducing the positional vector of the pupil relative to the center of the eye box (px, py, pz) as follows:

[00036]$I_{\mathrm{CRIS}}\left(\stackrel{.fwdarw.}{r},z,\mathrm{px},\mathrm{py},\mathrm{pz}\right)=I_{\mathrm{CRIS}}\left(\stackrel{.fwdarw.}{r},z,0,0,0\right),$

where I.sub.CRIS({right arrow over (r)}, z) represents the intensity of the first light, z is the distance along the longitudinal direction, and the Fourier transform function U({right arrow over (k)}) satisfying I.sub.CRIS({right arrow over (r)}, z)=I.sub.target, with I.sub.target being the intensity of the second light, which is the incoherent sum of the target light, may be derived and implemented.

[0221] The present invention provides a novel passive method to overcome the limited eye box of holographic displays while delivering image quality and responsiveness comparable to conventional displays. It utilizes incoherent light that may be decomposed into a sum of coherent light to expand the eye box. This invention theoretically proves the translational invariance of CRIS (Coherently-Reconstructed Incoherent Sum) and derives an eye box formula consistent with numerical simulation results.

[0222] According to the formula, the expanded volume of CRIS was 1,000 times larger than the unexpanded eye box without degradation in image quality. To validate the theory, 3D reconstruction and the translational invariance of CRIS were experimentally demonstrated. However, due to the limited numerical aperture of the proof-of-concept experimental setup, the eye box expansion was restricted to 32 times.

[0223] Additionally, simulating the sum of incoherent light requires hundreds of coherent propagations and significantly more computational resources than conventional holograms. To address this, a neural network model for real-time synthesis was developed. By solving key challenges such as image quality, real-time synthesis, eye box limitations, and responsiveness, CRIS may enable the widespread adoption of holographic displays.

[0224] This invention may be applied to various fields, including HMDs (head-mounted displays), VR (virtual reality), and AR (augmented reality).

[0225] As described above, the embodiments of the present invention have been explained with specific details such as concrete components and limited examples and drawings. These are provided solely to facilitate a better understanding of the invention and are not intended to limit the invention to the described embodiments.

[0226] Those skilled in the art to which this invention pertains will understand that various modifications and variations may be made based on the described disclosures. Accordingly, the scope of the present invention should not be construed as limited to the disclosed embodiments. Rather, it should encompass all equivalents or equivalent modifications within the scope of the appended claims and their equivalents.

DESCRIPTION OF SYMBOLS

[0227] 1: Holographic Display [0228] 10: Light Source [0229] 11: Light (emitted from different points of the light source) [0230] 20: SLM (Spatial Light Modulator) [0231] 30: Lens [0232] 40: Pupil [0233] 41: Pupil Plane