Method for processing a holographic image

Abstract

A method for processing a sequence of holographic images with a view to playing the images back on a holographic display device to at least one viewer. A subset of wavelet coefficients relevant for the reconstruction of a sub-hologram visible for the at least one viewer from at least one viewing point is selected from a decomposition of at least one holographic image on a wavelet basis and information representative of a location of the at least one viewer in a repository of the display device. In the method, decomposition of the at least one holographic image is carried out on a Shannon wavelet basis.

Claims

1. A method comprising: processing a sequence of holographic images by a processing device with a view to reproducing the sequence on a holographic display device displaying to at least one observer, wherein processing comprises: selecting a subset of relevant wavelet coefficients for reconstruction of a sub-hologram visible for the at least one observer from at least one observation point, the subset being selected from a decomposition of at least one holographic image on a Shannon wavelet basis and from information representing a location of the at least one observer with respect to the display device; and transmitting the sub-hologram to the holographic display device.

2. The method as claimed in claim 1, wherein the reconstruction of the sub-hologram visible by the at least one observer from the at least one observation point comprises: determining maximum and minimum diffraction angles of an incident ray on the hologram, which are subtended by the at least one observation point, determining minimum and maximum frequencies for creating a diffraction of an incident ray according to the determined angles, determining a Shannon wavelet having a frequency range equal to the difference between the determined frequencies and a central frequency equal to half the sum of the determined frequencies, and obtaining the sub hologram by convolution of the holographic image and the determined Shannon wavelet.

3. The method as claimed in claim 2, wherein the convolution is carried out in the frequency domain between the Fourier transform of the hologram and the Fourier transform of the determined Shannon wavelet.

4. A device comprising: a processor; and a non-transitory computer-readable medium comprising instructions stored thereon, which when executed by the processor configure the device to process a sequence of holographic images with a view to reproducing the sequence on a holographic display device displaying to at least one observer, wherein processing comprises: selecting a subset of relevant wavelet coefficients for reconstruction of the sub-hologram visible for the at least one observer from at least one observation point, the subset being selected from a decomposition of at least one holographic image on a Shannon wavelet basis and from information representing a location of the at least one observer with respect to the display device; and transmitting the sub-hologram to the holographic display device.

5. The device as claimed in claim 4, wherein the instructions further configure the processor to: determine maximum and minimum diffraction angles of an incident ray on the hologram, which are subtended by the at least one observation point, determine minimum and maximum frequencies for creating a diffraction of an incident ray according to the determined angles, determine a Shannon wavelet having a frequency range equal to the difference between the determined frequencies and a central frequency equal to half the sum of the determined frequencies, and obtain a sub-hologram by convolution of the holographic image and the determined Shannon wavelet.

6. A server comprising the device as claimed in claim 4.

7. A holographic reproduction terminal comprising the device as claimed in claim 4 and the holographic display device.

8. A non-transitory computer-readable information medium on which there is recorded a computer program comprising instructions for execution of a method of processing a sequence of holographic images by a processing device with a view to reproducing the sequence on a holographic display device displaying to at least one observer, when the instructions are executed by a processing device, wherein processing comprises: selecting a subset of relevant wavelet coefficients for reconstruction of a sub-hologram visible for the at least one observer from at least one observation point, the subset being selected from a decomposition of at least one holographic image on a Shannon wavelet basis and from information representing a location of the at least one observer with respect to the display device; and transmitting the sub-hologram to the holographic display device.

Description

LIST OF FIGURES

(1) Other features and advantages of the invention will become clearer upon reading the following description of a particular implementation, given by way of simple illustrative and nonlimiting example, and from the appended drawings in which:

(2) FIG. 1 illustrates an architecture suitable for implementing the processing method, according to a particular implementation.

(3) FIG. 2a illustrates the frequency spectrum of a Gabor wavelet, as described previously,

(4) FIG. 2b illustrates the frequency spectrum of a Shannon wavelet,

(5) FIG. 3a represents a hologram in one dimension and rays diffracted toward an observation point,

(6) FIG. 3b represents the minimum and maximum diffraction angles subtended by an observation point,

(7) FIG. 3c represents the minimum and maximum diffraction angles subtended by an observation window,

(8) FIG. 4a represents a frequency view of a hologram in one dimension,

(9) FIG. 4b illustrates a rectangular function, the frequency band and the central frequency of which are determined according to a particular implementation of the invention,

(10) FIG. 5 illustrates the characteristic angles for obtaining a diffraction from a point of a two-dimensional hologram toward a particular observation point,

(11) FIG. 6 illustrates in the form of a diagram the main steps of the processing method according to a particular implementation of the invention, and

(12) FIG. 7 represents the architecture of a processing device according to a particular implementation of the invention.

DETAILED DESCRIPTION

(13) In the description that follows, with reference to FIG. 1, a display device 100 comprising a Spatial Light Modulator (SLM), for example an LCD (Liquid Crystal Display) screen, is considered. The SLM is composed of pixels. The set of pixels of the SLM needed to display a wavelet forms a pixel of the hologram, which will be considered point-based and will hereinafter be referred to as point. The method is implemented on an item of server equipment 101 suitable for receiving information on the positioning of an observer 102 relative to the SLM. The positioning information can be captured by a movement detection device 105 and transmitted by a connection 104 to the server 101. The server 101 communicates with the display device 100 via a communication network 103. The calculated sub-holograms are transmitted in the form of wavelet coefficients to the SLM. According to a particular implementation, the method can be implemented in a reproduction device 100 or in the server 101.

(14) The general principle of the invention relies on the decomposition of a holographic image from a Shannon wavelet basis. The inventors observed that the decomposition of a holographic image into Shannon wavelets was exhibiting advantages related to the properties of these wavelets. These advantages provide for markedly reducing the complexity of the processing operations and the memory required to implement a method for compressing a sequence of holographic images, enabling real-time execution of the method.

(15) The properties of Shannon wavelets are for example described in an article by Carlo Cattani entitled Shannon Wavelets Theory, and published in 2008. The mother function of a Shannon wavelet is a sine cardinal, the Fourier transform of which is a rectangular function as illustrated in FIG. 2b representing the frequency spectrum of a Shannon wavelet. A Shannon wavelet is obtained by multiplication of a sinusoidal function of determined frequency with this mother function defined for a given interval. The direction of diffraction of a light beam by a Shannon wavelet is determined by the frequency of the sinusoid, the time and space dimensions of the mother function and the orientation of the wavelet.

(16) According to a particular implementation of the invention, a holographic image is decomposed on a basis of Shannon wavelets to which are applied particular rotation, translation and dilation parameters. These parameters are calculated for each of the points of the hologram in such a way that an incident ray passing through the wavelet thus parametered is diffracted toward at least one determined observation point. Thus, each image of the holographic image sequence is decomposed on a basis of N Shannon wavelets, thereby leading to obtaining a representation of each image of the sequence in the form of a set of N Shannon wavelet coefficients, where N is the number of points of the hologram.

(17) With reference to FIG. 3a, a two-dimensional space S containing a hologram H is considered, the hologram including n points, the points being indexed by k. By way of example, the hologram represented in FIG. 3a includes one dimension and is composed of seven points. Of course, the invention applies in the same way to two-dimensional holograms and it is only for the sake of simplification of the description that the example is given for a one-dimensional hologram. The hologram can be defined as follows:
H={H.sub.1,H.sub.2,H.sub.3, . . . ,H.sub.n}(1)

(18) An observation point V belonging to S is also considered. Of course, the invention also applies when the space S has three dimensions and/or a plurality of observation points. For example, when several observers look at a hologram, the invention provides an optimized selection of sub-holograms visible by the various observers.

(19) According to a first step, the method according to the invention proposes determining a frequency for each index point k of the hologram H, this frequency being such that the incident ray on this point is diffracted toward the observation point V. To that end, the method utilizes the properties of diffraction gratings.

(20) Let X: R.fwdarw.R.sup.2 be a function making any point of S correspond to a position according to two dimensions (in this example, the two dimensions correspond to the position of the index point k in the hologram and to its distance relative to the observation point V. The position of the index point k belonging to H is given by X(k). The position of the point V t in this same system is given by X(V).

(21) Let : R.sup.2.fwdarw.R be a function making X(V) correspond to the spherical coordinates (V)={.sub.1, .sub.2, .sub.3, . . . , .sub.n} corresponding to the angle subtended by the point V for each of the index points k of the hologram H represented in FIG. 3a.

(22) Also, let g: R.fwdarw.R be a function giving the localized spatial frequency from (V).

(23) Hence the frequency is obtained for creating a diffraction of an incident ray from each of the points of the hologram toward the observation point V, being the wavelength of the light:

(24) $\begin{array}{cc}f=g\left(\left(V\right)\right)& \left(2a\right)\\ f_V=\frac{\sin \left(\left(V\right)\right)}{}& \left(2b\right)\\ f_V=f_1,f_2,f_3,\mathrm{.Math.},f_n& \left(2c\right)\end{array}$

(25) It is appropriate to recall that the example used here to support the description corresponds to a hologram with only one dimension. Since the hologram is hence in the same plane as the observation point, the direction of diffraction is determined only by the angle subtended by the point V. In the case of a two-dimensional hologram in a three-dimensional space, it is appropriate to also take into account the orientation for causing a diffraction toward the point V For example, with reference to FIG. 5, which illustrates a two-dimensional holographic image 500 and a plane 501 comprising an observation point V (502) in a three-dimensional space, the diffraction of the incident ray 503 toward the observation point 502 is determined not only by the angle subtended by the point V denoted by (V) in the figure (zenith angle), but also by the rotation angle denoted by (V) (azimuth angle).

(26) According to a particular implementation, the method according to the invention includes a step for determining a sub-hologram able to generate a diffraction toward at least one observation point. To that end, the method proposes determining, for each index point k of the hologram, a Shannon wavelet obtained by multiplication of a sinusoidal function, the frequency of which is suitable for causing a diffraction at the index point k toward at least one observation point, and a sine cardinal type mother function defined over a particular interval. The sinusoidal function determines the central frequency of the wavelet and the mother function its frequency band. Thus, for each point k of the hologram H, a function is determined, centered on the point k, the frequency f.sub.k of which is suitable for producing a diffraction toward a particular observation point. Thus, the set of wavelets providing a diffraction toward a particular observation point is denoted by:
.sub.V={.sub.1,f.sub.1,.sub.2,f.sub.2,.sub.3,f.sub.3, . . . .sub.n,f.sub.n}(3)

(27) In the case of a two-dimensional hologram, a rotation is additionally applied to the wavelet so as to modify the azimuth angle of the diffracted light, denoted by (V) in FIG. 5.

(28) The decomposition of Shannon wavelets C.sub.V into coefficients corresponds to a set of scalar products of a function representing the holographic image H and the base function .sub.V. The scalar product of .sub.V and the hologram H is given by:
C.sub.V={<H,.sub.1,f.sub.1>,<H,.sub.2,f.sub.2>,<H,.sub.3,f.sub.3>, . . . ,<H,.sub.n,f.sub.n>}(4a)
C.sub.V(k)=.sub.iH.sub.k,f.sub.k(4b)
H.sub.V=.sub.kC.sub.V(k).sub.k,f.sub.k(4c)

(29) Thus, a sub-hologram H.sub.V is obtained, generating a diffraction to the observation point V. The wavelet coefficients thus calculated for each of the points can for example be transmitted over a telecommunication network and used to reconstruct the sub-hologram on a suitable display device.

(30) Nevertheless, since it is composed of points, a hologram is of discrete nature. It is hence appropriate to discretize the wavelets used. A Shannon wavelet, in its complex and continuous form, centered on the frequency F.sub.c, for which the frequency band is F.sub.b, is characterized by the following equation:
.sub.sh={square root over (F.sub.b)}(sin c(F.sub.bx).Math.exp(2iF.sub.cx))(5)

(31) The Fourier transform of this wavelet is given by:

(32) $\begin{array}{cc}=\mathrm{rect}\left(\frac{u-F_c}{F_b}\right)& \left(6\right)\end{array}$

(33) Since the hologram H is composed of N discrete points, it is appropriate to calculate a discrete Fourier transform (DFT) of the hologram and to take into account the spacing of these points (i.e. the pitch, denoted by P) in order to discretize the Shannon wavelet. The spacing between the points of the DFT is given by:

(34) $\begin{array}{cc}{D}_p=\frac{1}{\mathrm{PN}}& \left(7\right)\end{array}$

(35) The size of the wavelet must be at least sufficient to cover a half-period of the base sinusoid. Thus, the number of discrete points needed for a wavelet, i.e. its length in the spatial domain, is given by:

(36) $\begin{array}{cc}{L}_n=\frac{1}{f_V\left(k\right)P}& \left(8\right)\end{array}$

(37) The number of discrete points of the DFT of the wavelet, i.e. its size in the frequency domain, can thus be deduced:

(38) $\begin{array}{cc}M=\frac{N}{L_n}& \left(9\right)\end{array}$
and the frequency band:
f.sub.b=MD.sub.p(10)

(39) It is observed from equations (8) and (10) that when f.sub.V(k) increases, L.sub.n decreases, thereby indicating a better spatial localization, but f.sub.b increases, thereby indicating a less good frequency localization, and vice versa.

(40) FIG. 4a schematically illustrates a representation in the frequency domain of a sub-hologram H.sub.V obtained by equation (4b). Each rectangular function .sub.V(k) corresponds to a localized scalar product, the frequency band of which is denoted by f.sub.b(k) and the central frequency by f.sub.V(k).

(41) Thus, the processing method proposes a decomposition of a hologram on a discretized Shannon wavelet basis given by:
V.sub.k(m)={square root over (f.sub.b(k))}(sin c(f.sub.b(k)m).Math.exp(2if.sub.V(k)m))(11)

(42) FIG. 5 illustrates a two-dimensional holographic image 500 and a plane 501 comprising an observation point V (502) in a three-dimensional space. An incident ray 503 is picked out. It is observed that the Shannon wavelet that this incident ray passes through generates a light ray diffracted toward the observation point V according to an azimuth angle (V) and a zenith angle (V). Since the Shannon wavelet is discretized, continuous changes of the values (, ) do not generate continuous changes of the pair (O, S). Therefore, a range is defined in order that each pair (O, S) correspond to [, +] [, +]. The parameter corresponds to an approximation of the ideal direction of diffraction and provides for anticipating the movements of the observer. This parameter corresponds to the bandwidth f.sub.b of the Shannon wavelet. With reference to FIG. 2b, it is observed that the Fourier transform of a Shannon wavelet is a rectangular function. The wavelet thus has the same energy over its entire frequency band. On the other hand, as FIG. 2a shows, which represents a Fourier transform of a Gabor wavelet, the energy of a Gabor wavelet is concentrated around its central frequency and decreases rapidly upon moving away from this frequency. While according to the prior art a new sub-hologram must be calculated as soon as the user moves away from the observation point for which a first sub-hologram was calculated, the fact that a Shannon wavelet preserves the same energy over the entire range defined by means that the user can move away from the observation point to a certain extent determined by without it being necessary to recalculate a sub-hologram. Thus, while the prior art requires calculating several sub-holograms in order to anticipate movements of the user, the method according to the invention enables a reduction in the number of sub-holograms to be precalculated, this reduction resulting in reducing the complexity of the processing operations and the memory needed to store these sub-holograms. The method thus exhibits a certain advantage over the prior art.

(43) A sub-hologram H.sub.V is obtained from the minimum number of wavelet coefficients which is necessary to create a diffraction to the point V. Conventionally, the sub-hologram can be calculated by a scalar product of the set of parametered wavelets and the set of points of the holographic image. This calculation requires Ln complex multiplications and Ln complex additions, where L is the number of discrete points in the wavelet .sub.V(k) and n is the number of points in the holographic image. It is observed that the calculation of a sub-hologram causing a diffraction to the point V is particularly costly in calculation time according to this method. On the other hand, this approach also requires a significant quantity of memory to store the vectors .sub.V for each Ln points of the hologram. For the hologram to be visible from several observation points, for example from N points, the memory required will be LnN complex values. Likewise, the complexity of the calculation will be in the order of LnN complex multiplications and LnN complex additions. This complexity corresponds to the calculation of a sub-hologram according to only one dimension. For the case of a conventional two-dimensional hologram, the complexity of the calculation for a hologram including nm points which is visible from a viewing area having NM positions will be in the order of L.sup.2nmNM complex multiplications and of L.sup.2nmNM complex additions, with a memory occupation of L.sup.2nmNM complex values.

(44) Such complexity prevents a real-time calculation.

(45) In order to reduce the complexity and provide a real-time processing operation, the invention proposes, according to a particular implementation, a method suitable for calculating the sub-holograms in the frequency domain so as to reduce the complexity of the processing operation and the memory required. To that end, the invention takes advantage of a property of Shannon wavelets according to which the Fourier transform of such a wavelet is a rectangular function.

(46) With reference to FIG. 4b, the invention proposes determining a rectangular function suitable for covering the frequencies causing a diffraction of the light to the point V.

(47) FIG. 6 illustrates the main steps of the processing method according to a particular implementation.

(48) According to a particular implementation, the calculation of a sub-hologram visible from the observation point comprises a first step denoted by 600 in FIG. 6 for determining the maximum and minimum angles of diffraction of an incident ray subtended by the observation point. FIG. 3b represents a hologram H and an observation point V. The minimum incident diffraction angle on the observation point V is denoted by .sub.min. The maximum incident diffraction angle on the observation point V is denoted by .sub.max. These angles can be determined from the position of the observer relative to the holographic reproduction device. The position of the observer can be known by virtue of an eye-tracking or movement detection device.

(49) During a step 601, the method proposes determining a frequency for causing a diffraction of an incident ray according to the determined minimum angle and a frequency for causing a diffraction of an incident ray according to the determined maximum angle. The frequency corresponding to a diffraction angle is given by equation (2b).

(50) At step 602, a Shannon wavelet is determined, the wavelet being such that its frequency range is equal to the difference between the determined minimum and maximum frequencies, and its central frequency is equal to half the sum of the determined frequencies. Thus, the frequency band is given by the equation:
f.sub.b=max(f.sub.v)min(f.sub.v)(12)
and the central frequency by the equation:

(51) $\begin{array}{cc}{f}_V^{}=\frac{\max \left(f_V\right)+\min \left(f_V\right)}{2}& \left(13\right)\end{array}$

(52) The corresponding Shannon wavelet is hence given by:
.sub.V(m)={square root over (f.sub.b)}(sin c(f.sub.bm).Math.exp(2if.sub.Vm))(14)

(53) The sub-hologram corresponding to the observation point can be obtained by a scalar product of the hologram and the wavelet obtained at step 602. However, an advantage of the invention over the prior art is that it provides for obtaining the sub-hologram by a simple convolution operation performed at step 603. In the frequency domain, this convolution operation corresponds to a simple multiplication and provides for very significantly reducing the number of complex multiplications to be performed to calculate the sub-hologram.

(54) It is observed in FIG. 4b that the sub-hologram H.sub.v obtained is a superset of the sub-hologram H.sub.V obtained by the method represented in FIG. 4a. The sub-hologram H.sub.v contains all the frequencies required for a diffraction to the point V but also additional frequencies, since all the frequencies (discrete) between min (f.sub.V) and max (f.sub.V) are selected. This has the effect of slightly lowering the effectiveness of the compression, but the gain in terms of complexity of the processing operations and memory requirements is particularly advantageous.

(55) On the other hand, since the sub-hologram is entirely calculated in the frequency domain, the method according to the invention also exhibits advantages during the reconstruction of the sub-hologram, since it avoids having to recalculate Fourier transforms.

(56) In most cases, a single sub-hologram causing a diffraction toward only one observation point is not satisfactory. This is because the hologram is no longer visible when the observer moves slightly. It is hence necessary to calculate several sub-holograms corresponding to each of the points of a viewing window, such as for example the viewing window 504 represented in FIG. 5, i.e. a window from which the hologram must be visible. Thus, the observer can move in the limit of the viewing window and continue to see the hologram. According to the prior art, the complexity of the processing operation is then multiplied by the number of points of the viewing window.

(57) According to a particular implementation, the method proposes calculating the maximum .sub.max and minimum .sub.min diffraction angles incident on the viewing window and determining the frequencies causing such diffractions. Thus, an incident ray on any point of the viewing window will have an angle of incidence of between .sub.min and .sub.max. FIG. 3c represents a hologram H having one dimension and an observation window V, the endpoints of which are denoted by P1 and P2. The maximum and minimum angles subtended by the point P1 are denoted by .sub.min1 and .sub.max1. The maximum and minimum angles subtended by the point P2 are denoted by .sub.min2 and .sub.max2. The minimum and maximum angles incident on the observation plane V can hence be deduced, the minimum angle .sub.min being equal to min(.sub.min1, .sub.min2) and the maximum angle .sub.max being equal to max(.sub.max1,.sub.max2). It is thus possible to determine, from the equations specific to the diffraction gratings, the minimum and maximum frequencies causing an incident diffraction on the viewing window. From these minimum and maximum frequencies, steps 602 and 603 of the method provide for calculating a sub-hologram visible over the whole of the viewing window by a simple multiplication of the Fourier transform of the hologram by the Fourier transform of the Shannon wavelet obtained.

(58) FIG. 7 represents the architecture of a device 700 for processing a holographic image.

(59) The device 700 comprises a storage space 701, for example a memory MEM, and a processing unit 702 equipped for example with a processor PROC. The processing unit can be driven by a program 703, for example a computer program PGR, implementing the processing method as described in the invention with reference to FIG. 6, and notably the steps for determining the maximum and minimum diffraction angles of an incident ray on the hologram which are subtended by the at least one observation point, for determining the minimum and maximum frequencies for creating a diffraction of an incident ray according to the determined angles, for determining a Shannon wavelet, the frequency range of which is equal to the difference between the determined frequencies and the central frequency of which is equal to half the sum of the determined frequencies, and for obtaining a sub-hologram by convolution of the holographic image and the determined Shannon wavelet.

(60) Upon initialization, the instructions of the computer program 703 are for example loaded into a RAM (Random Access Memory) memory before being executed by the processor of the processing unit 702. The processor of the processing unit 702 implements the steps of the processing method according to the instructions of the computer program 703.

(61) To this end, the device comprises, in addition to the memory 701, a unit 704 configured to determine maximum and minimum diffraction angles of an incident ray on the hologram which are subtended by the at least one observation point. With this aim, the device can use data obtained from a device for detecting movement of an observer, this device being able to be directly connected to the processing device or communicate with the processing device via a communication network. In that case, the processing device can comprise a communication unit, such as for example a network card, suitable for transmitting and receiving data, such as for example positioning data of an observer relative to a holographic reproduction device. The communication device 708 can also be used to transmit a sub-hologram calculated according to the invention to a holographic reproduction device, such as for example an SLM. The processing device also comprises a module 705 (FREQ) configured to determine minimum and maximum frequencies for creating a diffraction of an incident ray according to the determined angles. The unit 705 can implement an algorithm suitable for calculating diffractions in a diffraction grating and in particular for implementing equation (2b). The device also comprises a unit 706 (SHAN) configured to obtain a Shannon wavelet, the frequency range of which is equal to the difference between the determined frequencies and the central frequency of which is equal to half the sum of the determined frequencies. To that end, the module 706 can implement equations (12) and (13) in the form of an algorithm for obtaining equation (14). Lastly, the processing device comprises a unit 707 (CONV) for calculating a sub-hologram through a convolution operation carried out from a Fourier transform of a hologram and a Fourier transform of the Shannon wavelet determined by the module 706. The communication unit 708 can be used to transmit the coefficients resulting from the convolution operation to a holographic reproduction device.

(62) According to a particular implementation, the device can be integrated in an item of server equipment suitable for communicating with a holographic reproduction device, or integrated directly in an item of holographic reproduction equipment.