Device and method for calculating holographic data

Abstract

An apparatus and a method for optimized calculation of 2D sub-holograms for object points of a three-dimensional scene and a pipeline for real-time calculation of holograms are provided. The invention shortens the calculation time of a hologram for representing a three-dimensional scene and/or to reduce the calculation complexity of such a hologram. This is achieved by a 2D sub-hologram of an object point, which has image elements of the spatial light modulator, comprises a half 1D sub-hologram, where the radius of each image element is determined and each image element of the 2D sub-hologram is fixedly assigned to at least one image element of the half 1D sub-hologram with identical or similar radius by way of an electronic circuit, by a method for encoding a hologram, and by a pipeline on the basis of FPGA and/or ASIC.

Claims

1. A pipeline for real-time calculation of holograms comprising a calculator to calculate sub-holograms and a controller to control a holographic display, the calculator and the controller are each configured as functional units, where the pipeline is realized on the basis of at least one application field programmable logic gate array and/or at least one application-specific integrated circuit, where the functional units are configurable during the runtime.

2. The pipeline as claimed in claim 1, further comprising an apparatus for calculating a 2D sub-hologram for representing an object point of a three-dimensional scene using a holographic display, comprising: a spatial light modulator with a matrix of image elements, the 2D sub-hologram containing image elements of the spatial light modulator and having a rotational symmetry, the 2D sub-hologram comprising a half 1D sub-hologram along a section through the 2D sub-hologram from the origin of the 2D sub-hologram up to a maximum radius of the 2D sub-hologram, the radius of each image element is determined and each image element of the 2D sub-hologram is fixedly assigned to at least one image element of the half 1D sub-hologram with identical or similar radius by way of an electronic circuit.

3. The pipeline as claimed in claim 1, wherein the functional units comprise a functional unit for receiving object points for describing a scene to be reconstructed, a functional unit for calculating the hologram and a functional unit for outputting the hologram for representation on a spatial light modulator, wherein the functional units of the pipeline are fixedly integrated, but are not assigned to a specific spatial light modulator or holographic display.

4. The pipeline as claimed in claim 1, further comprising calculator to calculate in real-time both of 1D hologram and 2D hologram and/or supporter to support various encoding types and output modes.

5. The pipeline as claimed in claim 1, wherein the resolutions for hologram and 3D scene to be represented are independent of the hardware used.

6. The pipeline as claimed claim 1, further comprising a scalability of the circuit for different display sizes and/or hologram resolutions and/or scene resolutions and/or display parameters by a variable activation of the computation paths.

7. The pipeline as claimed in claim 1, further comprising maximizing the workload of the circuit and minimizing the required resources by the realization of dynamic sub-hologram sizes.

8. The pipeline as claimed in claim 1, further comprising a hologram calculation and hologram output with a different frame rate which are asynchronous with respect to one another.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

(1) In the figures:

(2) FIG. 1: shows the calculation procedure for a complex-valued hologram of a three-dimensional scene according to the prior art

(3) FIG. 2: shows the mirror symmetry in 2D sub-holograms according to the prior art

(4) FIG. 3: shows an example for assigning the image elements of the 2D sub-hologram matrix to a 1D sub-hologram

(5) FIG. 4: shows the generation of intermediate values by way of interpolation

(6) FIG. 5: shows the calculation procedure for a complex-valued hologram of a three-dimensional scene by utilizing the method according to the invention for generating 2D sub-holograms of object points of this scene with utilization of a 1D sub-hologram, which extends along the x-axis of the 2D sub-hologram.

(7) FIG. 6: shows a holographic computational pipeline without using symmetry according to the prior art

(8) FIG. 7: shows a holographic computational pipeline for a method for encoding a hologram of a three-dimensional scene by using a symmetry by way of mirroring and alignment (first solution variant of a method for encoding a hologram of a three-dimensional scene)

(9) FIG. 8: shows a holographic pipeline with symmetry in a variant which is independent of the sequence

(10) FIG. 9: shows a holographic computational pipeline for a method for encoding a hologram of a three-dimensional scene by using a symmetry by way of mirroring and alignment, but in a variant which renders alignment of the unmirrored sub-hologram segment unnecessary (second solution variant of a method for encoding a hologram of a three-dimensional scene)

(11) FIG. 10: shows a holographic pipeline with symmetry in a variant that is dependent on the sequence

(12) FIG. 11: shows a pipeline for hardware-based real-time calculation of holograms with the aid of sub-holograms and direct controlling of a holographic display.

DETAILED DESCRIPTION OF THE INVENTION

(13) In the present invention, a rotational symmetry is advantageously used for the apparatus and the method for calculating 2D sub-holograms 2D-SH and as such for the calculation of holograms, as described above. Use of the mirror symmetry can be dispensed with. Since the phase calculation described in the prior art in principle tends to mean a calculation of phase values that describes or approximates the imaging function of a lens with a focus F, the function parameters x and y are used to describe a radius R, wherein the phase, i.e. the complex value, is identical for all x and y that generate the identical radius R. R is obtained from x and y with the rule: R.sup.2=x.sup.2+(S*y).sup.2, where S=p.sub.y/p.sub.x. The scalation with the parameter S serves for correctly taking into account the vertical pitch of the image elements BE in the radius calculation for rectangular image elements BE, for which p.sub.x!=p.sub.y. The new rule for calculating the phase is φ=(π*R.sup.2*p.sub.x.sup.2)/(λ*F)+φ0. The complex value is obtained from the phase φ and amplitude A.

(14) A complex value in this context is understood to mean in particular a complex number in the mathematical sense. Mirror symmetry can be understood to mean at least one axis symmetry and/or a point symmetry, wherein the point symmetry can relate in particular to the centerpoint of a sub-hologram SH.

(15) It is then possible for the associated radius to be determined for each image element BE in the first quadrant or also for all other quadrants. By calculating a phase profile or a section—for example along the x-axis or along a diagonal—a function profile is obtained which can be regarded as a type of look-up table and/or be used as a look-up table. It is then possible to determine the complex value for each image element by way of the fact that the radius serves as an argument for the look-up table.

(16) The utilization of such a method, however, is highly memory-intensive, since it is possible to operate in parallel here only to a limited extent. Accesses to a look-up table generally take place sequentially since memory modules, owing to the principle design, provide data only sequentially. That is to say, such an implementation using access to a look-up table on a computer system or on an FPGA (field programmable gate array) or an ASIC (application-specific integrated circuit) is limited in particular owing to the memory bandwidth of the data access. Even the use of multiples of the number of memory modules for processing a plurality of accesses in parallel is sensible only to a limited extent, since this is accompanied with enormous use of resources.

(17) This aspect is addressed according to the invention by way of a special electronic, preferably digital, circuit, which is particularly applicable to FPGAs and ASICs. In very general terms, however, it is possible to use any suitable computer hardware for this.

(18) Instead of calculating or applying the rotational symmetry sequentially consistently live by a look-up table, a hard-wired matrix is created as a special digital circuit, which matrix electrically connects each image element BE of a 2D sub-hologram 2D-SH with at least one specific image element BE in a half 1D sub-hologram 1D-SH. This is illustrated in FIG. 3, which shows an example of the assignment of the image elements of the 2D sub-hologram matrix 2DSHM to a 1D sub-hologram 1D-SH in the physical sense and also in the sense according to the method. Here, the following method steps are illustrated:

(19) the 1D sub-hologram calculation and transformation into the Cartesian space, SHBT,

(20) the 2D sub-hologram superposition, 2DSHSP.

(21) In addition, the following apparatus features are illustrated:

(22) a 1D sub-hologram half consisting of a plurality of image elements for different radii, wherein each image element stores a complex value, H1DSH.

(23) a 2D sub-hologram matrix, the image elements of which are directly connected to the corresponding image elements of the half 1D sub-hologram 1D-SH. 2DSHM,

(24) the direct electronic wiring of the register of an image element in the half 1D sub-hologram with all relevant image elements in the 2D sub-hologram, DEV. This takes place analogously for all other cells.

(25) In the first step, a 1D sub-hologram 1D-SH of half size is calculated. The result is a greater number of complex values k consisting of in each case imaginary value (or imaginary part) and real value (also referred to as real part). Each complex value is now buffered in two registers, one for real value, one for imaginary value. A register is a digital-electronic circuit for storing number values in digital circuits. There are thus k*2 registers in a 1D sub-hologram 1D-SH with a number of n image elements BE. Furthermore, a matrix of A′*B′ image elements BE exists, in each case consisting of two registers—this represents a 2D sub-hologram 2D-SH. A specific number of these image elements BE in the 2D sub-hologram 2D-SH was assigned the most suitable image element BE in the 1D sub-hologram 1D-SH. The assignment took place off-line. Said image elements BE are now connected by way of the fact that an electrical, fixedly installed, parallel, continuous and in particular complete data transfer from the two registers in the 1D sub-hologram 1D-SH to the registers in the 2D sub-hologram 2D-SH that are determined according to the assignment takes place. This means that every time a value in a register in the 1D sub-hologram 1D-SH changes, the corresponding registers in the 2D sub-hologram 2D-SH likewise automatically change. This transfer usually takes place with edge triggering. However, other transfer methods are also conceivable. Just as conceivable is the use of electric analog storage and transfer instead of digital transfer.

(26) The choice as to which image elements BE of the 2D sub-hologram 2D-SH are connected to which image elements BE of the half 1D sub-hologram 1D-SH takes place via the radius. Always that image element BE of the 1D sub-hologram 1D-SH the radius R of which corresponds most closely to the radius R′ of the image element BE of the 2D sub-hologram 2D-SH is selected, and the complex value thereof is then used for the image element BE of the 2D sub-hologram 2D-SH, see in this respect FIG. 3. Here, for the image element BE of a 2D sub-hologram 2D-SH with the radius R′=4.47, the image element BE of the half 1D sub-hologram 1D-SH with the closest value of the radius R=4.5 is chosen. Assignment for all other cells takes place analogously. However, instead of selecting the closest radius R, any other rules are also conceivable.

(27) It can thus be necessary to tolerate a certain quantization error, since the image elements BE are calculated only for a limited number of radii R.

(28) A great power gain lies in the fact that only a fraction of a 2D sub-hologram 2D-SH to be calculated needs to be calculated in the form of the half 1D sub-hologram 1D-SH. In a preferred variant of the apparatus according to the invention and of the method according to the invention, the very calculation-intensive transformation into the Cartesian space takes place.

(29) Another objective is to find a way to reduce the complexity involved in the hologram calculation with 2D sub-holograms 2D-SH. One way in this direction is the realization of an apparatus with a hardware architecture for calculating hologram data. This should be effected preferably in real time, in particular for terminals on the basis of FPGA and/or ASIC, which are characterized primarily by appropriate or low energy consumption. These would permit, for example, use in the mobile field and could be manufactured with realistic expenditure and manageable costs.

(30) The number of the transistors used for the electric circuit of the apparatus influences the costs of the manufacture but also the energy consumption thereof. That is to say, the larger the electrical circuit, the more transistors need to be provided and the more complicated and costly are the manufacture and development thereof. This is essential especially for mobile terminals with a limited energy reserve and limited heat dissipation.

(31) The advantages of 2D encoding, i.e. encoding with 2D sub-holograms 2D-SH, over 1D encoding, that is to say encoding with 1D sub-holograms 1D-SH, are known and were described in the publications mentioned in the introductory part. The important features to be mentioned in particular are:

(32) the elimination of the astigmatism in the holographic representation or reconstruction of individual object points with a great scene depth (wherein an object to be represented can be decomposed into individual object points) and

(33) a full holographic parallax (2D) in the holographic visualization.

(34) In order to reduce a quantization error, it is possible for the number k of the image elements BE in the half 1D sub-hologram 1D-SH to be increased, i.e. to reduce the step width to the increment of the radius R, see in this respect FIG. 3, with a doubling of the number k of image elements BE (that is to say a half radius step width). A finer assignment of image elements of the 2D sub-hologram 2D-SH to image elements of the half 1D sub-hologram 1D-SH is thus possible. For this purpose, a linear increase or any desired other increase function can be used. Since the quantization error increases toward the edge of the 2D sub-hologram 2D-SH, a non-linear (for example quadratic) increase of the radius R along the half 1D sub-hologram 1D-SH is sensible, i.e. in the edge region the resolution is significantly higher than in the region of the first image element with the radius 0. In contrast, a linear increase is less effective in causing increased accuracy in the edge region, but this has the advantage that the calculation of the 1D sub-hologram 1D-SH can be significantly simplified, and therefore also be implemented more effectively.

(35) In the hologram reconstruction, this leads, with linear increase, to the accuracy of the reconstruction being very good in the center of the viewing window, i.e. the “window” in which the reconstruction of the three-dimensional scene is visible to the observer, see in this respect for example WO 2006/066919 A1, but decreases depending on the selected resolution toward the edge, since here the quantization errors increase and become particularly noticeable at the edge of the viewing window. In contrast, a consistent quality can be achieved for a non-linear curve, i.e. a constant error can be achieved—substantially in the entire region of the viewing window.

(36) A further possibility for reducing the quantization errors is to carry out interpolations between neighboring image elements BE in the half 1D sub-hologram 1D-SH in order to generate further intermediate values in additional registers with low additional complexity. This is illustrated in FIG. 4. What is shown in particular here is how, from a 1D sub-hologram half, consisting of a plurality of image elements for different radii, wherein each image element stores a complex value, H1DSH, intermediate values of two or more image elements are generated, ZWG,

(37) and thus an extended 1D sub-hologram with additionally generated intermediate values, E1DSH, is produced.

(38) The simplest example is a linear interpolation of two neighboring image elements BE with the radii R.sub.1=4.5 and R.sub.2=5 to generate an intermediate value with R.sub.12=4.75. The calculation rule would be Re.sub.12=(Re.sub.1+Re.sub.2)/2 and Im.sub.12=(Im.sub.1+Im.sub.2)/2. In a digital circuit, only a single addition is necessary for this, since the division by two can take place by cutting off the last bits without effort. In addition to linear interpolation, quadratic or logarithmic or exponential interpolation is also possible.

(39) The decision as to which method is applied concretely, and the determination of the resolution on which the calculation and the “copying step” should be based, must be made depending on the specific case and error tolerance of the holographic system.

(40) The 2D sub-hologram matrix and its fixed link with the half 1D sub-hologram register are determined, on the basis of the maximum occurring sub-hologram size, off-line or not in real time and subsequently synthesized in an electric digital circuit, which can be implemented in an FPGA/ASIC.

(41) The novel method is then integrated into the procedure used thus far. The module, connected upstream, for calculating a half 1D sub-hologram 1D-SH generates the data, in particular the complex values in the Cartesian space, which are transferred into the above-mentioned 1D sub-hologram register. Immediately or, depending on the implementation, a few clock cycles later, the complete 2D sub-hologram 2D-SH is then available. The known accumulation—that is to say adding up of the calculated or determined 2D sub-holograms—into the sum hologram takes place in one or more processing steps.

(42) Despite the fixedly implemented matrix, the method makes possible the use of dynamic sub-hologram sizes, i.e. the sub-hologram size varies depending on the distance of the associated object point from the hologram plane or is dependent on the position of the object point to be represented, the current position of the viewing window which is positioned in the immediate vicinity of an observer's eyes, and the position of the spatial light modulator SLM, by way of the fact that only the relevant part of the half 1D sub-hologram 1D-SH is calculated and only the corresponding detail of the 2D sub-hologram 2D-SH is also further processed. This enables balancing of the calculational power for three-dimensional scenes with a very high and at the same time with low complexity, or it enables energy saving mechanisms to be used in less complex three-dimensional scenes.

(43) The method can be used for spatial light modulators SLMs with square or with rectangular image element structures. The half 1D sub-hologram 1D-SH is here calculated along the direction of higher resolution, possibly with additional image elements, thus smaller radius increments. Thus sufficient intermediate values are available to determine the direction of lower resolution. In principle it is also conceivable to use other sub-hologram geometries, for example a line, circle or polygon shape.

(44) The method can also be used to combine rotational symmetry and mirror symmetry. Here, for example only the first quadrant of the 2D sub-hologram 2D-SH or even only half of it is determined with the proposed method, wherein the halving of the first quadrant takes place along the diagonal. The remaining three quadrants can then be determined by mirror symmetry. This is sensible in particular if the accumulation of the 2D sub-hologram 2D-SH is to take place at another position within the calculation system and only a limited bandwidth for transmission is available.

(45) The advantage of the method according to the invention and of the apparatus according to the invention is the clear simplification of the calculation of 2D sub-holograms 2D-SH in FPGAs or ASICs. The order of magnitude for reducing the required calculational power is approximately a factor of 100; this can be more or less, depending on the display or SLM type. The calculation complexity for a complete 2D hologram for representing the three-dimensional scene, that is to say what is known as a 2D sum hologram, thus decreases substantially to the level of a 1D hologram calculation, but with corresponding additional complexity in terms of the accumulation and the hologram storage, since significantly higher quantities of data must be processed. However, since the calculation units for these large data quantities are omitted, corresponding resources are saved: especially the step of transforming into the Cartesian space (with sin/cos) is completely omitted at this point.

(46) A disadvantage could be that, owing to quantization effects, the accuracy is reduced. However, by correspondingly matching the sampling rate by way of increasing the number of image elements BE in the half 1D sub-hologram 1D-SH for the 1D sub-hologram 1D-SH to be calculated, this can be compensated for according to the requirements with acceptable additional effort.

(47) FIG. 5 finally illustrates the calculation procedure for a complex-valued hologram for representing a three-dimensional scene with utilization of the method according to the invention for generating 2D sub-holograms of object points of said scene with utilization of a 1D sub-hologram that extends along the x-axis of the 2D sub-hologram. This procedure is a significant simplification of the method as compared to a calculation procedure according to the prior art, see FIG. 1, since the first complex steps are reduced to the half 1D sub-hologram, while they always take place for the entire 2D sub-hologram according to the prior art.

(48) At this point, further hardware aspects should be described, which are useful for the achievement of the above-mentioned object of the invention:

(49) The starting point is a holographic computational pipeline without utilization of a symmetry according to the prior art, as is shown in FIG. 6.

(50) The sub-hologram segment generator SHSG generates here a sub-hologram segment with complex values for image elements in the polar coordinate system SHP. What follows is a sub-hologram segment transformation SHST, which results in a transformed sub-hologram segment with complex values for image elements in the Cartesian coordinate system SHK.

(51) Subsequently, the sub-hologram segment addition SHSA takes place using the hologram row memory HZS. The resulting hologram row HZ consists of a plurality of added-up sub-holograms (superposition of sub-holograms).

(52) Since no symmetries are used, the calculation and superposition for all image elements of the sub-hologram takes place, which leads to a high calculational complexity.

(53) In the development of circuits for application-specific integrated circuits ASICs and application field programmable logic gate arrangements FPGAs, there are many challenges and guidelines for implementing calculation procedures efficiently and in a space-saving manner. Important objectives are a low space requirement, since the costs are lower the fewer transistors are included, and a relatively high frequency, because the higher the frequency, the fewer transistors are necessary, since each transistor can produce more work in less time—this depends on the field of use and the energy requirement: the higher the frequency, the higher the energy consumption.

(54) In application-specific integrated circuits ASICs and application field programmable logic gate arrays FPGAs, in contrast to conventional computer systems, no linear program is executed, but produced is a fixed switching network to which data are supplied to the input and which outputs data again after a finite period of time. By logical operations within the fixed circuit, the data are processed, changed, linked and so on. Arithmetic operations (for example addition, subtraction, multiplication, comparisons) or memory cells or elements (registers, latches or addressable memory blocks) can be produced, for example.

(55) Binary and logical links, i.e. links with and, or etc. can be carried out therewith. Access to memory blocks in the application field programmable logic gate arrays FPGAs must only take place with data which are aligned on the basis of the memory addresses—in contrast to computers, where the data is automatically aligned by the processor, which on the other hand is also associated with delays. Generally, processor-based computer architectures (with von-Neumann-principle) are very slow in comparison to application-specific integrated circuits ASICs and field programmable gate arrays FPGAs, since the processor-based computer architectures operate quasi sequentially.

(56) An important feature of circuits is in particular the simplification of calculation procedures into as many simple and linear procedures as possible with as few data dependencies as possible, that is to say fixed paths for achieving high frequencies. Setting calculation procedures to run in parallel enables a high throughput overall. Ideal are especially image-based calculations, including holography, which process data in the sequence in which they arrive, since intermediate storage, in particular in the case of large data quantities, often represents an enormous bottleneck.

(57) The use of the symmetry mentioned above in generating sub-holograms SH is relatively simple to formulate and implement in theory or on a computer system with sequential processing and any desired sequential memory access. However, implementing this in a fixed, linear and parallel operating system is complicated. This is explained for 1D sub-holograms below. Since the method can be applied in principle also to 2D sub-holograms by repeating the method steps according to the number of the used hologram rows, the following makes reference only to sub-holograms.

(58) The motivation is to save the calculation of one of the two sub-hologram halves—so only one half of a sub-hologram SH would have to be calculated. This is noticeable especially in the transformation, i.e. the conversion, of the values of the memory elements that are assigned to the image elements of the sub-hologram SH from the polar representation into the Cartesian representation. For this circuit part, the space requirement is the greatest.

(59) For the sub-hologram calculation for ASICs or FPGAs, several complex values of neighboring image elements of the sub-hologram are calculated in parallel—so-called sub-hologram segments, for example with a segment width SB=16. A sub-hologram segment thus contains in each case one complex value (phase or value, or real part or imaginary part) per memory element, assigned to the image elements of a sub-hologram SH.

(60) The problem is now to duplicate the generated sub-hologram segments, to mirror them, and to align them with the segment width to which the memory width corresponds (in this case for example 16 memory elements) and to add them up in local hologram row memories. This should be effected in a manner which saves as much space as possible and/or uses as few resources as possible, so as to be significantly better than the normal execution without taking into account the symmetry. In the variant without symmetry, no additional alignment is necessary since the sub-hologram segments can already be correctly aligned in this case during generation.

(61) The generated sub-hologram segment of the first half is calculated and subsequently transformed from polar coordinates into Cartesian coordinates. During the calculation, two offset positions are determined, one for the sub-hologram segment of one half, and one for the sub-hologram segment, which is to be mirrored later, of the other half. The offset positions determine the position or the memory address in the hologram row at which the segments are added up in a complex-valued manner, wherein the memory address, that is to say the memory element, can in turn be assigned to a concrete image element BE, although such an assignment does not take place in a fixed manner.

(62) After the transformation, for example by a CORDIC algorithm, a sine-cosine algorithm in the FPGA, duplication of the sub-hologram segment into a second computation path takes place. In this step, the mirroring of the memory elements with respect to their arrangement inside the segment takes place simultaneously. That means that memory element 0 becomes (in the case of 16 memory elements) memory element 15, 1 becomes 14, 2 becomes 13 etc.

(63) This is illustrated in FIG. 7, which shows a first solution variant of a method for encoding a hologram of a three-dimensional scene, specifically a holographic computational pipeline for a method for encoding a hologram of a three-dimensional scene by utilizing a symmetry by way of mirroring and alignment.

(64) The following steps are shown therein:

(65) the sub-hologram segment transformation SHST,

(66) the mirroring of the incoming sub-hologram segment SHSP, which leads to the mirrored sub-hologram segment G1DSH in a second computation path,

(67) the enlargement of the sub-hologram segment to twice the width SHVGR in both computation paths, followed in each case by the

(68) displacement of the sub-hologram segment by N′ or N″ elements, either in stages, i.e. in N′ or N″ steps, or immediately with the aid of a multiplexer SHVSCH,

(69) separating the double-width sub-hologram segment into two segments aligned at the memory SHTR, and as a result

(70) the sub-hologram segment A1DSH that is aligned at the memory.

(71) In this first solution variant, both computation paths therefore now in each case include one sub-hologram segment, which must both be added up into the current hologram row. In order for this to happen, both segments must be aligned at the hologram row memory. To this end in each case an unaligned sub-hologram segment is mapped onto two neighboring aligned sub-hologram segments. This happens as follows: a sub-hologram segment is enlarged to twice the width (here an example from 16 to 32 elements with complex-valued content), wherein the upper 16 memory elements are zero-padded. The 16 memory elements are now displaced by N′ or N″ memory elements within the doubled sub-hologram segment. N′ or N″ is obtained from the offset position of the image element BE of the sub-hologram SH and the number of image elements or memory elements per sub-hologram segment. In the described example, the number N′=13 and N″=5. Subsequently, a separation occurs back into two sub-hologram segments, which are now both aligned, as is likewise illustrated in FIG. 7 for the first solution variant.

(72) The displacement by N′ or N″ memory elements can in turn be effected in two different ways. First, it is possible to implement a pipeline with a number of stages corresponding to the segment width, in the example described here this would be 16, which displaces in each stage the memory elements by one memory element of the hologram row memory, or passes them on without displacement. The last stage then contains the result. In a second variant of the displacement, all possible displacements are defined in the form of a fixed logic, in the described example this would again be 16 displacement possibilities. The current displacement N′ and N″ is selected and carried out by a multiplexer.

(73) Since all sub-hologram segments are aligned, the addition can then take place in the hologram row. Since the amount of data has quadrupled, it is also necessary to process, i.e., add up, four times the data amount. Four hologram row memory blocks are therefore necessary, which add up the four sub-hologram segments in parallel in four independent hologram rows. Since only ever two neighboring sub-hologram segments arrive per computation path, this means that, at any one time, only ever one even and one odd sub-hologram segment arrive. That is to say, instead of two full hologram row memories, two hologram row memories of half the size suffice, the first stores only the even sub-hologram segments, the other the odd ones.

(74) In a final step, after completion of the hologram row, the four partial regions in the independent hologram row memories are combined, that is to say added up.

(75) The corresponding steps in this regard are illustrated in FIG. 8, which shows a holographic pipeline with symmetry in a variant which is independent of the sequence of the incoming sub-hologram segments:

(76) A sub-hologram segment generator SHSG first generates a sub-hologram segment with image elements, that is to say with complex values for the image elements, in the polar coordinate system SHP. This is converted by the sub-hologram segment transformation SHST into a transformed sub-hologram segment with image elements in the Cartesian coordinate system SHK.

(77) Using the sub-hologram segment mirroring and sub-hologram segment alignment described in FIG. 7—in the first solution variant of a method for encoding a hologram of a three-dimensional scene—SHSSA_A—sub-hologram segments with an even address GSH and sub-hologram segments with an odd address USH are generated, in each case in the original and in the mirrored form.

(78) These sub-hologram segments are subsequently fed to the sub-hologram segment addition SHSA. The hologram row memory HZS necessary herefor requires only 50% of the size as opposed to a hologram row memory according to the prior art.

(79) In the hologram row adder HZA, the addition of the four hologram rows calculated in the preceding step takes place, the result of which is a hologram row HZ, consisting of a plurality of added-up sub-holograms, i.e. a superposition of sub-holograms.

(80) The first solution variant described here of a method for encoding a hologram of a three-dimensional scene can then be improved further: such an improved variant will be described below as second solution variant of a method for encoding a hologram of a three-dimensional scene, see in this respect also FIG. 10.

(81) The generated sub-hologram segment is here already generated such that no alignment at the hologram row memory is necessary anymore for this unmirrored sub-hologram segment. As a result, only one hologram row memory is required for this path.

(82) This is described first in FIG. 9. The latter shows a holographic computational pipeline for a method for encoding a hologram of a three-dimensional scene by using a symmetry by way of mirroring and alignment in a second solution variant of a method for encoding a hologram of a three-dimensional scene. In the variant described, alignment of the unmirrored sub-hologram segment is, however, unnecessary.

(83) In FIG. 9, the following steps are illustrated in this regard:

(84) the sub-hologram segment transformation SHST,

(85) the mirroring of the incoming sub-hologram segment SHSP, which leads to the mirrored sub-hologram segment SH-Seg.sub.T G1DSH in a second computation path,

(86) outputting the stored sub-hologram segment SH-Seg.sub.T−1 from the register for buffering SHR the current mirrored sub-hologram segment, and receiving the new sub-hologram segment SH-Seg.sub.T for storing for a unit of time

(87) connecting the current sub-hologram segment SH-Seg.sub.T with the buffered sub-hologram segment SH-Seg.sub.T−1 SHVB

(88) displacing the sub-hologram segment by N′ elements, either in stages, i.e. in N′ steps, or immediately with the aid of a multiplexer SHVSCH,

(89) separating off the sub-hologram segment aligned at the memory SHABSP,

(90) the sub-hologram segment A1DSH which is aligned at the memory.

(91) Therefore, the sequence of steps which are described for the mirrored sub-hologram segment in the second computation path is unnecessary in the first computation path, since the corresponding sub-hologram segment is aligned at the memory from the very start.

(92) For the mirrored path, the following processing therefore takes place: after the sub-hologram segment SH-Seg.sub.T is duplicated into the second computation path and mirrored, it is linked with the sub-hologram segment SH-Seg.sub.T−1, stored in the preceding cycle, to form a sub-hologram segment which is twice the width, that is to say in the example described here with 32 image elements BE. This results in the displacement and extraction of the 16 image elements BE, which are now aligned correctly at the hologram row memory: this is illustrated accordingly in FIG. 9.

(93) The displacement or extraction takes place analogously to the two above-described methods by pipeline or by multiplex, by displacing all image elements BE by N′ image elements BE. The bottom 16 image elements then correspond to the aligned sub-hologram segment.

(94) As a result, likewise only one hologram row memory is now necessary for the mirrored computation path.

(95) After the row is completed, both rows are combined, i.e. added up.

(96) FIG. 10 shows this as an overview for such a holographic pipeline with symmetry in a variant which is dependent on the sequence of the incoming sub-hologram segments.

(97) A sub-hologram segment generator SHSG first generates a sub-hologram segment with image elements, i.e. with complex values for the image elements, in the polar coordinate system SHP. This is converted by the sub-hologram segment transformation SHST into a transformed sub-hologram segment with image elements in the Cartesian coordinate system SHK.

(98) Using the sub-hologram segment mirroring and sub-hologram segment alignment described in FIG. 9—in the second solution variant of a method for encoding a hologram of a three-dimensional scene—SHSSA_B—sub-hologram segments are generated, in each case in the original and in the mirrored form.

(99) These sub-hologram segments are subsequently supplied to the sub-hologram segment addition SHSA. This in turn requires a hologram row memory HZS.

(100) The addition of the two hologram rows calculated in the preceding step finally takes place in the hologram row adder HZA, the result of which is a hologram row HZ, consisting of a plurality of added-up sub-holograms, i.e. a superposition of sub-holograms.

(101) This second alternative, improved solution variant has the advantage that only two hologram row memories are necessary, and that only two sub-hologram segments need to be added up. However, there is a time dependence, i.e. only continuously successive sub-hologram segments can be processed. The first solution variant described above, on the other hand, operates independently of the time sequence.

(102) As compared to a solution without symmetry according to the prior art, see FIG. 6, twice the amount of memory is necessary. However, the calculation is carried out twice as fast. For the same calculation speed in a solution without symmetry, two computational pipelines would need to be instantiated—in this case the amount of memory would then also be twice the amount.

(103) The additional logic complexity or the space requirement for mirroring, alignment and hologram row memory is significantly less than the requirement for a further conventional computational pipeline without symmetry, especially in the second solution variant of a method for encoding a hologram of a three-dimensional scene, which would be necessary to calculate the hologram in the same time. This means as a result twice the calculation speed with significantly less logic and a smaller space requirement. In the inversion of the argument, more sub-holograms SH can thus be calculated with unchanging space requirement.

(104) A pipeline for hardware-based real-time calculation of holograms with the aid of sub-holograms and direct controlling of a holographic display is illustrated in FIG. 11. It is realized on the basis of FPGA and/or ASIC and is described below.

(105) The calculation parameters for the hologram and the configuration parameters for the holographic display are stored in a non-volatile memory FLASH outside the FPGA or ASIC. A system controller SC (microcontroller or the like) within the FPGA or ASIC during the initialization phase loads the data from the memory FLASH and configures all modules of the holographic computational pipeline with parameters and tables. This data is stored within small local RAMs (random access memory) in the FPGA or ASIC. Furthermore, the modules which are necessary for the encoding and the outputting (formatting) according to the connected display are activated. It is furthermore determined in which format the content of the 3D scene to be represented is supplied and this is communicated to the content provider via a suitable interface, for example the DDC (display data channel) or CEC (consumer electronics control).

(106) The continuously arriving image frames, which contain color and depth information for various views, observer and transparency planes, are received by the source interface (data channel by which 3D contents, for example color and depth information, are received by a corresponding device, for example PC, games console, media player), for example HDMI (high definition media interface) or DVI (digital visual interface), filtered (input filter IF), decomposed into object point streams and stored in a RAM (input storage IS). By triple buffering, conflict-free and continuous reading and simultaneous writing with respect to the memory IS is made possible.

(107) The subsequent computational pipeline operates asynchronously therewith. The pipeline controller PC serves as an interface with the memory IS and controls the following parallel calculation instances (boxes which are illustrated one behind another in FIG. 11). It requests from the memory IS individual object points depending on currently calculated color/view/plane and current position in the hologram and distributes them to the individual instances depending on their workload. Each calculation instance consists of a 1D sub-hologram generator SHG, a converter module CVTF for converting complex values from the polar coordinate form into the Cartesian coordinate system, and an adder SHA, which adds the generated sub-hologram into the current hologram row. Each adder SHA comprises a hologram row buffer (not illustrated). A calculation instance typically processes all object points of the 3D scene to be illustrated, whose sub-holograms influence the currently stored hologram row in the adder SHA.

(108) Dynamic sub-hologram sizes are made possible by generating only a small segment of a sub-hologram (consisting for example of 8 complex values) per step of the calculation. This means that a calculation unit for large sub-holograms requires more steps than for small ones. Overall, this results in a high workload as compared to a realization of the calculation with static sub-hologram size, which on the other hand would be significantly more simple to realize and would also seem more obvious in an electronic circuit.

(109) The sub-hologram generator module SHG is also used in the calculation of 2D sub-holograms, wherein in this case the 2D sub-hologram is decomposed into a plurality of 1D sub-holograms.

(110) When one calculation instance finishes the calculation of a hologram row, this is passed on to the hologram memory HS and stored. By triple buffering, conflict-free and continuous reading and simultaneous writing with respect to the hologram memory HS is made possible. Along the way, the back-conversion of the hologram row into the polar coordinate form takes place in the transformation module CVTB and the pre-processing for the encoding in the display takes place in the module E1.

(111) The hologram output operates asynchronously therewith. The display controller DC controls the sequence, speed (frame rate) and formatting of the holograms to be output. The holograms can be output for example in a time-sequential manner with respect to the views and colors or interlaced. Mixed forms are likewise possible.

(112) Just before the row-wise transfer of the hologram to the display, the final encoding step is carried out in the module E2 to bring the complex hologram values into a form which can be represented on the display used. Thereafter, the hologram is output into the sink to the display (interface for transmitting the hologram data to a holographic 3D display or to the electronics for controlling one or more spatial light modulators (SLM)), wherein for example LVDS (low voltage differential signaling) can be used as the interface.

(113) The exemplary embodiment shown illustrates the fundamental possibility of implementing the calculation of holograms in the form of a fixed and efficient circuit and especially designing it in a configurable manner, such that it can be used for different display types and display sizes and application classes (desktop/mobile). It is characterized by real-time capability, high efficiency of the calculation sequences and its ability to use memory capacity sparingly, so as to consume as little energy as possible with as high a calculational power as possible. A further advantage is the high reusability and compatibility.

(114) Finally, a particular note should be made of the fact that the previously mentioned exemplary embodiments serve only to describe the claimed teaching, but do not limit it to the exemplary embodiments. In particular, the above-described exemplary embodiments could—as far as possible—be combined with one another.