Method and apparatus for converting resolution of digital hologram data
10948875 ยท 2021-03-16
Assignee
Inventors
Cpc classification
G03H1/0866
PHYSICS
G03H2001/0825
PHYSICS
G03H2001/0088
PHYSICS
G03H1/0841
PHYSICS
G03H1/0443
PHYSICS
International classification
Abstract
Provided is a method for converting a hologram resolution of an apparatus for converting a hologram resolution. The apparatus for converting a hologram resolution includes receiving a hologram data and determining a direction and a height of an envelope for the hologram data based on first information associated with the hologram data. The apparatus for converting a hologram resolution includes converting the resolution of the hologram data from a first resolution into a second resolution based on the envelop having the determined direction and height.
Claims
1. A method for converting a hologram resolution of an apparatus for converting a hologram resolution, comprising: receiving hologram data having a first resolution; configuring a plurality of hologram vectors each having a real part and an imaginary part of the hologram data; and performing a resolution conversion by using a vector prediction method for predicting a new hologram vector using sizes and angles of the plurality of hologram vectors, wherein the new hologram vector is a center vector corresponding to a center pixel, and the plurality of hologram vectors are a plurality of adjacent vectors corresponding to a plurality of adjacent pixels surrounding the center pixel, wherein the vector prediction method obtains a first average value by averaging the sizes of only some of the plurality of adjacent vectors, obtains a second average value by averaging the angles of the some of the plurality of adjacent vectors, and obtains a real part and an imaginary part of the new hologram vector based on the first average value and the second average value.
2. The method of claim 1, wherein: the performing of the resolution conversion includes determining a hologram vector combination for predicting the new hologram vector among the plurality of hologram vectors.
3. The method of claim 2, wherein: the determining of the hologram vector combination includes determining a first hologram vector combination based on first information associated with the hologram data; applying a first resolution conversion based on the vector prediction method to which the first hologram vector combination is applied to the hologram data having the first resolution; and restoring a first image based on a numerical restoration for the hologram data to which the first resolution conversion is applied.
4. The method of claim 3, wherein: the determining of the hologram vector combination further includes restoring a second image based on a numerical restoration for the hologram data having the first resolution; and determining a second hologram vector combination based on a correlation between the first image and the second image.
5. The method of claim 1, wherein: the configuring of the plurality of hologram vectors includes converting a domain for the hologram data from a hologram domain into a frequency domain by applying a frequency conversion to the hologram data; and the performing of the resolution conversion includes applying the resolution conversion to each of partition regions included in the hologram data of the frequency domain, wherein the partition region is a region in which the resolution conversion after the frequency conversion is independently applied.
6. The method of claim 5, wherein: the performing of the resolution conversion further includes determining sizes of the partition regions based on first information associated with the hologram data.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
(1)
(2)
(3)
(4)
(5)
(6)
(7)
(8)
(9)
(10)
(11)
(12)
(13)
(14)
DETAILED DESCRIPTION OF THE EMBODIMENTS
(15) In the following detailed description, only certain exemplary embodiments of the present invention have been shown and described, simply by way of illustration. As those skilled in the art would realize, the described embodiments may be modified in various different ways, all without departing from the spirit or scope of the present invention. Accordingly, the drawings and description are to be regarded as illustrative in nature and not restrictive. Like reference numerals designate like elements throughout the specification.
(16) In the present specification, the overlapping description of the same components will be omitted.
(17) Further, in the present specification, it is to be understood that when one component is referred to as being connected to or coupled to another element, it may be connected directly to or coupled directly to another element or be connected to or coupled to another element, having the other element intervening therebetween. On the other hand, in the present specification, it is to be understood that when one element is referred to as being connected directly to or coupled directly to another element, it may be connected to or coupled to another element without the other element intervening therebetween.
(18) Further, terms used in the present specification are used only in order to describe specific exemplary embodiments rather than limiting the present invention.
(19) Further, in the present specification, singular forms may be intended to include plural forms unless the context clearly indicates otherwise.
(20) Further, in the present specification, it will be further understood that the terms include or have used in the present specification, specify the presence of features, numerals, steps, operations, components, parts mentioned in the present specification, or a combination thereof, but do not preclude the presence or addition of one or more other features, numerals, steps, operations, components, parts, or a combination thereof.
(21) Further, in the present specification, the term and/or includes a combination of a plurality of relevant items or any of a plurality of relevant items. In the present specification, A or B may include A, B, or A and B.
(22) A digital hologram data is represented by a complex value having values of a real part and an imaginary part. In addition, a spatial correlation of the digital hologram data has different features from a general 2D image. Accordingly, in the case of the resolution conversion for the digital hologram, a need exists for a new method for converting a resolution instead of the technologies used in the existing one dimensional (1D) or 2D. Hereinafter, a method and an apparatus for efficiently converting a digital hologram resolution according to characteristics of a digital hologram data will be described.
(23)
(24) As a technique which is often used for the resolution conversion of a 2D digital image, there are an interpolation technique or a sampling technique. Generally, an averaging based filter is mainly used during the interpolation process and the sampling process. This is basically based on the premise that the spatial correlation of the 2D data is high.
(25) However, the hologram has the spatial correlation of a form different from the existing 2D image.
(26)
(27) As illustrated in
(28)
(29) As illustrated in
(30)
(31) In detail,
(32) As illustrated in
(33)
(34) In detail,
(35) As the form of the signal illustrated in
(36) The hologram is a data that is represented by superposing information of light spreading like a wave, and therefore has a form in which several hologram information on one point illustrated in
(37) Therefore, considering the characteristics, the performance of the interpolation technique or the sampling technique may be improved. In this case, the periodic characteristics of the envelope are changed depending on a pixel pitch, a wavelength, a reproduction distance, or a resolution of the hologram. A structure of the envelope-based hologram resolution conversion will be described with reference to
(38)
(39) An apparatus 100 for converting a hologram resolution illustrated in
(40) The apparatus 100 for converting a hologram resolution may analyze hologram parameters (additional information or meta data) associated with the hologram data (S101). The hologram parameters (additional information or meta data) may include a resolution, a pixel pitch, a wavelength, a reproduction distance, or the like.
(41) The apparatus 100 for converting a hologram resolution may model the envelope more finely or more sparsely depending on the hologram parameters (additional information or meta data) (S102). Here, the envelope for the hologram data depends on the characteristics of the hologram data described with reference to
(42) In detail, the apparatus 100 for converting a hologram resolution may determine the direction and size of the envelope in consideration of the tendency of the envelope depending on the origin hologram data (S102). For example, when the apparatus 100 for converting a hologram resolution enlarges the hologram resolution twice, the direction (or sign) of the envelope for the data to be interpolated (or data to be filled by the interpolation) may be determined as an upward direction (convex upward) or a downward direction (convex downward).
(43) The apparatus 100 for converting a hologram resolution performs the resolution conversion depending on the determined envelope direction (S103) and generates the hologram (or hologram data) of which the resolution is converted (S104). For example, the apparatus 100 for converting a hologram resolution may also apply the resolution conversion to the entire hologram data or apply the resolution conversion only to some area of the hologram data.
(44) The apparatus 100 for converting a hologram resolution may determine improvement in quality (e.g., image quality, or the like) of the hologram of which the resolution is converted (S105) and determine whether the determined envelope direction is an appropriate direction. Here, the apparatus 100 for converting a hologram resolution may determine the value of the interpolation position (data to be interpolated) as a value other than an intermediate value of adjacent data. If it is determined that the determined direction is not the proper direction, the image quality may rather deteriorate, and therefore the apparatus 100 for converting a hologram resolution may determine a more similar envelope direction (e.g., envelope direction in which the correlation between numerically restored images is increased) as an appropriate direction in consideration of the correlation between the numerically restored images from the hologram of the origin resolution (e.g., resolution before the resolution conversion) and the numerically restored image from the hologram of which the resolution is converted. The apparatus 100 for converting a hologram resolution may repeatedly perform steps S102 to S105 to determine the optimum value for the envelope direction. The numerical restoration is a method for confirming a hologram data based image based on a computer simulation. The numerical restoration is differentiated from an optical restoration that forms the hologram data based image on an actual space.
(45) By the similar method, the apparatus 100 for converting a hologram resolution may determine the optimum value for the envelope size (e.g., envelope height). That is, the apparatus 100 for converting a hologram resolution may determine the optimum value for the envelope height by the method similar to the method for determining an optimum value for an envelope direction described above. Due to the characteristics of the hologram data, the intermediate value is highly likely to be a value larger or smaller that the peripheral value thereof. Therefore, the apparatus 100 for converting a hologram resolution may apply the resolution conversion based on candidate values for the envelope height only to some area of the entire hologram data, and determine the optimum value among the candidate values for the envelope height in consideration of the correlation between the numerically restored image result from the hologram of the origin resolution and the numerically restored image result from the hologram of which the resolution is converted. The apparatus 100 for converting a hologram resolution may repeatedly perform steps S102 to S105 to determine the optimum value for the envelope direction.
(46) The apparatus 100 for converting a hologram resolution converts the resolution of the hologram data from a first value into a second value by using the optimum value of the envelope direction and the envelope height (S106). For example, the apparatus 100 for converting a hologram resolution may also apply the resolution conversion to the entire hologram data or apply the resolution conversion only to some area of the entire hologram data.
(47) The apparatus 100 for converting a hologram resolution may finally output the hologram data of which the resolution is changed (S107). For example, the hologram data of which the resolution is changed may be finally represented in an input data form. The hologram data of which the resolution is changed may be finally transmitted to another apparatus.
(48)
(49) As described above, the hologram has characteristics closer to a frequency correlation than the spatial correlation of the 2D image. Therefore, if the 2D frequency conversion is applied to the hologram data, a data that is not the 2D image but has a spatial correlation higher than that of the hologram data may be acquired.
(50) In detail,
(51) As illustrated in
(52) Accordingly, if the resolution conversion for the hologram (or hologram data) is required, the apparatus 100 for converting a hologram resolution may perform the interpolation (or sampling) in the frequency domain in which the frequency conversion is applied to the hologram (or hologram data), not in the hologram domain, and apply an inverse conversion to the hologram domain according to the performance result of the interpolation (or sampling). As a result, it is possible to perform the effective resolution conversion on the hologram (or hologram data).
(53) At this time, the size of the partition region (PR) in which the characteristics similar to those of the 2D image are shown is changed depending on the pixel pitch, the wavelength, the reproduction distance, or the resolution of the hologram as illustrated in
(54)
(55) The apparatus 100 for converting a hologram resolution receives the hologram data (S200). Here, the hologram data is a data that includes all or some of complex field information on the hologram and may be a matrix data, a text data, or an image data.
(56) The apparatus 100 for converting a hologram resolution applies the frequency conversion to the hologram data to convert the domain for the hologram data from the hologram domain into the frequency domain (S201). The apparatus 100 for converting a hologram resolution may use a general frequency conversion method (e.g., Fourier transform, or the like) applied to the 2D image for the frequency domain conversion.
(57) The apparatus 100 for converting a hologram resolution may analyze hologram parameters (additional information or meta data) associated with the hologram data (S202). The hologram parameters (additional information or meta data) may include a resolution, a pixel pitch, a wavelength, a reproduction distance, or the like.
(58) The apparatus 100 for converting a hologram resolution may determine the size of the partition region (PR) in which the resolution conversion after the frequency domain conversion is independently applied, based on the hologram parameters (e.g., pixel pitch, wavelength, reproduction distance, resolution, or the like) or the combination of the hologram parameters and a changed scale (S203). The apparatus 100 for converting a hologram resolution may divide the partition regions (PRs) included in the hologram data of the frequency domain depending on the size.
(59) The apparatus 100 for converting a hologram resolution independently performs the hologram resolution conversion on each partition region (PR) (S204). In detail, to convert the resolutions of each of the partition regions (PRs) of the frequency domain, the apparatus 100 for converting a hologram resolution may use a general resolution conversion algorithm for the 2D image. Alternatively, to convert the resolutions of each of the partition regions (PRs) of the frequency domain, the apparatus 100 for converting a hologram resolution may also use the resolution conversion method (e.g., the resolution conversion method illustrated in
(60) The apparatus 100 for converting a hologram resolution again performs the inverse conversion on the domain for the hologram data of which the resolution is converted for each partition region (PR) from the frequency domain into the hologram domain (S205). In detail, the apparatus 100 for converting a hologram resolution may use a general frequency conversion method (e.g., Fourier inverse transform, or the like) applied to the 2D image for the frequency domain inverse conversion.
(61) The apparatus 100 for converting a hologram resolution may finally output the hologram data of which the resolution is changed (S206). For example, the hologram data of which the resolution is changed may be finally represented in an input data form. The hologram data of which the resolution is changed may be finally transmitted to another apparatus.
(62) Meanwhile, the apparatus 100 for converting a hologram resolution may determine the optimum value for the partition region (PR) by using the method similar to the method for determining an optimum value for an envelope direction and size described above in step (S203) of determining the sizes of each partition region (PR).
(63) For example, the apparatus 100 for converting a hologram resolution may apply the resolution conversion based on candidate values for the sizes of the respective partition regions (PRs) to some or all of the hologram data of the frequency domain and determine the optimum value among the candidate values for the sizes of the partition regions (PRs) in consideration of the correlation between the image result numerically restored from the hologram of the origin resolution and the image result numerically restored from the hologram of which the resolution is converted.
(64) That is, the apparatus 100 for converting a hologram resolution may determine the size of the partition regions (PRs) as the candidate value based on the hologram parameters (e.g., pixel pitch, wavelength, reproduction distance, resolution, or the like) and apply the resolution conversion based on the partition regions (PRs) to which the candidate values are applied to the hologram data. In addition, the apparatus 100 for converting a hologram resolution may restore the image by the numerical restoration for the hologram data to which the resolution conversion is applied. In addition, the apparatus 100 for converting a hologram resolution may restore the image by the numerical restoration for the origin hologram data. In addition, the apparatus 100 for converting a hologram resolution may change the size of the partition region (PR) based on the correlation between the restored images.
(65)
(66) As described above, the hologram (or hologram data) is a complex data. Accordingly, the hologram (or hologram data) may be represented by a vector having a real part and an imaginary part as axes, respectively..sub.0
(67) As illustrated in
(68)
(69) In detail,
(70) As illustrated in
(71) As a method for predicting a center vector from eight vectors representing eight pixels a1, a2, a3, a4, a6, a7, a8, and a9 around the center pixel a5, there is a method for obtaining an average of eight vectors.
(72) For example, there is a method (hereinafter, method M10) for predicting a new vector in consideration of a real part and an imaginary part of adjacent vectors. In detail, the method M10 may predict a new vector V1 by independently averaging the real part and the imaginary part of the adjacent vectors.
(73) The case where the method M10 uses the eight adjacent vectors a1 to a4 and a6 to a9 for the vector prediction is exemplified in the following Equation 1.
real(v1)={real(a1)+real(a2)+real(a3)+real(a4)+real(a6)+real(a7)+real(a8)+real(a9)}/8
imag(v1)={imag(a1)+imag(a2)+imag(a3)+imag(a4)+imag(a6)+imag(a7)+imag(a8)+imag(a9)}/8(Equation 1)
(74) In the above Equation 1, real ( ) represents the real part of the pixel (or vector) and imag ( ) represents the imaginary part of the pixel (or vector).
(75) However, as illustrated in
(76) Unlike this, there is a method (hereinafter, method M20) for predicting a new vector in consideration of sizes (or strength and amplitude) and phases (or angles) of adjacent vectors (or pixels). In detail, the method M20 may average the sizes (or strength, amplitude) and the phases (or angles) of the adjacent vectors to predict a new vector v2.
(77) The case where the method M20 uses the eight adjacent vectors a1 to a4 and a6 to a9 for the vector prediction is exemplified in the following Equation 2.
abs(v2)={abs(a1)+abs(a2)+abs(a3)+abs(a4)+abs(a6)+abs(a7)+abs(a8)+abs(a9)}/8
angle(v2)={angle(a1)+angle(a2)+angle(a3)+angle(a4)+angle(a6)+angle(a7)+angle(a8)+angle(a9)}/8
real(v2)=abs(v2)*cos(angle(v2)
imag(v2)=abs(v2)*sin(angle(v2))(Equation 2)
(78) In the following Equation 2, abs( ) represents the size of the pixel (or vector), angle( ) represents the angle (or phase) of the pixel (or vector), real( ) represents the real part of the pixel (or vector), and imag( ) represents the imaginary part of the pixel (or vector).
(79) As illustrated in
(80) However, in view of the accuracy, it is difficult to say that the vector v2 is the prediction vector for the vector a5.
(81) To improve the prediction performance (or prediction accuracy), the method M20 may use some of the adjacent vectors for the vector prediction. For example, as illustrated in
(82) The new vector v2 may be obtained based on the following Equation 3.
abs(v2)={abs(a1)+abs(a3)+abs(a7)+abs(a9)}/4
angle(v2)={angle(a1)+angle(a3)+angle(a7)+angle(a9)}/4
real(v2)=abs(v2)*cos(angle(v2))
imag(v2)=abs(v2)*sin(angle(v2))(Equation 3)
(83)
(84) If the above-mentioned vector-based prediction is performed in the frequency domain, not in the hologram domain, as illustrated in
(85) Even in the case of the similarity of the adjacent pixels, the vectors facing each other with respect to the center vector have higher similarity. For example, when a new vector is predicted based on some vectors b4 and b6 facing each other with respect to a vector b5 among the vectors b1 to b4 and b6 to b9, the similarity between the predicted vector and the vector b5 is high.
(86) Accordingly, if the apparatus 100 for converting a hologram resolution performs the resolution conversion on the hologram (or hologram data) to which the frequency conversion is applied by using the above-mentioned vector-based prediction, the resolution conversion performance may be more improved.
(87)
(88) In detail,
(89) In
(90) In
(91) The method for performing, by the apparatus 100 for converting a hologram resolution illustrated in
(92) The apparatus 100 for converting a hologram resolution receives the hologram data (S300).
(93) The apparatus 100 for converting a hologram resolution configures the hologram vectors using the hologram data (S307). For example, the apparatus 100 for converting a hologram resolution may configure the hologram vectors having the real part and the imaginary part of the hologram data as the axis component.
(94) The apparatus 100 for converting a hologram resolution performs the hologram resolution conversion based on the above-mentioned vector prediction (S308). In detail, the apparatus 100 for converting a hologram resolution may apply the resolution converter based on the method M10 or the resolution conversion based on the method M20 to the hologram data.
(95) The apparatus 100 for converting a hologram resolution may finally output the hologram data of which the resolution is changed (S306).
(96) The method for performing, by the apparatus 100 for converting a hologram resolution illustrated in
(97) The apparatus 100 for converting a hologram resolution receives the hologram data (S300).
(98) The apparatus 100 for converting a hologram resolution converts the domain for the hologram data for the hologram domain into the frequency domain (S301).
(99) The apparatus 100 for converting a hologram resolution may analyze the hologram parameters (e.g., resolution, pixel pitch, wavelength, reproduction distance, or the like) associated with the hologram data (S302).
(100) The apparatus 100 for converting a hologram resolution may determine the size of the partition region (PR) to which the resolution conversion is independently applied, based on the hologram parameters (e.g., pixel pitch, wavelength, reproduction distance, resolution, or the like) or the combination of the hologram parameters and the changed scale (S303). The apparatus 100 for converting a hologram resolution may divide the partition regions (PRs) included in the hologram data of the frequency domain depending on the size.
(101) The apparatus 100 for converting a hologram resolution independently applies the hologram resolution conversion based on the above-mentioned vector prediction to each partition region (PR) (S304). To this end, the apparatus 100 for converting a hologram resolution may configure the hologram vectors using the hologram data of the frequency domain. For example, the apparatus 100 for converting a hologram resolution may use the above-mentioned method M10 or M20 for the resolution conversion.
(102) The apparatus 100 for converting a hologram resolution again performs the inverse conversion on the domain for the hologram data of which the resolution is converted for each partition region (PR) from the frequency domain into the hologram domain (S305).
(103) The apparatus 100 for converting a hologram resolution may finally output the hologram data of which the resolution is changed (S306).
(104) Meanwhile, the apparatus 100 for converting a hologram resolution may determine which of adjacent vectors is used for the vector prediction based on a method similar to the method for determining optimum values for the envelope direction and size. For example, the apparatus 100 for converting a hologram resolution may apply the resolution conversion depending on the candidates of the vector combination used for the vector prediction only to some area of the entire hologram data, and determine the optimum vector combination among the candidates of the vector combination in consideration of the correlation between the numerically restored image result from the hologram of the origin resolution and the numerically restored image result from the hologram of which the resolution is converted.
(105) That is, the apparatus 100 for converting a hologram resolution may determine the candidates of the vector combination to be used for the prediction of the new vector based on the hologram parameters (for example, resolution, pixel pitch, wavelength, reproduction distance, or the like). Further, the apparatus 100 for converting a hologram resolution may apply the resolution conversion based on the vector prediction to which the candidates of the vector combination is applied to the origin hologram data. In addition, the apparatus 100 for converting a hologram resolution may restore the image by the numerical restoration for the hologram data to which the resolution conversion is applied. In addition, the apparatus 100 for converting a hologram resolution may restore the image by the numerical restoration for the origin hologram data. In addition, the apparatus 100 for converting a hologram resolution may change the candidates of the vector combination based on the correlation between the restored images.
(106)
(107) The apparatus 100 for converting a hologram resolution may analyze the hologram parameters (e.g., resolution, pixel pitch, wavelength, reproduction distance, or the like) associated with the hologram data (S400).
(108) The apparatus 100 for converting a hologram resolution may determine the candidate value of the information for the hologram resolution conversion based on the hologram parameters (e.g., resolution, pixel pitch, wavelength, reproduction distance, or the like) (S401). In detail, the information (hereinafter, first information) for the hologram resolution conversion may be the envelope direction and size illustrated in
(109) The apparatus 100 for converting a hologram resolution performs the resolution conversion depending on the candidate value determined in the step S401 (S402). For example, the apparatus 100 for converting a hologram resolution may also apply the resolution conversion to the entire origin hologram data or apply the resolution conversion only to some area of the origin hologram data.
(110) The apparatus 100 for converting a hologram resolution applies the numerical restoration to the hologram data (e.g., hologram data before the resolution conversion) of the origin resolution to restore the image (S403) and applies the numerical restoration to the hologram data of which the resolution is converted based on the step S402 to restore the image (S403).
(111) The apparatus 100 for converting a hologram resolution may analyze the correlation between the image result numerically restored from the hologram data of the origin resolution and the image result numerically restored from the hologram data of the converted resolution (S404) to change the value of the first information. That is, the apparatus 100 for converting a hologram resolution may repeatedly perform the steps S401 to S404 to derive the optimum value of the first information. For example, the apparatus 100 for converting a hologram resolution may determine the value having the highest (best) correlation analyzed in the step (S404) among the candidate values of the first information as the optimum value of the first information.
(112) The apparatus 100 for converting a hologram resolution may use the optimum value of the first information for the hologram resolution conversion (S405).
(113) Meanwhile, the method for deriving an optimum value illustrated in
(114)
(115) In the exemplary embodiment of
(116) The processor TN110 may run a program command that is stored in at least one of the memory TN130 and the storage device TN140. The processor TN110 may mean a central processing unit (CPU), a graphics processing unit (GPU), or an exclusive processor that performs the methods according to the exemplary embodiments of the present invention. The processor TN110 may be configured to implement the procedures, the functions, and the methods described with reference to the exemplary embodiment of the present invention. The processor TN110 may control each of the components of the computing apparatus TN100.
(117) The memory TN130 and the storage apparatus TN140 may each store various information associated with the operation of the processor TN110. The memory TN130 and the storage apparatus TN140 may each be configured of at least one of a volatile storage medium and a non-volatile storage medium. For example, the memory TN130 may be configured of at least one of a read only memory (ROM) and a random access memory (RAM).
(118) The transmitting/receiving apparatus TN120 may transmit or receive a wired signal or a wireless signal.
(119) The exemplary embodiments of the present invention are not implemented only by the apparatus and/or method as described above, but may be implemented by programs realizing the functions corresponding to the configuration of the exemplary embodiments of the present invention or a recording medium recorded with the programs, which may be readily implemented by a person having ordinary skill in the art to which the present invention pertains from the description of the foregoing exemplary embodiments.
(120) Although the exemplary embodiment of the present invention has been described in detail hereinabove, the scope of the present invention is not limited thereto. That is, several modifications and alterations made by those skilled in the art using a basic concept of the present invention as defined in the claims fall within the scope of the present invention.