Abstract
The invention relates firstly to a method for determining a mechanical deviation on a displacement path of an optical zoom lens, in particular on a displacement path of an optical zoom lens of a microscope. The optical zoom lens is arranged in a beam path between an object to be recorded and an electronic image sensor. In a first method step, an optical marker is introduced into the beam path at a position of the beam path located between the object to be recorded and the optical zoom lens, such that the optical marker passes the optical zoom lens and then is depicted on an image in which
a position of the optical marker is detected and determined. This is compared with a reference position of the optical marker in order to determine the mechanical deviation on the displacement path of the optical zoom lens. The invention further relates to a method for correction of a displacement error of an image recorded by an electronic image sensor and to an electronic image recording device.
Claims
1. A method for determining a mechanical deviation on a displacement path of a zoom optical unit, wherein the zoom optical unit is arranged in a beam path between an object to be recorded and an electronic image sensor, and wherein the method comprises the following steps: introducing an optical marking into the beam path at a position of the beam path that is situated between the object to be recorded and the zoom optical unit, as a result of which the optical marking passes the zoom optical unit and is subsequently imaged on an image, wherein an objective lens is arranged in the beam path between the object to be recorded and the position of the beam path at which the optical marking is introduced; detecting and determining a position of the optical marking in the image of the optical marking; and comparing the determined position of the detected optical marking with a reference position of the optical marking in order to determine the mechanical deviation on the displacement path of the zoom optical unit.
2. The method as claimed in claim 1, wherein the optical marking is formed by a light modified by an imageable mark or by a light beam.
3. A method for correcting a displacement error of an image recorded using an electronic image sensor, wherein the displacement error is caused by a mechanical deviation on a displacement path of a zoom optical unit, and wherein the method comprises the following steps: determining the mechanical deviation on the displacement path of the zoom optical unit using a method as claimed in claim 1; and selecting a region of the image to be corrected according to the mechanical deviation determined previously.
4. A method for correcting a displacement error of an image recorded using an electronic image sensor, wherein the displacement error is caused by a mechanical deviation on a displacement path of a zoom optical unit, and wherein the method comprises the following steps: determining the mechanical deviation on the displacement path of the zoom optical unit using a method as claimed in claim 2; and selecting a region of the image to be corrected according to the mechanical deviation determined previously.
5. The method as claimed in claim 3, wherein the mechanical deviation on the displacement path of the zoom optical unit is determined continuously while the object to be recorded is recorded.
6. An electronic image recording apparatus for recording an image of an object and for allowing a determination of a mechanical deviation on a displacement path of a zoom optical unit of the electronic image recording apparatus, comprising: an electronic image sensor; a zoom optical unit, which is arranged in a beam path between the object to be recorded and the electronic image sensor; and a marking means for introducing an optical marking into the beam path at a position of the beam path that is situated between the object to be recorded and the zoom optical unit; and an objective lens arranged in the beam path between the object to be recorded and the position of the beam path at which the optical marking is introduced.
7. The electronic image recording apparatus as claimed in claim 6, wherein the marking means comprises a partly transmissive splitter element, which is arranged at the position of the beam path that is situated between the object to be recorded and the zoom optical unit.
8. The electronic image recording apparatus as claimed in claim 7, wherein it comprises a reflected light illumination source, wherein the marking means comprises an imageable mark, which is arranged in a beam path between the reflected light illumination source and the partly transmissive splitter element.
9. The electronic image recording apparatus as claimed in claim 8, wherein the imageable mark is arranged in a stationary intermediate image plane.
10. The electronic image recording apparatus as claimed in claim 6, wherein the marking means comprises a laser light source which is directed on a partly transmissive splitter element that is arranged at the position of the beam path that is situated between the object to be recorded and the zoom optical unit.
11. The electronic image recording apparatus as claimed in claim 6, furthermore comprising a marking sensor for receiving light that has passed through the zoom optical unit and that includes the optical marking.
12. The electronic image recording apparatus as claimed in claim 6, wherein it comprises an image processing unit that is embodied to carry out a method as claimed in claim 1.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
[0059] Further details and developments of the invention will become apparent from the following description of preferred embodiments of the invention, with reference being made to the drawing. In the figures:
[0060] FIG. 1: shows an imaging beam path in a preferred embodiment of an image recording apparatus according to the invention;
[0061] FIG. 2: shows an image of an imageable mark as shown in FIG. 1, in the case of different image field dimensions;
[0062] FIG. 3: shows the image illustrated in FIG. 2 on an image sensor shown in FIG. 1;
[0063] FIG. 4: shows a first preferred embodiment of the image recording apparatus according to the invention;
[0064] FIG. 5: shows a second preferred embodiment of the image recording apparatus according to the invention;
[0065] FIG. 6: shows a third preferred embodiment of the image recording apparatus according to the invention;
[0066] FIG. 7: shows a fourth preferred embodiment of the image recording apparatus according to the invention;
[0067] FIG. 8: shows a fifth preferred embodiment of the image recording apparatus according to the invention;
[0068] FIG. 9: shows a sixth preferred embodiment of the image recording apparatus according to the invention;
[0069] FIG. 10: shows a seventh preferred embodiment of the image recording apparatus according to the invention;
[0070] FIG. 11: shows, in detail, an imageable mark shown in FIG. 10;
[0071] FIG. 12: shows an eighth preferred embodiment of the image recording apparatus according to the invention;
[0072] FIG. 13: shows a visualization of a first image recorded according to a first preferred embodiment of a method according to the invention;
[0073] FIG. 14: shows a visualization of a second image following the first image visualized in FIG. 13;
[0074] FIG. 15: shows a visualization of a first image recorded according to a second preferred embodiment of the method according to the invention;
[0075] FIG. 16: shows a visualization of a second image following the first image visualized in FIG. 15; and
[0076] FIG. 17: shows a visualization complementing the second image shown in FIG. 16.
DETAILED DESCRIPTION OF THE INVENTION
[0077] FIG. 1 shows an imaging beam path 01 in a preferred embodiment of an image recording apparatus according to the invention. In particular, the image recording apparatus is a microscope with an electronic image recording apparatus, of which an objective lens or tube lens 02, a zoom optical unit 03 and an image sensor 04 are illustrated. An imageable mark 07 is arranged in a stationary intermediate image plane 06 of the imaging beam path 01. The imageable mark 07 can be embodied as a plane parallel plate with a mark attached or a corresponding stop, which has the structure of the mark. The nature of the imageable mark 07 is such that, depending on a selected level of the zoom optical unit 03, the region to be resolved by the image sensor 04 lies outside of an image section (not shown) provided for an operator. Since the image region in the intermediate image plane 06 becomes smaller when zooming the zoom optical unit 03, the imageable mark 07 is preferably embodied as a wedge or as a triangle (shown in FIG. 2 and FIG. 3) such that the widest point of the wedge or of the triangle can just still be resolved at an edge of the image region.
[0078] The illustrated beam path 01 is produced to determine a mechanical deviation on a displacement path of the zoom optical unit 03 according to a preferred embodiment of a method according to the invention.
[0079] FIG. 2 shows an image 09 of the imageable mark 07, shown in FIG. 1, in the form of a wedge in the case of image fields of different sizes. A first image field 11, a second image field 12 and a third image field 13 have different sizes and lead to different sizes of the image 09 of the imageable mark 07 (shown in FIG. 1).
[0080] FIG. 3 shows the image 09, illustrated in FIG. 2, on the image sensor 04. The image sensor 04 comprises a multiplicity of pixels 15, wherein only the pixels 15 below an image field edge 16 record a visible image for the operator. The image 09 of the imageable mark 07 (shown in FIG. 1) can be removed by calculation by way of an interpolation of the surrounding pixels 15 such that the operator cannot perceive the image 09 of the imageable mark 07 (shown in FIG. 1). If the imageable mark 07 (shown in FIG. 1) is applied to a plane parallel plate, care should be taken that appropriate cleanliness values are observed. Contaminations and artifacts are imaged in focus on account of their position in the intermediate image and may influence the image even in the case of unsharp imaging. Preferably, the images of the contaminations and artifacts are removed by calculation by way of a calibration with a reference image.
[0081] FIG. 4 shows a first preferred embodiment of the image recording apparatus according to the invention, in the form of a microscope with an electronic image recording apparatus. Like the embodiment shown in FIG. 1, the microscope comprises the objective lens or the tube lens 02, the zoom optical unit 03 in the imaging beam path 01, the image sensor 04 and the imageable mark 07. The microscope furthermore comprises a reflected light illumination source 18 for reflected light illumination of an object 19 to be recorded, light of the reflected light illumination source 18 being coupled into the imaging beam path 01 via a reflected light mirror 21 and a partly transmissive reflected light splitter element 22. Moreover, light from the reflected light illumination source 18 is coupled into the imaging beam path 01 via a first partly transmissive splitter element 23 after said light has passed through a second partly transmissive splitter element 24, an optical unit 26 for creating an intermediate image plane and the imageable mark 07. The optical unit 26 for creating an intermediate image plane preferably comprises a filter (not illustrated) for reducing a beam intensity.
[0082] FIG. 5 shows a second preferred embodiment of the image recording apparatus according to the invention, which initially resembles the embodiment shown in FIG. 4. In contrast to the embodiment shown in FIG. 4, the light of the reflected light illumination source 18 is directed via the reflected light mirror 21 on the first partly transmissive splitter element 23, from where it partly is coupled into the imaging beam path 01 as reflected light and partly reaches a mirror 28. An optical unit 26 (shown in FIG. 4) for creating an intermediate image plane is preferably arranged on the mirror 28. The imageable mark 07 is situated in the intermediate image plane between the first partly transmissive splitter element 23 and the mirror 28. Alternatively preferably, the mirror 28 is arranged in the intermediate image plane and the imageable mark 07 is embodied as a coating on the mirror 28, for example.
[0083] FIG. 6 shows a third preferred embodiment of the image recording apparatus according to the invention, in the form of a microscope with an electronic image recording apparatus. Like the embodiment shown in FIG. 4, the microscope comprises the objective lens or the tube lens 02, the zoom optical unit 03, the image sensor 04, the imageable mark 07 and the first partly transmissive splitter element 23 in the imaging beam path 01. Furthermore, the microscope comprises a laser diode 30 which is directed on the first partly transmissive splitter element 23 such that a laser beam of the laser diode 30 reaches the image sensor 04 through the zoom optical unit 03.
[0084] FIG. 7 shows a fourth preferred embodiment of the image recording apparatus according to the invention, which initially resembles the embodiment shown in FIG. 6. In contrast to the embodiment shown in FIG. 6, a marking sensor 32 serves to receive the beam of the laser diode 30 that has passed through the zoom optical unit 03. To this end, the image recording apparatus comprises a third partly transmissive splitter element 31 for output coupling onto the marking sensor 32 the beam of the laser diode 30 which has passed through the zoom optical unit 03. In this embodiment, the wavelength of the light of the laser diode 30 lies outside of the wavelength range that can be detected by the image sensor 04. Consequently, the beam of the laser diode 30 is not visible to the operator. The marking sensor 32 is preferably formed by a camera sensor or a spectral sensor and it detects a shift of the beam of the laser diode 30 when adjusting the zoom optical unit 03. From this, the necessary displacement of the image region of the image sensor 04 displayed to the operator is calculated. By way of example, the image sensor 04 detects in a wavelength range between 325 nm and 1000 nm while the laser diode 30 produces light at a wavelength of 980 nm, for example. The marking sensor 32, which is sensitive in a wavelength range from 400 nm to 1000 nm, for example, can detect this light of the laser diode 30. The wavelength range up to 980 nm preferably removed by a filter (not shown) upstream of the marking sensor 32. Preferably, a further filter is arranged upstream of the image sensor 04, said filter absorbing light with a wavelength above 950 nm.
[0085] FIG. 8 shows a fifth preferred embodiment of the image recording apparatus according to the invention, which initially resembles the embodiment shown in FIG. 7. In contrast to the embodiment shown in FIG. 7, the image recording apparatus comprises a marking light source 34 instead of the laser diode, said marking light source being directed on the imageable mark 07 in the form of a periodic structure. The imageable mark 07 is situated in an input pupil and leads to a pupil manipulation which introduces a fixed point into the imaging beam path 01, the displacement of which is detectable. The marking sensor 32 is embodied to determine a phase of the brightness of the light incident thereon. The marking sensor 32 is preferably formed by a Bertrand system, by means of which the intensity of the pupil function can also be ascertained in addition to the intensity of the image. The phase is preferably determined from these two information items using the Gerchberg-Saxton algorithm. Likewise, use can be made of the phase-lift algorithm or the Wirtinger-Flow algorithm, for example. To this end, the marking light source 34 is embodied to produce coherent monochromatic light. Alternatively, the phase can be back-calculated by way of a digital phase gradient with angle-selective partially coherent illumination. Should the image sensor 04 be embodied to detect the phase, it can be used as an alternative to the marking sensor 32. In principle, the phase can be detected by applying the Fourier transform. The measured phase is a measure for the displacement. The imageable mark 07 is not imaged optically in focus and consequently not visible to the operator. A further alternative consists of recording a defocus stack (not shown) and determining the phase by way of the transport of intensity equation (TIE). This likewise requires coherent light and displaceability of the image sensor 04 in the z-direction. The path of the displacement of the image sensor 04 in the z-direction is sufficiently large to bring about defocusing.
[0086] FIG. 9 shows a sixth preferred embodiment of the image recording apparatus according to the invention, which initially resembles the embodiment shown in FIG. 7. In contrast to the embodiment shown in FIG. 7, the image recording apparatus comprises the marking light source 34 instead of the laser diode, said marking light source being directed on the imageable mark 07 in the form of a perforated stop provided with four pinholes 36. The four pinholes 36 have a regular arrangement and lead to a simple oscillation on the image sensor 04. This oscillation can easily be localized and is accordingly removed by calculation from the image of the image sensor 04. The light produced by the marking light source 34 and the imageable mark 07 with the four pinholes 36 is coherent.
[0087] FIG. 10 shows a seventh preferred embodiment of the image recording apparatus according to the invention, which initially resembles the embodiment shown in FIG. 9. In contrast to the embodiment shown in FIG. 9, the imageable mark 07 is formed by a switchable LCD shutter. The light of the marking light source 34 reaches the imageable mark 07 formed by the LCD shutter via a fourth partly transmissive splitter element 37 and two mirrors 38 that are arranged perpendicular to one another.
[0088] FIG. 11 shows the imageable mark 07 in the form of the LCD shutter, shown in FIG. 10, in detail. The imageable mark 07 in the form of the LCD shutter comprises a multiplicity of switchable elements 40. Switching an individual one of the switchable elements 40 leads to a pinhole 41. Preferably, the switchable elements 40 are switched individually in a regular pattern such that, for example, four of the pinholes 41 are provided in a matrix-shaped arrangement, leading to the same effect as the imageable mark 07, shown in FIG. 9, in the form of the perforated stop with the four pinholes 36. If there is a change in the zoom of the zoom optical unit 03 (shown in FIG. 10), four different switchable elements 40 can be switched, these elements having a distance from one another that deviates from that of the shown example.
[0089] FIG. 12 shows an eighth preferred embodiment of the image recording apparatus according to the invention, which initially resembles the embodiment shown in FIG. 9. In contrast to the embodiment shown in FIG. 9, the imageable mark 07 is formed by a digital micromirror device, which is illuminated by the marking light source 34 via the partly transmissive splitter element 23 and which reflects back into the imaging beam path 01. The imageable mark 07 formed by the digital micromirror device can be switched, for example like the imageable mark 07 in the form of the LCD shutter shown in FIG. 11, and so four light beams with coherent light in each case are directed into the zoom optical unit 03.
[0090] In order to ascertain the displacement on the image sensor 04, the distortion and the displacement of the pixels in the image of the image sensor 04 on account of the change in magnification by zooming the zoom optical unit 03 are initially removed by calculation. The Fourier transform is formed of each image recorded by the image sensor 04, said image containing the image of the object 19 to be recorded and the superimposed oscillation of the structure of the imageable mark 07. The frequencies of the oscillation are clearly identifiable in this Fourier transform. In frequency space, these frequencies have the coordinates ξ.sub.i, η.sub.i. If the recorded image is displaced in relation to the original image, there is a change in the phase Φ of the oscillation but the coordinate of the frequency does not change. Consequently, the phase φ can be uniquely ascertained in each recorded image. The phase φ emerges from a real and imaginary part of the Fourier transform, which span a vector on a unit circle, at these points:
FT{f(x−Δ.sub.2x,y−Δ.sub.2y)}=e.sup.−2πiΔ.sup.2x.sup.ξ.Math.e.sup.−2πiΔ.sup.2y.sup.η.Math.F{ξ,η}
[0091] In frequency space, the displacement on the image sensor 04 is defined by FT{f(x−Δ.sub.2x, y−Δ.sub.2y)}. The exponential terms define the phase. In order to obtain the displacement vector
[00001]
the quotient of the live Fourier coefficient e.sup.−2πiΔ.sup.2y.sup.ξ.Math.e.sup.−2πiΔ.sup.2y.sup.η F{ξ, η} and the respective Fourier coefficient F{ξ.sub.i, η.sub.i}, i=1, 2 is formed at two different points in the frequency space (ξ.sub.1, η.sub.1) and (ξ.sub.2, η.sub.2), and the phases φ.sub.1 and φ.sub.2 are determined. The Fourier coefficients F{ξ.sub.i, η.sub.i}, i=1, 2 are therefore known because they belong to two frequency coordinates (ξ.sub.i, η.sub.i), i=1, 2 of the at least two superimposed base frequencies, which form the superimposed oscillation. Consequently, a linear system of equations arises for the displacement:
[00002]
[0092] with the phase angles φ.sub.1 and φ.sub.2, and the frequency coordinates (ξ.sub.i, η.sub.i), i=1, 2.
[0093] The displacement
[00003]
arises by solving this system of equations. In order to obtain the correct value for the displacement, the frequency of the structure of the imageable mark 07 is chosen in such a way that a period is always greater than the maximum displacement of the images on the image sensor 04. By way of example, the distances in the structure of the imageable mark 07 should be modified accordingly. Moreover, the frequencies of the superimposed oscillation and the amplitude thereof should be adapted in such a way that they are disjoint to the typical spectrum of the actual image information. The image of the structure of the imageable mark 07 is subtracted from the image recorded by the image sensor 04 in a further step and the latter is displaced counter to the ascertained displacement such that the corrected image of the object 19 is displayed to the operator.
[0094] FIG. 13 visualizes a first image recorded according to a first preferred embodiment of a method according to the invention. This first image was recorded with the aid of a microscope (not shown) equipped with a zoom optical unit. When zooming with the zoom optical unit, the magnification factor thereof changes. The zoom optical unit has a mechanical deviation on its displacement path. The image is recorded by an electronic image sensor. In the image a point of interest (POI) of a recorded object is localized at a point P.sub.1, the latter being ascertained using methods of image recognition. Under the assumption that the magnification factor of the zoom optical unit is known continuously while zooming the zoom optical unit, this only one POI is enough to carry out the method according to the invention for determining or for correcting the mechanical deviation of the zoom optical unit. An invariant point I.sub.1 of the zoom optical unit is placed, as per definition and preceding calibration, into an image center of the first image.
[0095] FIG. 14 visualizes a second image following the first image visualized in FIG. 13, after the focal length of the zoom optical unit was changed. To this end, a component of the zoom optical unit was displaced along the displacement path aligned in a z-direction. In the second image, the POI is localized to a point P.sub.2ist, which is ascertained using methods of image recognition. This yields a difference in relation to the point P.sub.1 in the first image of (Δ.sub.Ges2x, Δ.sub.Ges2y). This difference comprises a displacement (Δ.sub.zoom2x, Δ.sub.zoom2y) caused by the change in the magnification factor and a displacement (Δ.sub.2x, Δ.sub.2y) caused by the statistical mechanical deviation. The change in the magnification factor represents a change in the imaging scale Δβ.sub.12, with Δβ.sub.12=1.5 applying in the shown example. Proceeding from the change in the imaging scale Δβ.sub.12 and the distance between P.sub.1 and I.sub.1, a point P.sub.2soll at which the POI would be imaged in the second image if there were no mechanical deviation of the zoom optical unit present is calculated according to the invention. The following applies in the example:
Δ.sub.Zoom_a2x=Δβ.sub.12.Math.d.sub.a1x
Δ.sub.Zoom_a2y=Δβ.sub.12.Math.d.sub.a1y
[0096] In general, the following applies:
Δ.sub.Zoom_ix=Δβ.sub.j.Math.d.sub.kx
Δ.sub.Zoom_iy=Δβ.sub.j.Math.d.sub.ky
[0097] The coordinates of the points P.sub.2soll and P.sub.2ist are compared. The difference in the x- and y-direction (Δ.sub.2x, Δ.sub.2y) represents a displacement error of the second image in relation to the first image, which is also exhibited by the invariant point that has been shifted between the two images. The second image is displaced counter to this displacement error before it is displayed to the operator, as a result of which the displacement error caused by the mechanical deviation is corrected. The same procedure likewise is carried out with further recorded images.
[0098] The magnification factor is ascertained proceeding from the real magnifications of all optical elements of the microscope and proceeding from movements of these optical elements in the z-direction. These information items are preferably ascertained when adjusting the microscope and stored in a storage medium.
[0099] FIG. 15 visualizes a first image of a microscope (not shown) equipped with a zoom optical unit, wherein this first image serves as a starting point to carry out a second preferred embodiment of the method, described with respect to FIG. 13 and FIG. 14, for determining or for correcting the mechanical deviation of the zoom optical unit. The magnification factor is assumed to be initially unknown in this second embodiment. Therefore, two POIs of the recorded object are selected, said POIs being localized in the first image at a point P.sub.a1 and at a point P.sub.b1, which are ascertained using methods of image recognition. An invariant point I.sub.1 of the zoom optical unit is placed, as per definition and preceding calibration, into an image center of the first image.
[0100] FIG. 16 visualizes a second image following the first image visualized in FIG. 15, after the focal length of the zoom optical unit was changed. To this end, a component of the zoom optical unit was displaced along the displacement path aligned in a z-direction. The change in the magnification factor represents a change in the imaging scale Δβ.sub.12, with Δβ.sub.12=1.5 applying in the shown example. The two POIs are localized in the second image at a point P.sub.a2Ist and at a point P.sub.b2Ist, which are ascertained using methods of image recognition. A straight line is produced through the points P.sub.a1 and P.sub.a2Ist and a straight line is produced through the points P.sub.b1 and P.sub.b2Ist. A point of intersection of these two straight lines is ascertained. The distance of this point of intersection in the x- and y-direction from the image center is multiplied by the reciprocal of the change in the imaging scale, from which one had previously been subtracted:
{right arrow over (Δ)}={right arrow over (Δ)}.sub.S.Math.(Δβ.sub.i−1)
[0101] A displacement error is ascertained proceeding therefrom, the former being corrected by displacement of the image. The change in the imaging scale Δβ.sub.12 is ascertained by equating the following vector equations:
{right arrow over (Δ′)}={right arrow over (v)}.sub.a12−{right arrow over (v)}.sub.a1.Math.(Δβ.sub.12−1)
{right arrow over (Δ′)}={right arrow over (v)}.sub.b12−{right arrow over (v)}.sub.b1.Math.(Δβ.sub.12−1)
[0102] Consequently, the following condition is used:
{right arrow over (v)}.sub.b12−{right arrow over (v)}.sub.a12=(Δβ.sub.12−1).Math.{right arrow over (v)}.sub.b1−{right arrow over (v)}.sub.a1
[0103] Complementing FIG. 16, FIG. 17 visualizes further method steps in relation to the second image. A vector {right arrow over (v)}.sub.12 and a vector {right arrow over (v)}.sub.1 are ascertained by way of vector addition:
{right arrow over (v)}.sub.b12−{right arrow over (v)}.sub.a12={right arrow over (v)}.sub.12
{right arrow over (v)}.sub.b1−{right arrow over (v)}.sub.a1={right arrow over (v)}.sub.1
[0104] The vectors {right arrow over (v)}.sub.12 and {right arrow over (v)}.sub.1 differ only in their magnitude by the factor (Δβ.sub.12−1). Accordingly, there is a back calculation to the change in the imaging scale. The calculation is preferably applied in each case to all recorded images and a subsequent image.
