HIGH-RESOLUTION CONFOCAL MICROSCOPE
20210373306 · 2021-12-02
Assignee
Inventors
Cpc classification
G02B27/58
PHYSICS
G02B21/0072
PHYSICS
G02B21/008
PHYSICS
International classification
Abstract
A method for the high-resolution scanning microscopy of a specimen where the specimen is illuminated with illuminating radiation such that the illuminating radiation is focused to a diffraction-limited illuminating spot at a point in or on the specimen. The point is projected in a diffraction-limited manner in a diffraction image onto a flat panel detector having pixels. The flat panel detector, owing to the pixels thereof, have a spatial resolution which resolves a diffraction structure of the diffraction image. The point is shifted relative to the specimen into different scanning positions by an increment which is smaller than the diameter of the illuminating spot. The flat panel detector is read, and, from the data of the flat panel detector and from the scanning positions assigned to these data, a 3D image of the specimen is generated. The 3D image has a resolution which is greater than a resolution limit of the projection, and the pixels of the flat panel detector are divided into groups which have a central group lying on an optical axis and a further group which surrounds the central group in a ring. A pre-calculated raw image is calculated for each group and the pre-calculated raw images are unfolded three-dimensionally to generate the image of the specimen.
Claims
1-13. (canceled)
14. A method for high-resolution scanning microscopy of a sample, comprising: illuminating the sample with illumination radiation by focusing the illumination radiation to a point in or on the sample to form a diffraction-limited illumination spot comprising a diameter; imaging the point in diffraction-limited manner to a diffraction image comprising a diffraction structure onto a 2D detector, the detector having pixels and a spatial resolution that resolves the diffraction structure of the diffraction image, wherein imaging in diffraction-limited manner comprises a resolution limit displacing the point relative to the sample into different scanning positions with a scanning increment less than the diameter of the illumination spot reading data from the 2D detector and assigned an actual scanning position to the data for each of the scanning positions, and generating an image of the sample from the data and from the scanning positions assigned, wherein the generating step comprises: assigning each of the pixels of the 2D detector either to a central group located on an optical axis or to at least one further group that surrounds the central group in ring- shaped fashion, calculating a pre-processed raw image for each group, and combining and deconvoluting the pre-processed raw images to generate the image of the sample having a resolution that is increased beyond the resolution limit.
15. The method as claimed in claim 1, wherein the pre-processed raw images are deconvoluted in three dimensions and a 3D image of the sample is generated.
16. The method as claimed in claim 1, wherein the groups include at least one additional ring-shaped surrounding group, wherein each ring-shaped group has an individual distance from the central group.
17. The method as claimed in claim 1, wherein four or five groups are provided.
18. The method as claimed in claim 1, wherein each pixel is assigned to exactly one of the groups.
19. The method as claimed in claim 1, wherein the pre-processed raw images are calculated by virtue of a two-dimensional deconvolution being carried out for each group.
20. The method as claimed in claim 1, wherein relative displacements of raw image signals contained in the data are calculated, the raw image signals are compensated with respect to displacement and the compensated raw image signals belonging to the same group are added to calculate the pre-processed raw image for this group.
21. A microscope for high-resolution scanning microscopy of a sample, comprising: an illumination beam path for illuminating the sample with illumination radiation by focusing the illumination radiation to a point in or on the sample to form a diffraction-limited illumination spot having a diameter; an imaging beam path for diffraction-limited imaging of the point into a diffraction image image comprising a diffraction structure onto an 2D detector, the detector having pixels and a spatial resolution resolving the diffraction structure, wherein diffraction-limited imaging manner comprises a resolution limit a scanning device for displacing the point relative to the sample into different scanning positions with a scanning increment less than the diameter of the illumination spot, and an evaluation device configured to read data from the 2D detector and to assign an actual scanning position to the data for each scanning position, wherein the evaluation device is further configured to generate an image of the sample from the data and from the scanning positions assigned by assigning each of the pixels of the 2D detector either to a central group located on an optical axis or to at least one further group that surrounds the central group in ring-shaped fashion, calculating a pre-processed raw image for each group, and combining and deconvoluting the pre-processed raw images to generate the image of the sample having a resolution that is increased beyond the resolution limit.
22. The microscope as claimed in claim 21, wherein the evaluation device is configured to deconvolute the pre-processed raw images in three-dimensions in order to generate the image the sample.
23. The microscope as claimed in claim 21, wherein the groups include at least one additional ring-shaped surrounding group, wherein each ring-shaped group has an individual distance from the central group.
24. The microscope as claimed in claim 21, wherein four or five groups are provided.
25. The microscope as claimed in claim 21, wherein each pixel is assigned to exactly one of the groups.
26. The microscope as claimed in claim 21, wherein the evaluation device is further configured to calculate the pre-processed raw images by carrying out a two-dimensional deconvolution for the groups.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
[0017] The invention is explained in even more detail below on the basis of embodiments, with reference being made to the appended drawings, which likewise disclose features essential to the invention. These embodiments are only illustrative and should not be construed as restrictive. By way of example, a description of an embodiment with a multiplicity of elements or components should not be construed as meaning that all of these elements or components are necessary for implementation. Rather, other embodiments may also contain alternative elements and components, fewer elements or components or additional elements or components. Elements or components of different embodiments can be combined with one another, unless stated otherwise. Modifications and variations, which are described for one of the embodiments, can also be applicable to other embodiments. In order to avoid repetition, the same elements or corresponding elements in different figures are denoted by the same reference signs and are not explained a number of times. In the figures:
[0018]
[0019]
[0020]
[0021]
[0022]
[0023]
[0024]
[0025]
[0026]
[0027]
DETAILED DESCRIPTION OF THE INVENTION
[0028]
[0029] The sample light D produced by the sample in the illumination spot B is collected by the objective 13 and guided, on the reverse path with respect to the illumination light B, to the beam splitter 9. The beam splitter 9 is configured to transmit the greatest possible portion of the sample light D. The sample light D thus transmitted by the beam splitter 9 is passed to the detector 17 via a further filter 15 and a further beam shaping optical unit 16. The detector 17 detects the sample light D, generates electrical signals therefrom and passes them on, via conductors 23, 24, 25, to a control and evaluation device C, e.g., a computer. In this way, a diffraction image 18 is acquired which image is diffraction-limited, as the diffraction structure 18 a demonstrates.
[0030] In order to obtain an image of the sample P, the illumination spot 14 is moved in point-wise fashion over the sample P by the scanner 10. From the point-related sample signals thus obtained, the control and evaluation device C composes an image which can be presented, e.g., using a monitor. In this set-up, the scanner 10 allows to acquire a two-dimensional image which extends laterally, i.e., in a plane perpendicular to the optical axis of the objective. In order to acquire a three-dimensional image, the distance between the objective 26 and the sample P is varied and a two-dimensional image of the sample P is taken at each distance. The evaluation unit can combine the signals obtained thus to form a three-dimensional image.
[0031]
[0032] The size of the pixels 31 is chosen to have pixels significantly smaller than the diffraction image 18 that is generated on the detector 17. At the same time, the number of the pixels 31, and consequently the entire area of the detector 17, is chosen such that a substantial portion of the sample light D can be detected for the diffraction image 18.
[0033] For comparison purposes,
[0034] For each scanned point P ({right arrow over (r)}, z), the detector 17 of the confocal microscope 20 with high resolution captures many detected signals D.sub.h({right arrow over (r)}, z), corresponding to the number of the detector elements 31. {right arrow over (r)} denotes the lateral sample position, z denotes the axial sample position, the index h denotes the detector element (pixel). The further description assumes 32 pixels; however, a different number of pixels can also be used.
[0035] Each of the detector elements 31 captures a raw image signal from the sample P, consisting of the signals D.sub.h({right arrow over (r)}, z). The raw image signals differ from one another, the differences being determined by the lateral distance of the illumination light spot relative to the sample region detected by the respective detector element. The raw image signals are described mathematically by a convolution of the “real” sample image O({right arrow over (r)}, z) with the point spread function (PSF) I.sub.h({right arrow over (r)}, z) of the respective detector element h:
D.sub.h({right arrow over (r)}, z)=I.sub.h({right arrow over (r)}, z)*O({right arrow over (r)}, z)
[0036] It is the function of the evaluation unit C to combine all D.sub.h({right arrow over (r)}, z) into an image Õ({right arrow over (r)}, z) corresponding to the original of the sample O({right arrow over (r)}, z) as accurately as possible. This is achieved by a deconvolution (DCV) and a subsequent combination of the thus deconvolved raw image signals, wherein the processes of deconvolution and combination can merge into one another in terms of processing.
[0037]
[0038]
[0039] The dashed line 60 separates domains of the entire confocal microscope 20 between which the data of the raw image signals have to be transmitted. In this case, the domain LSM (laser scanning microscope) comprises hardware-related sectors of the confocal microscope, that is to say optics, mechanics and the electronics, which facilitate a data transfer to the PC domain. The domain PC constitutes the control and evaluation device C and consequently everything that is required for controlling the confocal microscope 20, for the further processing and for displaying the data. Usually, a personal computer C is the core of this domain. Specifically, the data of all raw image signals must be transferred from the LSM to the PC. This requires a powerful interface, and data transfer speed can be limited by an interface provided.
[0040]
[0041] Two aspects are important to understand the deconvolution formula in 50a:
[0042] 1. The raw image signals correspond to the sample which is convoluted with the system PSF; as already illustrated above:
[0043] 2. A convolution in real space corresponds to a multiplication in Fourier space. Thus, the raw image signals can be written in Fourier space as product of sample and OTF:
D.sub.h({right arrow over (k)}.sub.r, k.sub.z)=I.sub.h({right arrow over (l)}.sub.r, k.sub.z).Math.O({right arrow over (k)}.sub.r, k.sub.z)
[0044] If the formula for D.sub.h({right arrow over (k)}.sub.r, k.sub.z) is inserted into the deconvolution formula, one obtains:
[0045] I*.sub.h({right arrow over (k)}.sub.r, k.sub.z) denotes the OTF with conjugate-complex phase, the product with the (original) OTF then produces the square of the absolute value thereof.
[0046] w is a real positive number and shall be denoted as Wiener parameter in this case. If the Wiener parameter is small with respect to Σ.sub.h|I.sub.h({right arrow over (k)}.sub.r, k.sub.z)|.sup.2, the fraction approaches l and Õ({right arrow over (k)}.sub.r, k.sub.z) nearly corresponds to the original of the sample O({right arrow over (k)}.sub.r, k.sub.z). On the other hand, a w that is not too low ensures that the fraction does not diverge at points ({right arrow over (k)}.sub.r, k.sub.z) where Σ.sub.h|I.sub.h({right arrow over (k)}.sub.r, k.sub.z)|.sup.2 is small. Such divergences would increase noise, which is inevitably present in the raw image signals D.sub.h({right arrow over (k)}.sub.r, k.sub.z) and not illustrated here, beyond all measure and render the resulting image Õ({right arrow over (k)}.sub.r, k.sub.z) unusable.
[0047] In order to be able to display the image, the image Õ({right arrow over (k)}.sub.r, k.sub.z) which is given in spatial frequency coordinates must be transformed back into real space using inverse Fourier transform. Even though the actual deconvolution 50a is composed of comparatively simple computational steps (only additions and multiplications), the entire deconvolution process is complex. In particular, the Fourier transformation of all raw image signals D.sub.h({right arrow over (r)}, z) is computationally intensive and therefore time-consuming.
[0048] One alternative for processing of the raw image signals is shown in
[0049] However, this advantage of the processing route is offset by a serious disadvantage. Reason is, the pre-processing does not take into account any properties of the raw image signals with respect to z, leading to the resolution in the z direction being reduced to the level achieved in a conventional confocal microscope. This is illustrated in
[0050] In order to achieve the advantages of fast processing without having to accept the disadvantages of the lower z-resolution, the evaluation is now implemented in a special way, namely in two stages.
[0051] To this end,
[0052] The advantage of this two-stage evaluation is that the pre-processing of the images is considerably less computationally intensive, especially if it is carried out in two dimensions, i.e., does not generate any depth information—as a direct three-dimensional pre-processing according to
[0053]
[0054]
[0055]
[0056]
[0057] It should be noted that the differences between the two evaluation examples also depend on the parameters of the system. A total diameter of the detector area corresponding to 1.5 times the diameter of the Airy diffraction disk was used for all simulations. It was found that the differences are smaller for smaller diameters. Different forms of evaluation can be advantageous depending on the choice of parameters, and the provision of a plurality of forms of evaluation in one system, which are used depending on the choice of parameters, is also conceivable.
[0058] For comparison with the prior art, an MTF with an evaluation as per the prior art according to
[0059] This is not restricted to the specifically mentioned embodiments. Even on a detector with different matrix ring regions, e.g., square matrix ring regions, it is possible to identify pixels that have a sufficiently similar PSF in the z direction, and to pre-process these pixels together. The number of zones can also be varied, e.g., 2, 3 or 5, 6.
[0060] Likewise, different pre-processings and final processings are conceivable for as long these largely maintain the advantage of increased resolution. By way of example, use can also be made of iterative deconvolution processes, which avoid the specific disadvantages of a Wiener deconvolution. In individual cases, use can also be of a method that does not give quite as good results, i.e., a slightly lower MTF, but requires less computational outlay. Use can also be made of deconvolution methods that do not require any knowledge of the system PSF (so-called blind deconvolution).
[0061] The format in which the pre-processed raw images are transferred to the final processing is not decisive. By way of example, it is conceivable to perform the pre-processing in spatial frequency space while the pre-processed raw images are not transformed back into real space but transferred as spatial frequency data. This is particularly advantageous if the final processing is likewise done in spatial frequency space. In this case, the final processing only still has to transform the data with respect to z, but not with respect to the lateral coordinates. This results in further time savings.
[0062] The location at which the pre-processing and the final processing occur is not decisive for the method. Thus, all raw image signals can also initially be transmitted via line 141 and the two-stage evaluation can be carried out entirely on the PC. In that case, the advantage of lower/faster data transfer is eliminated, but there is still an acceleration due to the lower overall computational outlay.
[0063] While the invention has been illustrated and described in connection with currently preferred embodiments shown and described in detail, it is not intended to be limited to the details shown since various modifications and structural changes may be made without departing in any way from the spirit of the present invention. The embodiments were chosen and described in order to best explain the principles of the invention and practical application to thereby enable a person skilled in the art to best utilize the invention and various embodiments with various modifications as are suited to the particular use contemplated.