High-resolution scanning microscopy

Abstract

A method for high-resolution scanning microscopy of a sample in which a sample is illuminated at a point in or on the sample by means of illumination radiation. The point is imaged along an optical axis and according to a point spread function into a diffraction image on a spatially resolving surface detector that comprises detector pixels in which a diffraction structure of the diffraction image is resolved. The point is displaced relative to the sample in at least two scanning directions and pixel signals are read from the detector pixels in various scanning posi- tions, wherein the pixel signals are respectively assigned to that scanning position at which they were read out and adjacent scanning positions overlap one another and are disposed according to a scanning increment. An image of the sample having a resolution that is increased beyond a resolution limit of the imaging is generated from the read pixel signals and the assigned scanning positions, wherein a deconvolution is carried out. Intermediate positions are generated for at least one of the scanning directions in the deconvolution on the basis of the pixel signals and the image of the sample, which contains more image points than scanning positions, is generated.

Claims

1. A method for high-resolution scanning microscopy of a sample comprising: a) illuminating the sample at a point in or on the sample by means of illu- mination radiation, b) imaging the point along an optical axis and according to a point spread function into a diffraction image on a spatially resolving surface detector that comprises detector pixels, wherein a diffraction structure of the diffraction image is resolved, c) displacing the point relative to the sample in at least two scanning directions and reading out pixel signals from the detector pixels in various scanning positions, wherein the pixel signals are respectively assigned to that scanning position at which they were read out and adjacent scanning positions overlap one another and are disposed according to a scanning increment, d) increasing resolution of an image of the sample beyond a resolution limit of the imaging being generated from the read pixel signals and the assigned scanning positions, wherein a deconvolution is carried out in step d) on the basis of the read pixel signals and the assigned scanning positions and on the basis of the point spread function, wherein intermediate positions are generated for at least one of the scanning directions in the deconvolution on the basis of the pixel signals and the image of the sample, which contains more image points than scanning positions, is generated.

2. The method as claimed in claim 1, wherein the scan- ning increment in step c) is greater than a full width at half maximum of the point spread function.

3. The method as claimed in claim 1, wherein, during the deconvolution, the pixel signals are Fourier transformed for each scanning position and the intermediate positions are generated in Fourier space.

4. The method as claimed in claim 1, wherein a linear deconvolution is carried out.

5. The method as claimed in claim 1, wherein the scanning direction comprises a z-direction aligned along the optical axis and the point is displaced relative to the sample in at least two scanning planes that are spaced apart in the z-direction and, in step d), a z-stack of images of the sample is generated, wherein the z-spacing in the stack is less than the scanning increment in the z-direction.

6. The method as claimed in claim 5, wherein the point spread function is not manipulated for the purposes of generating asymmetry and an absolute position with respect to the focal plane is ascertained by means of image information from the overlap region in the three-dimensional reconstruction.

7. The method as claimed in claim 1, wherein the point is displaced continuously in at least one of the scanning directions and readout times set scanning positions in this scanning direction.

8. A microscope for high-resolution scanning microscopy of a sample, comprising: an illumination beam path for illuminating a point on or in the sample, an imaging beam path for diffraction-limited imaging of the point along an optical axis into a diffraction image on a spatially resolving surface detector that has detector pixels, wherein the imaging beam path has a point spread function and the surface detector resolves the one diffraction structure of the diffraction image, a scanning device for displacing the point relative to the sample in at least two scanning directions, an evaluation device, which is connected in controlling fashion to the surface detector and the scanning device and which is configured to: (i) actuate the scanning device for displacing the point relative to the sample; (ii) read pixel signals from the detector pixels in respect of scanning position and to assign the pixel signals to the respective scanning position at which they were read, wherein adjacent scanning positions overlap and are disposed according to a scanning increment; and (iii) generate an image of the sample having a resolution that is increased beyond a resolution limit of the imaging beam path from the read pixel signals and the assigned scanning positions, wherein the evaluation device is further configured to carry out a deconvolution on the basis of the read pixel signals and the scanning positions belonging to the respective readout time and on the basis of the point spread function, wherein the evaluation device is configured to generate intermediate positions for at least one of the scanning directions in the deconvolution on the basis of the pixel signals and to generate the image of the sample, which contains more image points than scanning positions.

9. The microscope as claimed in claim 8, wherein the scanning increment is greater than a full width at half maximum of the point spread function.

10. The microscope as claimed in claim 8, wherein the evaluation device is configured, during the deconvolution, to Fourier transform the pixel signals for each scanning position and to generate the intermediate positions in Fourier space.

11. The microscope as claimed in claim 8, wherein the evaluation device is configured to carry out a linear deconvolution.

12. The microscope as claimed in claim 8, wherein the specified direction comprises a z-direction aligned along the optical axis and the evaluation device is configured to actuate the scanning device in such a way that the point is displaced relative to the sample in at least two scanning planes that are spaced apart in the z-direction and said evaluation device is further configured to generate a z-stack of images of the sample, wherein the z-spacing in the stack is less than the scanning incre- ment in the z-direction.

13. The microscope as claimed in claim 12, wherein the point spread function is not manipulated for the purposes of generating asymmetry and the evaluation device is configured to ascertain an absolute position with respect to the focal plane by means of image information from the overlap region in the three-di-mensional reconstruction.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

(1) The invention will be explained in even more detail below on the basis of exemplary embodiments, with reference being made to the attached drawings, which likewise disclose features essential to the invention. These exemplary embodiments serve merely for elucidation and should not be interpreted as restrictive. By way of example, a description of an exemplary embodiment with a multiplicity of elements or components should not be interpreted to the effect that all these elements or components are necessary for implementation purposes. Rather, other exemplary embodiments also may contain alternative elements and components, fewer elements or components or additional elements or components. Elements or components of different exemplary embodiments can be combined with one another, unless indicated otherwise. Modifications and variations which are described for one of the exemplary embodiments may also be applicable to other exemplary embodiments. In order to avoid repetition, the same elements or corresponding elements in different figures are denoted by the same reference signs and are not explained a number of times. In the figures:

(2) FIG. 1 is a schematic illustration o a microscope for high-resolution microscopy,

(3) FIG. 2 is a schematic illustration of detector pixels,

(4) FIG. 3 is a schematic illustration of a scanning pattern, and

(5) FIG. 4 is a flowchart for the microscopy method.

DETAILED DESCRIPTION OF EMBODIMENTS OF THE INVENTION

(6) FIG. 1 schematically illustrates a confocal microscope 20 with a high resolution, e.g., a resolution which is increased beyond the diffraction limit in accordance with the principle of so-called Airy scans, as is known, e.g., from EP 2317362 A1. It has a light source 6 for illuminating the sample P with an illumination spot 14. The illumination light B is guided via a beam shaping means 7, a mirror 8 to a beam splitter 9. The beam splitter 9 is embodied such that it reflects as much as possible of the illumination light B and guides it to a scanner 10. The illumination light B is guided, from the scanner 10, via further beam shaping optical units 11 and 12 to an objective 13. The objective 13 focuses the illumination light B onto the sample P to form an illumination spot, which is located in a focal plane.

(7) The sample light D generated by the sample in the illumination spot, i.e., at the point 14 is collected by the objective 13 and guided, on the reverse path with respect to the illumination light B, to the beam splitter 9. The beam splitter 9 is embodied such that it transmits as great a portion as possible of the sample light D. The sample light D thus transmitted by the beam splitter 9 is passed to the detector 17 via a further filter 15 and a further beam shaping optical unit 16. The detector 17 detects the sample light D, generates electrical signals therefrom and passes them on, via conductors 23, 24, 25, to a control and evaluation device C, e.g., a computer. In this way, a diffraction image 18 of the point 14 is recorded, which is diffraction-limited, as the diffraction structure 18a demonstrates. Mathematically, this is described by the point spread function (PSF).

(8) In order to obtain an image of the sample P, the point 14 is moved with the scanner 10 over the sample P and the detector 17 is read out in the process. From the signals obtained, an image which can be presented, e.g., using a monitor is compiled by the control and evaluation device C.

(9) The scanner 10 permits a two-dimensional displacement laterally, i.e., in a plane perpendicular to the optical axis of the objective. Moreover, the objective 13 is adjustable by way of a drive 26 in such a way that the position of the focal plane in the sample P is displaced.

(10) FIG. 2 schematically illustrates the detector 17 in a plan view. To achieve the high resolution, the detector 17 of the confocal microscope 20 comprises a plurality of detection elements or pixels 31. In exemplary fashion, the arrangement comprises twenty-one pixels 31; a different number of pixels may also be used. The size of the pixels 31 is chosen such that they are significantly smaller than the diffraction image 18 that is generated on the detector 17. Consequently, the diffraction structure 18a is resolved overall. At the same time, the number of the pixels 31, and consequently the entire surface of the detector 17, is chosen such that a significant portion of the sample light D can be detected for the diffraction image 18. For comparison, FIG. 2 indicates a detector area 30 in dashed fashion, as would be used for a confocal microscope with typical resolution. The term typical resolution is to be understood here to mean that the resolution achieved at most corresponds to the Abbe limit. By contrast, the confocal microscope 20 having increased resolution operates according to the Airy principle, and so theoretically twice as high a resolution can be achieved. In practice, the resolution increase is slightly lower, because structures near the resolution limit can be transmitted only with very poor contrast. Resolutions of realistically up to 1.7 times the Abbe limit can be achieved.

(11) The detector 17 of the confocal microscope 20 with high resolution captures P pixel signals for each scanned point, corresponding to the number of the detector pixels 31. The sample P is scanned by virtue of the point 17 being displaced over the sample P. This is elucidated in FIG. 3. The detector pixels 31 are read at successive readout times. Since the scanner 10 simultaneously displaces the point 14 over the sample, a scanning position 32 is present at each readout time (drawn as circles for simplicity in FIG. 3). In the case of a continuously advancing scanner in the x-direction, the time interval between the readout times sets a local scanning increment d together with the scanning speed, adjacent scanning positions 31 being disposed according to said scanning increment (e.g., spacing of the centers or other fixed reference points). This applies along a scanning axis, specifically the fastest scanning axis, of a raster-type scan. This is the x-direction in FIG. 3. In the case of a completely intermittently operating scanner, the readout clock is unimportant for the definition of the scanning position. For the up to two other scanning axes (y, z) of the raster-type scan, the scanner actuation alone sets the scanning increment d.

(12) In at least one certain direction (e.g., the z-direction), the scanning increment d is at least as large as the full width at half maximum of the PSF. Thus, this lower limit is directionally dependent. This is tantamount to half the resolution, i.e., half of the still resolvable structure, in the respective direction. On the other hand, the scanning increment d is not smaller than greater than twice the full width at half maximum of the PSF in the respective direction such that the scanning positions overlap. Consequently, the upper limit is also directionally dependent.

(13) Although each sample point is located at least once in each scanning pixel which is defined by the projection of the detector surface into the sample at the scanning position assigned to the scanning pixel—it is not located there at least twice, as required by the Nyquist theorem.

(14) As a pixel signal, each detector pixel 31 captures a raw image signal from the sample. The raw image signals differ from one another, wherein the differences are determined by the lateral distance of the point 14 relative to the sample region detected by the respective detector pixel. The raw image signals are described mathematically by a convolution of the “real” sample image with the PSF of the respective detector pixel 31.

(15) It is the function of the evaluation unit C to reconstruct from all pixel signals an image that as accurately as possible corresponds to the original of the sample. A deconvolution (DCV) and a subsequent joining of the thus deconvolved raw image signals are used to this end, wherein the processes deconvolution and joining can merge into one another in process-technological fashion.

(16) The procedure for generating an image is illustrated schematically in figure 4, which represents four basic steps S1-S4. In step S1, a scanning process of the sample is started, within the scope of which the point 14 is displaced over the sample surface along the x-direction. After one line was traversed in the x-direction, there is a displacement in the y-direction, as a line jump. Once a plurality of lines were worked through in this way, a second scanning plane is set, i.e., there is an adjustment in the z-direction. For reasons of simplicity, this is combined with the adjustment in the y-direction in FIG. 3. The displacement in the y-direction occurs in FIG. 3 if the representation is considered to be a plan view of the sample P. By contrast, the displacement in the z-direction appears if the representation in FIG. 3 is considered to be a sectional illustration perpendicular to the sample surface, i.e., along the optical axis of the radiation incidence.

(17) In step S2, the signal of the detector 17 is read for all detector pixels 31 continuously during the adjustment, introduced in step S1, at certain scanning positions that are stacked along a scanning increment, e.g., given by a continuous scanner displacement and equidistant readout times. By way of example, together with the scanning displacement process, each readout time sets one of the scanning positions 32. Then, the readout times are chosen in such a way that the scanning increment d set in FIG. 3 is reached in the x-direction together with the displacement speed.

(18) The scanning positions 32 overlap; however, the overlap is less than half the dimension of the scanning spot or the full width at half maximum of the point spread function.

(19) In step S3, the pixel signals of the detector pixels 31 are assigned to the scanning positions 32 and the scanning procedure is completed.

(20) Subsequently, an image of the sample P is generated in step S4 from the pixel signals with the associated scanning positions by means of a computational reconstruction, the PSF being taken into account.

(21) As already mentioned, the illumination beam path and the imaging beam path in one embodiment obtain no manipulation element to make the point spread function asymmetric in a targeted fashion; in particular, no astigmatic lenses or phase masks are provided. The term manipulation is here directed at a targeted influencing of the point spread function with which an asymmetry is generated that prevents, in particular in the 3D reconstruction, ambiguity between layers that are situated below the focal plane and layers that are situated above the focal plane. Consequently, targeted manipulation means that layers below the focal plane have a uniquely different point spread function than layers which are situated above the focal plane. Such manipulation typically requires the use of phase masks and/or astigmatic elements in the beam path.

(22) The maximum resolution is achieved in the illumination that is additionally diffraction-limited. This case is therefore portrayed in exemplary fashion for the purposes of explaining the reconstruction. Moreover, the resolution is increased in the z-direction in exemplary fashion.

(23) The signal D(r,p) detected overall by all pixels 31 can be considered to be a product of the excitation PSF E(r) and the intensity distribution O(r)of the sample light, convolved with the imaging PSF H(r):

(24) $\begin{array}{cc}D\left(r,p\right)={\int }_{r}O\left(p-r^{\prime }\right)E\left(r^{\prime }\right)H\left(r^{\prime }+r\right){\mathrm{dr}}^{\prime }& \left(1\right)\end{array}$

(25) In equation (1), p=(p.sub.x,p.sub.y,p.sub.z) denotes the location of excitation, i.e., on the sample, and r=(x,y,z) denotes the location of the detector pixel 31. On account of the surface detector, z=0 can be set in D since the surface detector only extends in x and y but not in z.

(26) Hence, equation (1) can be written as:

(27) $\begin{array}{cc}D\left(x,y,p_x,p_y,p_z\right)={\int }_x{\int }_y{\int }_{z}O\left(p_x-x^{\prime },p_y-y^{\prime },p_z-z^{\prime }\right)E\left(x^{\prime },y^{\prime }{z}^{\prime }\right)H\left(x+x^{\prime },y+y^{\prime },z^{\prime }\right){\mathrm{dx}}^{\prime }{\mathrm{dy}}^{\prime }{\mathrm{dz}}^{\prime }& \left(2\right)\end{array}$

(28) In the case of a conventional Airy scan evaluation, the summation was carried out in x and y over the coordinates of the surface detector, i.e., over the individual pixels of the surface detector. This is not implemented in this case. Instead, intermediate positions are generated on the basis of the pixel signals. Since the deconvolution operates in the Fourier space according to equation (2), these intermediate positions correspond to additional, higher frequencies in this case. Nevertheless, they are considered as intermediate positions in this case since they represent intermediate positions to the scanning positions when viewed in real space, even if they appear as frequencies in Fourier space. By way of example, intermediate positions are generated by means of a Dirac comb δ (cf.https://en.wikipedia.org/wiki/Dirac_comb) in x, y and z:

(29) $\begin{array}{cc}{D}_S\left(x,y,p_x,p_y,p_z\right)=\underset{k=-\infty }{\overset{+\infty }{.Math.}}D\left(x,y,p_x,p_y,p_z\right)\delta \left(p_z-k\Delta p_z\right)& \left(3\right)\end{array}$

(30) Here, Δp.sub.z represents the distance between the additional intermediate positions in z. A Fourier transformation of the equation (3) in respect of p.sub.x,p.sub.y,p.sub.z supplies

(31) $\begin{array}{cc}{D}_S^f\left(x,y,{\omega }_{p_x},{\omega }_{p_y},{\omega }_{p_z}\right)=\underset{k=-\infty }{\overset{+\infty }{.Math.}}{D}^f\left(x,y,{\omega }_{p_x},{\omega }_{p_y},{\omega }_{p_z}-\frac{k}{\Delta p_z}\right)& \left(4\right)\end{array}$

(32) The Fourier transform of the Dirac comb with the period Δp.sub.z, is a Dirac comb with the period 1/Δp.sub.z. If the Fourier-transformed equation (1) is inserted in equation (4), a general system of equations of the decomposition is obtained:

(33) $\begin{array}{cc}{D}_S^f\left(x,y,{\omega }_{p_x},{\omega }_{p_y},{\omega }_{p_z}\right)=\underset{k=-\infty }{\overset{+\infty }{.Math.}}{0}^f\left({\omega }_{p_x},{\omega }_{p_y},{\omega }_{p_z}-\frac{k}{\Delta p_z}\right){\mathrm{FT}}_{x^{\prime },y^{\prime },z^{\prime }}\left\{E\left(x^{\prime },y^{\prime },z^{\prime }\right)H\left(x+x^{\prime }+x^{\prime },y+y^{\prime },z^{\prime }\right)\right\}\left({\omega }_{p_x},{\omega }_{p_y},{\omega }_{p_z}-\frac{k}{\Delta p_z}\right)& \left(5\right)\end{array}$

(34) If FT.sub.x′,y′,z′{E(x′,y′,z′)H(x+x′,y+y′,z′)} is substituted by EH(x,y,ω.sub.p.sub.x,ω.sub.p.sub.y,ω.sub.p.sub.z), the following is obtained:

(35) $\begin{array}{cc}{D}_S^f\left(x,y,{\omega }_{p_x},{\omega }_{p_y},{\omega }_{p_z}\right)=\underset{k=-\infty }{\overset{+\infty }{.Math.}}{0}^f\left({\omega }_{p_x},{\omega }_{p_y},{\omega }_{p_z}-\frac{k}{\Delta p_z}\right)\mathrm{EH}\left(x,y,{\omega }_{p_x},{\omega }_{p_y},{\omega }_{p_z}-\frac{k}{\Delta p_z}\right)& \left(6\right)\end{array}$

(36) For reasons of simplicity, let x and y be the detector pixel position and let i be the pixel index:

(37) $\begin{array}{cc}{D}_{S_i}^f\left({\omega }_{p_x},{\omega }_{p_y},{\omega }_{p_z}\right)=\underset{k=-\infty }{\overset{+\infty }{.Math.}}{0}^f\left({\omega }_{p_x},{\omega }_{p_y},{\omega }_{p_z}-\frac{k}{\Delta p_z}\right){\mathrm{EH}}_i\left({\omega }_{p_x},{\omega }_{p_y},{\omega }_{p_z}-\frac{k}{\Delta p_z}\right)& \left(7\right)\end{array}$

(38) Due to taking account of the x, y-coordinates of the detector pixels, equation (7) increases the sampling in the object space in the z-direction in relation to the sampling by the measured scanning positions. The transformation is completely defined by the “confocal” Fourier transform of the PSF

(39) ${\mathrm{EH}}_i\left({\omega }_{p_x},{\omega }_{p_y},{\omega }_{{\mathrm{pp}}_z}-\frac{k}{\Delta p_2}\right).$
The z-dimension in the object space can be obtained, e.g., by a linear regression analysis in Fourier space. In the deconvolution, the aforementioned additional positions are generated due to the x, y-coordinates of the detector pixels being taken into account such that the image of the sample overall contains more pixels in the specifed direction (z-direction in this case) than there were scanning positions in this specified scanning direction.

(40) This procedure is not restricted to improving in z-direction but can be applied very generally to all three directions x, y and z. Here, use can then be made of a three-dimensional Dirac combs, which is as follows:

(41) $\begin{array}{cc}{D}_S\left(x,y,p_x,p_y,p_z\right)=\underset{k=-\infty }{\overset{\infty }{.Math.}}\underset{j=-\infty }{\overset{\infty }{.Math.}}\underset{i=-\infty }{\overset{\infty }{.Math.}}D\left(x,y,p_x,p_y,p_z\right)\delta \left(p_x-i\Delta p_x\right)\delta \left(p_y-j\Delta p_y\right)\delta \left(p_z-k\Delta p_z\right)& \left(8\right)\end{array}$

(42) Analogously to the calculation of equation (3), the Fourier transform of this equation then is:

(43) 0$\begin{array}{cc}{D}_S^f\left(x,y,{\omega }_{p_x},{\omega }_{p_y},{\omega }_{p_z}\right)=\underset{k=-\infty }{\overset{\infty }{.Math.}}\underset{j=-\infty }{\overset{\infty }{.Math.}}\underset{i=-\infty }{\overset{\infty }{.Math.}}{D}^f\left(x,y,{\omega }_{p_x}-\frac{i}{\Delta p_x},{\omega }_{p_y}-\frac{j}{\Delta p_y},{\omega }_{p_z}-\frac{k}{\Delta p_z}\right)& \left(9\right)\end{array}$

(44) If equation (9) is then inserted in equation (8), a set of equations for the deconvolution is obtained, which can be written as follows:

(45) $\begin{array}{cc}{D}_S^f\left(x,y,{\omega }_{p_x},{\omega }_{p_y},{\omega }_{p_z}\right)=\underset{k=-\infty }{\overset{\infty }{.Math.}}\underset{j=-\infty }{\overset{\infty }{.Math.}}\underset{i=-\infty }{\overset{\infty }{.Math.}}{O}^f\left({\omega }_{p_x}-\frac{i}{\Delta p_x},{\omega }_{p_y}-\frac{j}{\Delta p_y},{\omega }_{p_z}-\frac{k}{\Delta p_z}\right)\times {\mathrm{FT}}_{x^{\prime },y^{\prime },z^{\prime }}\left\{E\left(x^{\prime },y^{\prime },z^{\prime }\right)H\left(x+x^{\prime },y+y^{\prime },z^{\prime }\right)\right\}\left({\omega }_{p_x}-\frac{i}{\Delta p_x},{\omega }_{p_y}-\frac{j}{\Delta p_y},{\omega }_{p_z}-\frac{k}{\Delta p_z}\right)& \left(10\right)\end{array}$

(46) If only one direction is considered, the general system of equations simplifies to equation (7) (for the z-direction in that case). Naturally, a reduction to two directions is also possible in analogous fashion.

(47) What is common to all embodiments is that the individual pixel signals of the detector pixels are used to ascertain intermediate positions in at least one specified direction in the deconvolution and hence complement the scanning position and, ultimately, the scanning pixels with further intermediate positions. As a result, there are more pixels in the image than there were scanning pixels, i.e., scanning positions.

(48) While the invention has been illustrated and described in connection with currently preferred embodiments shown and described in detail, it is not intended to be limited to the details shown since various modifications and structural changes may be made without departing in any way from the spirit of the present invention. The embodiments were chosen and described in order to best explain the principles of the invention and practical application to thereby enable a person skilled in the art to best utilize the invention and various embodiments with various modifications as are suited to the particular use contemplated.