Fluorescence-lifetime imaging microscopy method having time-correlated single-photon counting

Abstract

A fluorescence-lifetime imaging microscopy method with time-correlated single-photon counting includes using excitation light pulses separated in each case by a measurement interval to excite a sample to emit fluorescence photons. A detector signal that represents the captured fluorescence photons is generated. Detection times are determined based on the detector signal. Imaging is performed based on the detection times. The detection times of all captured fluorescence photons are compiled in a first data memory, common to a plurality of image pixels. The detection times of only those fluorescence photons which were captured in a predetermined number within the respective measurement intervals are compiled in a second data memory, common to the same plurality of image pixels. The detection times compiled in the data memories are combined within a calculation step. The results of the calculation step are stored in a third data memory.

Claims

1. A fluorescence-lifetime imaging microscopy method with time-correlated single-photon counting, the method comprising: using excitation light pulses of a pulsed light source to periodically excite a sample to emit fluorescence photons, with a measurement interval being defined between two successive excitation light pulses in each case, capturing, using a detector, the fluorescence photons, and generating a detector signal that represents the captured fluorescence photons, determining detection times at which the fluorescence photons are detected by the detector within the respective measurement intervals based on the detector signal, performing a calculation step, and performing imaging based on the detection times, wherein there is a determination within the respective measurement intervals as to whether a predetermined number of fluorescence photons have been captured within the respective measurement interval, wherein the detection times of all captured fluorescence photons are compiled in a first data memory, common to a plurality of image pixels, wherein the detection times of only those fluorescence photons which were captured in the predetermined number within the respective measurement intervals are compiled in a second data memory, common to the same plurality of image pixels, wherein the detection times compiled in the first data memory are combined with the detection times compiled in the second data memory within the calculation step, and wherein results of the calculation step are stored in a third data memory.

2. The fluorescence-lifetime imaging microscopy method as claimed in claim 1, wherein: the detection times compiled in the first data memory are ascertained in the form of a first histogram, which represents a non-corrected overall histogram, the detection times compiled in the second data memory are ascertained in the form of a second histogram, which represents a subset of the first histogram, and the results of the calculation step stored in the third data memory are ascertained in the form of a third histogram, which represents a corrected overall histogram.

3. The fluorescence-lifetime imaging microscopy method as claimed in claim 2, wherein the corrected overall histogram represents an ideal histogram, which would have been recorded without dead time.

4. The fluorescence-lifetime imaging microscopy method as claimed in claim 2, wherein columns of the first histogram are combined with columns of the second histogram, which are assigned to different detection times than the columns of the first histogram, within the calculation step.

5. The fluorescence-lifetime imaging microscopy method as claimed in claim 2, wherein an iterative refinement of the third histogram is undertaken within the calculation step.

6. The fluorescence-lifetime imaging microscopy method as claimed in claim 5, wherein the iterative refinement is implemented using the relationship: $f^{″}\left(t-t_d\right)=\frac{M_a\left(t\right).Math.M_1\left(t-t_d\right)}{P_0.Math.f^{\prime }\left(t\right)}.$

7. The fluorescence-lifetime imaging microscopy method as claimed in claim 1, wherein the predetermined number equals 1.

8. The fluorescence-lifetime imaging microscopy method as claimed in claim 1, wherein the detector signal is generated in the form of a data stream, in which a data word is inserted for each captured fluorescence photon.

9. The fluorescence-lifetime imaging microscopy method as claimed in claim 8, wherein a flag bit is provided in the data word, the flag bit specifying whether the respective fluorescence photon is part of the predetermined number of fluorescence photons which have been captured in the respective measurement interval.

10. The fluorescence-lifetime imaging microscopy method as claimed in claim 1, wherein a nonlinear optimization is performed within the calculation step.

11. The fluorescence-lifetime imaging microscopy method as claimed in claim 10, wherein the relationship:
M.sub.a(t).Math.M.sub.1(t−t.sub.d)=P.sub.0.Math.ƒ(t).Math.ƒ(t−t.sub.d) is used to perform the nonlinear optimization.

12. The fluorescence-lifetime imaging microscopy method as claimed in claim 1, wherein the results of the calculation step are visualized on an output device.

13. The fluorescence-lifetime imaging microscopy method as claimed in claim 1, wherein the results of the calculation step are used to determine fluorescence lifetimes.

14. A microscope for carrying out a fluorescence-lifetime imaging microscopy method with time-correlated single-photon counting, the microscope comprising: a light source configured to excite a sample with excitation light pulses to emit fluorescence photons, with a measurement interval being defined between two successive excitation light pulses in each case, a detector configured to capture the fluorescence photons and generate a detector signal representing the captured fluorescence photons, and a processor configured to: determine detection times at which the fluorescence photons are detected by the detector within the respective measurement intervals based on the detector signal and to perform imaging based on the detection times, determine within the respective measurement intervals whether a predetermined number of fluorescence photons have been captured within the respective measurement interval, compile the detection times of all captured fluorescence photons in a first data memory, common to a plurality of image pixels, compile the detection times of only those fluorescence photons which were captured in the predetermined number within the respective measurement intervals in a second data memory, common to the same plurality of image pixels, perform a calculation step in which the detection times compiled in the first data memory are combined with the detection times compiled in the second data memory, and store results of the calculation step in a third data memory.

15. The microscope as claimed in claim 14, wherein the microscope is a confocal scanning microscope or a multiphoton microscope.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

(1) Embodiments of the present invention will be described in even greater detail below based on the exemplary figures. The present invention is not limited to the exemplary embodiments. All features described and/or illustrated herein can be used alone or combined in different combinations in embodiments of the present invention. The features and advantages of various embodiments of the present invention will become apparent by reading the following detailed description with reference to the attached drawings which illustrate the following:

(2) FIG. 1 shows a confocal scanning microscope, which represents an exemplary embodiment of the microscope according to the invention,

(3) FIG. 2 shows the non-corrected overall histograms of a homogeneous sample and a non-homogeneous sample, for the purposes of elucidating the problem,

(4) FIG. 3 shows overall histograms with the associated dead time effects for all photons, for the photons from the measurement intervals in which only a single photon was detected, and a corrected overall histogram,

(5) FIG. 4 shows the sequence of the individual processing steps in a preferred embodiment of the method, and

(6) FIG. 5 shows the subunits in the processing unit of the microscope for a preferred embodiment.

DETAILED DESCRIPTION

(7) Embodiments of the invention provide a fluorescence lifetime imaging microscopy method and a microscope embodied to carry out such a method, which allow an increase in the excitation light intensity while avoiding a pile-up effect with comparatively little technical outlay and which are suitable for universal use of detector types and types of electronics. Another or a same one of the embodiments of the invention provides a method that facilitates a representation of the overall histogram in corrected form.

(8) An embodiment of the invention provides for a fluorescence-lifetime imaging microscopy method with time-correlated single-photon counting, in which excitation light pulses of a pulsed light source are used to periodically excite a sample to emit fluorescence photons, with a measurement interval being defined between two successive excitation light pulses in each case, a detector is used to capture the fluorescence photons and generate a detector signal that represents the captured fluorescence photons, detection times at which the fluorescence photons are detected by the detector within the respective measurement intervals are determined on the basis of the detector signal, and imaging is performed on the basis of the detection times of the captured fluorescence photons.

(9) According to an embodiment of the invention, there is a determination within the respective measurement interval as to whether a predetermined number of fluorescence photons have been captured within the measurement interval.

(10) For a plurality of image pixels, the detection times of all captured photons are compiled in a first data memory (or data region).

(11) For the same plurality of image pixels, the detection times of only those photons from measurement intervals with the predetermined number of captured photons are compiled in a second data memory.

(12) The information from the two data memories are combined in a calculation step and, in turn, stored in a data memory.

(13) The data in the data memory mentioned last can be displayed on an output device or can be used to determine lifetimes of the components of the molecules within the plurality of image pixels.

(14) The measurement interval, which is defined between two successive excitation light pulses in each case, is not necessarily the same as the time interval between these two pulses. Thus, it is also conceivable to let the measurement interval start and end a short time before or after the first of the two aforementioned excitation light pulses. By way of example, typical values of the time before and/or after the first of the two aforementioned excitation light pulses are 1% to 10% of the mean pulse duration of the excitation light or of the time interval between two respectively successive excitation light pulses.

(15) By way of example, the number of detected photons in a measurement interval can be determined by means of a counter which is reset (e.g., set to zero) when a excitation light pulse is identified and which is incremented with each detected photon. The ascertained detection times can be supplemented with information specifying the numerical rank of the respective photon within the measurement interval. Alternatively, an additional data word can be inserted into the data information stream when an excitation light pulse is identified. The corresponding operations for generating the information as to whether the predetermined number of photons has been captured in the measurement interval can be implemented in a logic circuit, in a freely programmable logic unit or in software.

(16) In a particularly simple embodiment, the predetermined number of captured photons equals one.

(17) The detection electronics of the system generates a data stream, with a data word with the detection time being inserted into the data stream for each identified photon. A flag bit is provided in the data word. By way of example, this bit receives a value of 1 if this relates to the first photon detected after the excitation light pulse; otherwise it receives the value of 0. The data stream is processed in a logic circuit, in a freely programmable logic unit or in software. Precisely when two successive data words are flagged, exactly one (1) photon was detected in the measurement interval from which the first data word originates.

(18) According to an embodiment of the invention, the detection times of all captured photons for a plurality of image pixels are compiled in a first data memory and the detection times of the photons from measurement intervals with the predetermined number of captured photons for the same plurality of image pixels are compiled in a second data memory and the information from the two data memories are combined using a calculation step. This plurality of image pixels can be all pixels of an image in this case, or else of pixels in an image region. Conventional image formats have pixels arranged in two dimensions, in rectangular fashion as a rule, wherein, e.g., 1024×1024 or 512×512 pixels are respectively provided in the x- and y-direction. Individual image regions (so-called regions of interest, ROIs) can have a rectangular, square or round region or a two-dimensional form that is freely definable by a user.

(19) In a preferred embodiment, the detection times are compiled by virtue of the frequency of detected photons for a detection time range being ascertained in the form of a histogram. Here, the histogram in the first data memory is the non-corrected overall histogram. Then, the histogram in the second data memory represents a subset of the non-corrected overall histogram in the first data memory. However, the detection times could also be compiled differently, e.g., by determining moments for the frequency of the detected photons as a function of the detection time.

(20) According to an embodiment of the invention, the information in the two data memories is combined using a calculation step. The non-corrected overall histogram, referred to as M.sub.a(t) below, is determined in a preferred embodiment. Additionally, the aforementioned subset is determined as histogram M.sub.1(t) with the number of photons from the measurement intervals in which only a single photon was detected. The histogram columns correspond to the respective detection time following the excitation light pulse. The intention is to ascertain the histogram ƒ(t) that would be recorded in the case of a recording using an ideal device without a dead time.

(21) A photon can only be detected at the time t if no photon was detected within the dead time before the time t. Therefore, the following applies to a column in the overall histogram M.sub.a(t) with the detection time t:
M.sub.a(t)=P.sub.0(t−t.sub.d,t).Math.ƒ(t)  (1)
where t.sub.d is the dead time of the system. P.sub.0(a,b) is the probability of no photon being detected in the time interval a to b. A similar relationship applies to M.sub.1 at the time t−t.sub.d:
M.sub.1(t−t.sub.d)=P.sub.0(t.sub.0,t−t.sub.d).Math.P.sub.0(t,t.sub.m).Math.ƒ(t−t.sub.d)  (2)

(22) It is only possible to detect exactly one photon at the time t−t.sub.d if no photon was detected beforehand or after the dead time.

(23) Multiplying equations (1) and (2) yields:
M.sub.a(t).Math.M.sub.1(t−t.sub.d)=P.sub.0(t.sub.0,t−t.sub.d).Math.P.sub.0(t−t.sub.d,t).Math.P.sub.0(t,t.sub.m).Math.ƒ(t).Math.ƒ(t−t.sub.d)  (3)

(24) Using:
P.sub.0=P.sub.0(t.sub.0,t−t.sub.d).Math.P.sub.0(t−t.sub.d,t).Math.P.sub.0(t,t.sub.m)  (4)
equation (3) can be simplified to read:
M.sub.a(t).Math.M.sub.1(t−t.sub.d)=P.sub.0.Math.ƒ(t).Math.ƒ(t−t.sub.d)  (5).

(25) Here, P.sub.0 is the probability of not detecting a photon throughout the entire measurement interval. P.sub.0 can be determined approximately from the total number of detected photons M.sub.a and the total number of measurement intervals L with the aid of the Poisson distribution function:
P.sub.0˜e.sup.−M.sup.a.sup./L  (6)

(26) A precise determination is possible if the number of measurement intervals L.sub.0 in which no photon was detected is additionally determined during the recording:

(27) $\begin{array}{cc}{P}_0=\frac{L_0}{L}.& \left(7\right)\end{array}$

(28) This determination is precise since dead time effects in the measurement intervals without detected photon are negligible, in contrast to measurement intervals with detected photons.

(29) The sought-after values ƒ(t) of the corrected histogram are now determined using a nonlinear optimization method. In a particularly simple embodiment, the method of least-squares approximation is carried out, which minimizes:

(30) $\begin{array}{cc}\underset{t=t_0+t_d}{\overset{t_m}{.Math.}}{\left\{\frac{M_a\left(t\right).Math.M_1\left(t-t_d\right)}{P_0}-f\left(t\right).Math.f\left(t-t_d\right)\right\}}^2& \left(8\right)\end{array}$

(31) A description of a nonlinear optimization method with least-squares approximation is contained in D. W. Marquardt, “An algorithm for least-squares estimation of nonlinear parameters,” J. Soc. Indust. Appl. Math., (1963). 11(2) 431-441. It is self-evident to a person skilled in the art that further optimization methods could also be used here, which likewise do justice to the statistical nature of the data.

(32) However, the combination of information from the two data memories using a calculation step need not necessarily be performed using an optimization method. By way of example, it could also be implemented with an iterative refinement of the histogram with successive calculation steps:

(33) $\begin{array}{cc}{f}^{″}\left(t-t_d\right)=\frac{M_a\left(t\right).Math.M_1\left(t-t_d\right)}{P_0.Math.f^{\prime }\left(t\right)}& \left(9\right)\end{array}$
and the initial values:
ƒ′(t)=M.sub.1(t)  (10)

(34) By way of example, the corrected overall histogram can be presented to the user on a computer monitor.

(35) Since the corrected overall histogram has comparable properties to a histogram at a lower excitation light intensity, it is possible to determine fluorescence lifetimes.

(36) A confocal scanning microscope 1, shown in FIG. 1, comprises a pulsed laser light source 2, which is embodied to emit light with periodic excitation light pulses. The excitation light denoted by 3 in FIG. 1 is incident on a beam splitter 4, which splits the excitation light 3 into a transmitted component 5t and a reflected component 5r.

(37) The excitation light 5t transmitted through the beam splitter 4 passes through an excitation pinhole 6 and is then reflected at a dichroic beam splitter 7 in the direction of a scanning unit 8. The scanning unit 8 contains a gimbal-mounted scanning mirror 9, which reflects the excitation light 5t in the direction of a scanning lens 10. Following the passage through the scanning lens 10 and a tube lens 11, the excitation light 5t reaches a microscope objective 12, which guides the excitation light 5t onto a sample 13.

(38) Fluorescent molecules are excited to emit fluorescence light 14 in the region of the sample 13 illuminated by the excitation light St. The fluorescence photons forming the fluorescence light 14 propagate along the light path, used by the excitation light 5t to reach the sample 13 from the dichroic beam splitter 7, in the reverse direction back to the beam splitter 7. After passing through the beam splitter 7 and a detection pinhole 15, the fluorescence light 14 then reaches a first detector 16. The first detector 16 converts the received fluorescence light 14 into a detector signal 17, the latter being fed to a processing unit 18 comprising a monitor M.

(39) In addition to the first detector 16, which converts the received fluorescence light 14 into the detector signal, the scanning microscope 1 comprises a second detector 19, which is arranged in the beam path split off by the beam splitter 4. Consequently, the second detector 19 receives the component 5r of the excitation light 3 emitted by the laser light source 2 that has been reflected by the beam splitter 7. The second detector 19 converts the excitation light 5r reflected at the beam splitter 4 into an excitation signal 20 and feeds the latter to the processing unit 18.

(40) In order to avoid the pile-up effect, as explained at the outset, the processing unit 18 is embodied according to the invention to determine, on the basis of the detector signal, whether a predetermined number of photons were identified within a measurement interval defined, e.g., by two successive excitation light pulses, to compile the detection times of all captured photons in a data memory, to compile the detection times of only those photons from measurement intervals with the predetermined number of captured photons in a second common data memory, to combine the information in the two data memories using a calculation step, and to store the results of this calculation step in a data memory.

(41) The processing unit 18 is further embodied to drive the scanning unit 9 in a manner known per se. Moreover, the processing unit 18 comprises a display device M, e.g., a monitor.

(42) FIG. 2 elucidates the problem solved by the calculation step for combining the information from the two memories. It shows the non-corrected overall histogram 22 from a recording of a homogeneous sample 21 and the non-corrected overall histogram 24 from a recording of an inhomogeneous sample 23, with the entire excitation light intensity, i.e., the total number of recorded photons, being approximately the same in both cases; however, clear intensity maxima are present locally in the image of the inhomogeneous sample 23. From the different form of the overall histograms according to FIG. 2, it is evident that the information in an overall histogram alone is not sufficient to be able to undertake a robust correction. Many methods according to the prior art cannot be applied to an overall histogram because the dead time effects depend on the concentrations of the different components at different locations. In the example according to FIG. 2, it is possible, in particular, to identify that no pile-up effect occurs in the recording of the homogeneous sample 21 since the excitation light intensity in each individual image pixel is (constantly) relatively low in this case. By contrast, a pile-up effect can be seen in the recording of the inhomogeneous sample 23 since this includes image regions whose pixels receive a comparatively high excitation light intensity and the component of the photons striking the detector within the dead time after the last detected photon increases with the excitation light intensity.

(43) FIG. 3 illustrates a measurement interval 25 with a measurement interval start 26 and a measurement interval end 27, a typical ideal overall histogram ƒ(t) (reference sign 28), to be determined according to the invention, and a non-corrected overall histogram M.sub.a(t) (reference sign 29) with a very high excitation light intensity.

(44) A further histogram M.sub.1(t) (reference sign 30) only contains the data of photons from those measurement intervals in which only a single photon was detected. A reduction of intensity in M.sub.a(t) (reference sign 29) as a result of the dead time effects with the dead time t.sub.d (reference sign 33) at a time t (reference sign 31) is always linked to an increase of intensity in M.sub.1(t) (reference sign 30) at the time t−t.sub.d (reference sign 32).

(45) The corrected overall histogram ƒ(t) can be ascertained from the variables M.sub.a(t) and M.sub.1(t), for example by applying the relationships (1) to (8) specified further above or the relationships (9) and (10).

(46) FIG. 4 illustrates a sequence of operations that can be carried out by the processing unit 18. Initially, a detection time for each identified photon is determined from the detector signals 17 and 20 by means of a time measurement between the signal of the photon 17 and the signal 20 of the excitation light pulse the detection time (not illustrated as a dedicated method step in FIG. 4). In a step 34, a sequence of data words is generated using the detection times of the photons, the data words being arranged in the sequence of incidence of the associated photon. In subsequent step 35, the data words in which no other photon was identified in the measurement interval before the associated photon are flagged. In step 36, the non-corrected overall histogram M.sub.a(t) then is generated from all data words. The overall histogram is stored in a first data memory. Subsequently, the histogram M.sub.1(t) is generated in step 37 from all the flagged data words whose subsequent data word is likewise flagged. This histogram M.sub.1(t) is stored in a second data memory. In step 38, the corrected overall histogram ƒ(t) is calculated from the histograms M.sub.a(t) and M.sub.1(t) in the two data memories, wherein a nonlinear optimization method minimizes the expression (8) already derived further above:

(47) $\underset{t=t_0+t_d}{\overset{t_m}{.Math.}}{\left\{\frac{M_a\left(t\right).Math.M_1\left(t-t_d\right)}{P_0}-f\left(t\right).Math.f\left(t-t_d\right)\right\}}^2$

(48) In so doing, the corrected overall histogram ƒ(t) is stored in a further data memory. The data from the memory specified last are presented on a monitor and used to determine fluorescence lifetimes.

(49) FIG. 5 illustrates constituent parts of the processing unit 18 of the microscope 1, which the method according to the invention use. In a time measurement unit 39, a data stream of data words with the detection times of the identified photons is generated from the detector signal 17 and the excitation signal 20. A flagging unit 40 flags data words whose associated photon has been captured as first photon in the sampling interval. In a first histogram generating unit 41, the non-corrected overall histogram M.sub.a(t) is calculated and stored in a first memory. The histogram M.sub.1(t) is generated in a second histogram generating unit 42 from the flagged data words whose subsequent data word is likewise flagged. The histogram M.sub.1(t) is stored in a second memory by the histogram generating unit 42. The data from the two memories are linked in a calculation unit 43, as a result of which the corrected overall histogram ƒ(t) arises, the latter being stored in a result data memory 44. The data from the result data memory 44 are presented on a monitor and used to determine fluorescence lifetimes.

(50) The aforementioned memories, in which the histograms M.sub.a(t), M.sub.1(t) and ƒ(t) are stored, are embodied, for example, as separate memory areas of a memory and which is part of the processing unit 18. However, such an embodiment should be understood as being purely exemplary. All that needs to be ensured is that the aforementioned histograms are kept available, in any way per se, in order to be processed as described above.

(51) While embodiments of the invention have been illustrated and described in detail in the drawings and foregoing description, such illustration and description are to be considered illustrative or exemplary and not restrictive. It will be understood that changes and modifications may be made by those of ordinary skill within the scope of the following claims. In particular, the present invention covers further embodiments with any combination of features from different embodiments described above and below. Additionally, statements made herein characterizing the invention refer to an embodiment of the invention and not necessarily all embodiments.

(52) The terms used in the claims should be construed to have the broadest reasonable interpretation consistent with the foregoing description. For example, the use of the article “a” or “the” in introducing an element should not be interpreted as being exclusive of a plurality of elements. Likewise, the recitation of “or” should be interpreted as being inclusive, such that the recitation of “A or B” is not exclusive of “A and B,” unless it is clear from the context or the foregoing description that only one of A and B is intended. Further, the recitation of “at least one of A, B and C” should be interpreted as one or more of a group of elements consisting of A, B and C, and should not be interpreted as requiring at least one of each of the listed elements A, B and C, regardless of whether A, B and C are related as categories or otherwise. Moreover, the recitation of “A, B and/or C” or “at least one of A, B or C” should be interpreted as including any singular entity from the listed elements, e.g., A, any subset from the listed elements, e.g., A and B, or the entire list of elements A, B and C.