Method to keep the excitation light sheet in focus in selective plane illumination microscopy
11262306 · 2022-03-01
Assignee
Inventors
Cpc classification
G02B21/367
PHYSICS
G02B21/16
PHYSICS
International classification
G02B21/36
PHYSICS
Abstract
Keeping the excitation light sheet in focus is critical in selective plane illumination microscopy (SPIM) to ensure its 3D imaging ability. Unfortunately, an effective method that can be used in SPIM on general biological specimens to find the axial position of the excitation light sheet and keep it in focus is barely available. Here, we present a method to solve the problem. We investigate its mechanism and demonstrate its performance on a lattice light sheet microscope.
Claims
1. A method to automatically focus a light sheet in a single plane illumination microscopy device comprising: illuminating a sample at a fixed plane using a patterned light sheet; obtaining an image stack comprising a plurality of images of an emission pattern; performing a 2D transform of the image stack at a plurality of image planes; determining which of the plurality of image planes contains a highest signal intensity at a modulation frequency; and identifying and outputting a light sheet axial position.
2. The method of claim 1, further comprising measuring an off focus distance/axial position of the patterned light sheet in Selective Plane Illumination Microscopy (SPIM).
3. The method of claim 1, wherein the patterned light sheet is created and used to illuminate the sample and a generated fluorescent emission pattern is imaged at a plurality of different planes in which a distance between an imaging objective and the light sheet varies, and information from the plurality of images is used to determine an off focus distance/axial position of the light sheet.
4. The method of claim 1, wherein a modulation depth of the plurality of images is compared to find the position of the light sheet.
5. The method of claim 4, wherein a maximum modulation depth is determined by comparing a signal intensity at the modulation frequency with a non-modulated Direct Current (DC) signal intensity in frequency space.
6. The method of claim 3, wherein the patterned light sheet is a one-dimensional array of illuminated beams.
7. The method of claim 3, wherein the patterned light sheet is a one-dimensional array of Bessel beams.
8. The method of claim 3, wherein the patterned light sheet is an optical lattice.
9. The method of claim 3, wherein the patterned light sheet is a two-dimensional array of illuminated beams or an optical lattice with components in two directions.
10. The method of claim 9, wherein the determining is based on a modulation depth in both directions.
11. A system to automatically focus a light sheet in a single plane illumination microscopy device comprising: a patterned light sheet that illuminates a sample at a fixed plane; an image processor and image capturing device that obtains an image stack comprising a plurality of images of an emission pattern; a processor and associated memory that: perform a 2D transform of the image stack at a plurality of image planes; determine which of the plurality of image planes contains a highest signal intensity at a modulation frequency; and identify and output a light sheet axial position.
12. The system of claim 11, further comprising measuring an off focus distance/axial position of the patterned light sheet in Selective Plane Illumination Microscopy (SPIM).
13. The system of claim 11, wherein the patterned light sheet is created and used to illuminate the sample and a generated fluorescent emission pattern is imaged at a plurality of different planes in which a distance between an imaging objective and the light sheet varies, and information from the plurality of images is used to determine an off focus distance/axial position of the light sheet.
14. The system of claim 11, wherein a modulation depth of the plurality of images is compared to find the position of the light sheet.
15. The system of claim 14, wherein a maximum modulation depth is determined by comparing a signal intensity at the modulation frequency with a non-modulated direct current (DC) signal intensity in frequency space.
16. The system of claim 13, wherein the patterned light sheet is a one-dimensional array of illuminated beams.
17. The system of claim 13, wherein the patterned light sheet is: a one-dimensional array of Bessel beams, an optical lattice, or a two-dimensional array of illuminated beams or an optical lattice with components in two directions.
18. The system of claim 17, wherein the determining is based on a modulation depth in both directions.
19. A system that corrects the off focus of the excitation light sheet in Selective Plane Illumination Microscopy (SPIM) by controlling a position offset of one or more of an excitation light sheet and a detection objective based on a acquired light sheet off focus distance determined by: illuminating a sample at a fixed plane using a patterned light sheet; obtaining an image stack comprising a plurality of images of an emission pattern; performing a 2D transform of the image stack at a plurality of image planes; determining which of the plurality of image planes contains a highest signal intensity at a modulation frequency; and identifying and outputting a light sheet axial position.
20. A non-transitory computer-readable information storage media having stored thereon instructions, that when executed by one or more processors, cause to be performed a method to automatically focus a light sheet in a single plane illumination microscopy device comprising: illuminating a sample at a fixed plane using a patterned light sheet; obtaining an image stack comprising a plurality of images of an emission pattern; performing a 2D transform of the image stack at a plurality of image planes; determining which of the plurality of image planes contains a highest signal intensity at a modulation frequency; and identifying and outputting a light sheet axial position.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
(1) The exemplary embodiments of the invention will be described in detail, with reference to the following figures wherein:
(2)
(3)
(4)
(5)
(6)
(7)
DETAILED DESCRIPTION
(8)
(9) In particular, control begins in step S100 and continues to step S110. In step S110, a sample is illuminated at a fixed plane using a patterned light sheet. A patterned light sheet is parked at a fixed sample plane, while a 3D widefield image stack of the fluorescence emission pattern is collected and used to find the axial position of the excitation light sheet. More particularly, in step S120, a widefield 3D image stack of the fluorescent emission pattern is taken and stored. Next, in step S130, a 2D Fourier transform of the 3D image stack at all image planes is performed by a computing device and associated image processing software, storage/memory (not shown). Control then continues to step S140.
(10) In step S140, the light sheet axial position is identified by finding the plane which contains the highest signal intensity at the modulation frequency. Next, in step S150, the offset position of the galvanometer is reset with control continuing to step S160 where the control sequence ends.
(11)
(12)
(13)
(14) The theory behind the method can be understood as follows. The patterned light sheet creates a fluorescence emission pattern that contains a specific spatial frequency in the modulation direction, in which the relative intensity of the modulated signal at the modulation frequency to the non-modulated signal is mainly determined by the illumination pattern. During the acquiring process of the 3D widefield image stack of the fluorescence emission pattern, the emission pattern is focused and defocused as the detection focal plane is scanned across the excited sample plane. Because the high frequency signal attenuates faster than the low frequency signal in widefield detection when the emission pattern becomes off focus, the ratio between the modulated signal and the non-modulated signal is the maximum when the light sheet is in focus.
(15) The collected 3D image stack is the convolution of the microscope PSF (Point Spread Function) and the produced fluorescence emission pattern. The relationship can be expressed as D(r)=H(r).Math.E(r), in which D(r) is the recorded image, H(r) is the PSF (Point Spread Function) of the microscope, E(r) is the fluorescence emission pattern, and r denotes the spatial coordinates (x, y, z) in real space. Meanwhile, the fluorescence emission pattern can be described as E(r)=S(r)I(r), where S(r) represents the fluorescent labeled specimen, and I(r) is the intensity profile of the patterned light sheet parked on the specimen.
(16) In SPIM, the intensity profile of a patterned light sheet can be described as the sum of multiple modulation harmonics:
(17)
where I.sub.m(z), f.sub.m and φ.sub.m are the intensity, frequency and phase of each harmonic, respectively, which are determined by how the patterned light sheet is generated. By substituting I(r) with Eq. 1, it can be obtained that
(18)
Therefore, the 2D Fourier transform of each image plane takes the form
(19)
In which O(k.sub.xy,z) is the optical transfer function of H(r.sub.xy,z), and k.sub.xy defines the lateral coordinate in frequency space. As a result, the ratio between the modulated DC component by the m.sup.th harmonic and non-modulated DC component at each image plane is given by
(20)
(21) As long as the sample structure is roughly uniform, the equation can be approximately simplified as
(22)
Obviously, the ratio R(z) reaches the maximum when the patterned light sheet is in focus, because O(0,0,z) is a constant, and both O(f.sub.m,0,z) and I.sub.m(z) are the maximal at z=0.
(23) The performance of the exemplary method has been evaluated by imaging fixed HeLa cells labeled with Vybrant CM-Dil on a lattice light sheet microscope. In lattice light sheet microscopy, a coherent Bessel beam array, which contains several modulation harmonics, is either dithered to illuminate the sample in the sheet scan mode, or scanned in discrete steps to illuminate the sample in the structured illumination mode.
(24) At first, a coherent Bessel beam array generated with a 0.55 NA.sub.od, 0.5 NA.sub.id annulus was parked at an arbitrary plane of the selected cell. The excitation objective was scanned for a 20 μm distance at a 150 nm step size to collect a 3D image stack.
(25) Next, the long term focusing stability of the microscope on the same cell was determined by repeating the same procedure described above every 10 minutes for ˜12 hours. The results show that the light sheet drifted about ˜2.5 μm according to the detection focal plane during the monitoring period (
(26) One exemplary embodiment provides a method to find the axial position of the excitation light sheet in SPIM on general biological specimens so that a correction (for example via feedback) can be applied based on the result to keep the light sheet in focus. Description has been provided of a mechanism for the method and confirmed its ability on a lattice light sheet microscope by imaging HeLa cells labeling cell membrane. The exemplary method has several major advantages. First, the method can be software based and it can be implemented in an automatic process easily. Second, it can be applied to biological specimens without requiring special features, so that the method is applicable on most biological specimens. Third, it only requires dozens of 2D images to locate the axial position of the excitation light sheet, which isn't a big problem as the excitation light is highly confined in SPIM. It not only returns the result in seconds but also avoids the waste of the limited photon budget.
(27) Besides the lattice light sheet microscopy technique used to demonstrate the method, the method can be implemented in any other SPIM technique that is capable of creating a patterned light sheet, such as the SPIM techniques that create the excitation light sheet using a scanning beam or beam array (Keller et al., 2008, 2010; Planchon et al., 2011; Gao et al., 2012). Additionally, although the modulation frequency of the patterned light sheet is not critical, the method works better when the modulation frequency is half of that of the detection lateral diffraction limit, at which the widefield detection PSF has the best axial resolution, so that the variation of the modulation signal intensity is the most sensitive to the defocusing of the patterned light sheet. Finally, the described method assumes that the labeled sample structure is roughly uniform. Therefore, it could fail when the sample is only sparsely labeled, e.g., the sample is a fluorescent particle. Fortunately, such situation is rare in most biological specimens, and other methods developed previously can be used instead to find the excitation light sheet.
(28) Exemplary aspects are directed toward:
(29) A method to automatically focus a light sheet in a single plane illumination microscopy device comprising:
(30) illuminating a sample at a fixed plane using a patterned light sheet;
(31) obtaining an image stack comprising a plurality of images of an emission pattern;
(32) performing a 2D transform of the image stack at a plurality of image planes;
(33) determining which of the plurality of image planes contains a highest signal intensity at a modulation frequency; and
(34) identifying and outputting a light sheet axial position.
(35) Any one or more of the above aspects, further comprising measuring an off focus distance/axial position of the patterned light sheet in SPIM.
(36) Any one or more of the above aspects, wherein the patterned light sheet is created and used to illuminate the sample and a generated fluorescent emission pattern is imaged at a plurality of different planes in which the distance between an imaging objective and the light sheet varies, and information from the plurality of images is used to determine the off focus distance/axial position of the light sheet.
(37) Any one or more of the above aspects, wherein a modulation depth of the plurality of images is compared to find the position of the light sheet.
(38) Any one or more of the above aspects, wherein a maximum modulation depth is determined by comparing a signal intensity at the modulation frequency with a non-modulated DC signal intensity in frequency space.
(39) Any one or more of the above aspects, wherein the patterned light sheet is a one-dimensional array of illuminated beams.
(40) Any one or more of the above aspects, wherein the patterned light sheet is a one-dimensional array of Bessel beams.
(41) Any one or more of the above aspects, wherein the patterned light sheet is an optical lattice.
(42) Any one or more of the above aspects, wherein the patterned light sheet is a two-dimensional array of illuminated beams or an optical lattice with components in two directions.
(43) Any one or more of the above aspects, wherein the determining is based on a modulation depth in both directions.
(44) A system to automatically focus a light sheet in a single plane illumination microscopy device comprising:
(45) a patterned light sheet that illuminates a sample at a fixed plane;
(46) an image processor and image capturing device that obtains an image stack comprising a plurality of images of an emission pattern;
(47) a processor and associated memory that: perform a 2D transform of the image stack at a plurality of image planes; determine which of the plurality of image planes contains a highest signal intensity at a modulation frequency; and identify and output a light sheet axial position.
(48) Any one or more of the above aspects, further comprising measuring an off focus distance/axial position of the patterned light sheet in SPIM.
(49) Any one or more of the above aspects, wherein the patterned light sheet is created and used to illuminate the sample and a generated fluorescent emission pattern is imaged at a plurality of different planes in which the distance between an imaging objective and the light sheet varies, and information from the plurality of images is used to determine the off focus distance/axial position of the light sheet.
(50) Any one or more of the above aspects, wherein a modulation depth of the plurality of images is compared to find the position of the light sheet.
(51) Any one or more of the above aspects, wherein a maximum modulation depth is determined by comparing a signal intensity at the modulation frequency with a non-modulated DC signal intensity in frequency space.
(52) Any one or more of the above aspects, wherein the patterned light sheet is a one-dimensional array of illuminated beams.
(53) Any one or more of the above aspects, wherein the patterned light sheet is:
(54) a one-dimensional array of Bessel beams,
(55) an optical lattice, or
(56) a two-dimensional array of illuminated beams or an optical lattice with components in two directions.
(57) Any one or more of the above aspects, wherein the determining is based on a modulation depth in both directions.
(58) A system that corrects the off focus of the excitation light sheet in SPIM by controlling a position offset of either or both of an excitation light sheet and/or a detection objective based on a acquired light sheet off focus distance determined by:
(59) illuminating a sample at a fixed plane using a patterned light sheet;
(60) obtaining an image stack comprising a plurality of images of an emission pattern;
(61) performing a 2D transform of the image stack at a plurality of image planes;
(62) determining which of the plurality of image planes contains a highest signal intensity at a modulation frequency; and
(63) identifying and outputting a light sheet axial position.
(64) A computer-readable information storage media having stored thereon instructions, that when executed by one or more processors, cause to be performed a method to automatically focus a light sheet in a single plane illumination microscopy device comprising:
(65) illuminating a sample at a fixed plane using a patterned light sheet;
(66) obtaining an image stack comprising a plurality of images of an emission pattern;
(67) performing a 2D transform of the image stack at a plurality of image planes;
(68) determining which of the plurality of image planes contains a highest signal intensity at a modulation frequency; and
(69) identifying and outputting a light sheet axial position.
(70) A system to automatically focus a light sheet in a single plane illumination microscopy device comprising:
(71) means for illuminating a sample at a fixed plane using a patterned light sheet;
(72) means for obtaining an image stack comprising a plurality of images of an emission pattern;
(73) means for performing a 2D transform of the image stack at a plurality of image planes;
(74) means for determining which of the plurality of image planes contains a highest signal intensity at a modulation frequency; and
(75) means for identifying and outputting a light sheet axial position.
(76) The exemplary techniques illustrated herein are not limited to the specifically illustrated embodiments but can also be utilized with the other exemplary embodiments and each described feature is individually and separately claimable.
(77) Additionally, the systems, methods and protocols can be implemented to improve one or more of a special purpose computer, a programmed microprocessor or microcontroller and peripheral integrated circuit element(s), an ASIC or other integrated circuit, a digital signal processor, a hard-wired electronic or logic circuit such as discrete element circuit, a programmable logic device such as PLD, PLA, FPGA, PAL, a microscope or microscope system, any comparable means, or the like. In general, any device capable of implementing a state machine that is in turn capable of implementing the methodology illustrated herein can benefit from the various communication methods, protocols and techniques according to the disclosure provided herein.
(78) Examples of the processors as described herein may include, but are not limited to, at least one of Qualcomm® Snapdragon® 800 and 801, Qualcomm® Snapdragon® 610 and 615 with 4G LTE Integration and 64-bit computing, Apple® A7 processor with 64-bit architecture, Apple® M7 motion coprocessors, Samsung® Exynos® series, the Intel® Core™ family of processors, the Intel® Xeon® family of processors, the Intel® Atom™ family of processors, the Intel Itanium® family of processors, Intel® Core® i5-4670K and i7-4770K 22 nm Haswell, Intel® Core® i5-3570K 22 nm Ivy Bridge, the AMD® FX™ family of processors, AMD® FX-4300, FX-6300, and FX-8350 32 nm Vishera, AMD® Kaveri processors, Texas Instruments® Jacinto C6000™ automotive infotainment processors, Texas Instruments® OMAP™ automotive-grade mobile processors, ARM® Cortex™-M processors, ARM® Cortex-A and ARM926EJ-S™ processors, Broadcom® AirForce BCM4704/BCM4703 wireless networking processors, the AR7100 Wireless Network Processing Unit, other industry-equivalent processors, and may perform computational functions using any known or future-developed standard, instruction set, libraries, and/or architecture.
(79) Furthermore, the disclosed methods may be readily implemented in software using object or object-oriented software development environments that provide portable source code that can be used on a variety of computer, microscope, imaging or workstation platforms. Alternatively, the disclosed system may be implemented partially or fully in hardware using standard logic circuits or VLSI design. Whether software or hardware is used to implement the systems in accordance with the embodiments is dependent on the speed and/or efficiency requirements of the system, the particular function, and the particular software or hardware systems or microprocessor or microcomputer systems being utilized. The systems, methods and protocols illustrated herein can be readily implemented in hardware and/or software using any known or later developed systems or structures, devices and/or software by those of ordinary skill in the applicable art from the functional description provided herein and with a general basic knowledge of the computer and imaging arts.
(80) Moreover, the disclosed methods may be readily implemented in software and/or firmware that can be stored on a storage medium to improve the performance of: a programmed general-purpose computer with the cooperation of a controller and memory, a special purpose computer, a microprocessor, imaging system, microscopy imaging system, or the like. In these instances, the systems and methods can be implemented as program embedded on personal computer such as an applet, JAVA® or CGI script, as a resource residing on a server or computer workstation, as a routine embedded in a dedicated imaging system or system component, or the like. The system can also be implemented by physically incorporating the system and/or method into a software and/or hardware system, such as the hardware and software systems of an imaging system or microscope system.
(81) Furthermore, the disclosed control methods may be readily implemented in software using object or object-oriented software development environments that provide portable source code that can be used on a variety of computer or workstation platforms. Alternatively, the disclosed control methods may be implemented partially or fully in hardware using standard logic circuits or VLSI design. Whether software or hardware is used to implement the systems in accordance with this invention is dependent on the speed and/or efficiency requirements of the system, the particular function, and the particular software or hardware systems or microprocessor or microcomputer systems being utilized.
(82) It is therefore apparent that there has been provided, in accordance with various embodiments an improvement to selective plane illumination microscopy. While aspects have been described in conjunction with a number of embodiments, it is evident that many alternatives, modifications and variations would be or are apparent to those of ordinary skill in the applicable arts. Accordingly, it is intended to embrace all such alternatives, modifications, equivalents and variations that are within the spirit and scope of this disclosure.
REFERENCES (ALL OF WHICH ARE INCORPORATED HEREIN BY REFERENCE IN THEIR ENTIRETY)
(83) Chen, B. C. et al. (2014) Lattice light-sheet microscopy: imaging molecules to embryos at high spatiotemporal resolution. Science 346, 1257998. Gao, L. et al. (2012) Noninvasive imaging beyond the diffraction limit of 3D dynamics in thickly fluorescent specimens. Cell 151, 1370-1385. Gao, L., Chen, B. C., Shao, L. & Betzig, E. (2014) 3D live fluorescence imaging of cellular dynamics using Bessel beam plane illumination microscopy. Nat. Protoc. 9, 1083-1101. Gao, L. (2015) Optimization of the excitation light sheet in selective plane illumination microscopy. Biomed. Opt. Express 6, 881-890. Huisken, J., Swoger, J., Del Bene, F., Wittbrodt, J. & Stelzer, E. H. K. (2004) Optical sectioning deep inside live embryos by selective plane illumination microscopy. Science 305, 1007-1009. Keller, P. J., Schmidt, A. D., Wittbrodt, J. & Stelzer, E. H. K. (2008) Reconstruction of zebrafish early embryonic development by scanned light sheet microscopy. Science 322, 1065-1069. Keller, P. J., Schmide, A. D., Santella, A., Khairy, K., Bao, Z., Wittbrodt, J. & Stelzer, E. H. K. (2010) Fast, high-contrast imaging of animal development with scanned light sheet-based structured-illumination microscopy. Nat. Methods. 7, 637-642. Liu, X. Y., Wang, W. H. & Sun, Y. (2007) Dynamic evaluation of autofocusing for automated microscopic analysis of blood smear and pap smear, J. Microsc. 227, 15-23. Oliva, M. A., Bravo-Zanoguera, M. & Price, J. H. (1999) Filtering out contrast reversals for microscopy autofocus, Appl. Opt. 38, 638-646. Planchon, T. A., Gao, L., Milkie, D. E., Davidson, M. W., Galbraith, J. A., Galbraith, C. G. & Betzig, E. (2011) Rapid three-dimensional isotropic imaging of living cells using Bessel beam plane illumination, Nat. Methods. 8, 417-423. Royer, L. A., Lemon, W. C., Chhetri, R. K., Wan Y., Coleman, M., Myers, E. W. & Keller, P. J. (2016) Adaptive light-sheet microscopy for long-term, high-resolution imaging in living organisms, Nature Biotechnology 34, 1267-1278. Sun, Y., Duthaler, S. & Nelson, B. J. (2004) Autofocusing in computer microscopy: Selecting the optimal focus algorithm, Microsc. Res. Tech. 65, 139-149. Wu, Y. et al. (2011) Inverted selective plane illumination microscopy (iSPIM) enables coupled cell identity lineaging and neurodevelopmental imaging in Caenorhabditis elegans. Proc. Natl Acad. Sci. USA 108, 17708-17713. Yazdanfar, S., Kenny, K. B., Tasimi, K., Corwin, A. D., Dixon, E. L. & Filkins, R. J. (2008) Simple and robust image-based autofocusing for digital microscopy, Opt. Express 16, 8670-8677.