ADAPTIVE ILLUMINATION APPARATUS, METHOD, AND APPLICATIONS

Abstract

A system and method for adaptive illumination, the imaging system comprising an excitation source having a modulator, which generates a pulse intensity pattern having a first wavelength when the excitation source receives a modulation pattern. The modulation pattern is a data sequence of a structural image of a sample. An amplifier of the imaging system is configured to receive and amplify the pulse intensity pattern from the modulator. A frequency shift mechanism of the imaging system shifts the first wavelength of the pulse intensity pattern to a second wavelength. A laser scanning microscope of the imaging system receives the pulse intensity pattern having the second wavelength.

Claims

1. An imaging system for adaptive illumination, comprising: an excitation source having a modulator, which generates a pulse intensity pattern having a first wavelength when the excitation source receives a modulation pattern; wherein the modulation pattern is a data sequence of at least a portion of a structural image of a sample; an amplifier configured to receive and amplify the pulse intensity pattern from the modulator; a frequency shift mechanism which shifts the first wavelength of the pulse intensity pattern to a second wavelength; and a laser scanning microscope, which receives the pulse intensity pattern having the second wavelength.

2. The imaging system of claim 1, wherein the frequency shift mechanism a soliton self-frequency shift (SSFS).

3. The imaging system of claim 2, wherein the SSFS is in a photonic crystal rod.

4. (canceled)

5. The imaging system of claim 1, wherein the sample is a neuron.

6. The imaging system of claim 1, wherein the excitation source is an all-fiber chirped pulse amplifier.

7. The imaging system of claim 1, wherein the modulator is an electro-optic modulator.

8. One or more machine-readable media comprising instructions that, when executed by a machine, cause the machine to: generate a structural image of a sample; process the structural image to determine one or more regions of interest; convert the regions of interest into a data sequence representing a modulation pattern having a first wavelength; transmit the modulation pattern to an excitation source; encode, at the excitation source, the modulation pattern in time; amplify the modulation pattern with an amplifier at the excitation source; shift the first wavelength of the modulation pattern to a second wavelength; and transmit the modulation pattern having the second wavelength to a laser scanning microscope.

9. The one or more machine-readable media of claim 8, wherein the instructions further cause the machine to conduct a raster scan of a sample.

10. The one or more machine-readable media of claim 8, wherein to amplify the modulation pattern with an amplifier comprises to amplify the modulation pattern with an Erbium-doped-fiber amplifier.

11. The one or more machine-readable media of claim 8, wherein to transmit the modulation pattern to an excitation source comprises to transmit the modulation pattern to a chirped pulse amplifier.

12. The one or more machine-readable media of claim 8, wherein to generate a structural image of a sample comprises to generate a structural image of a neuron.

13. The one or more machine-readable media of claim 13, wherein to process the structural image to determine one or more regions of interest comprises to proves the structural image to determine one or more regions of interest defined by somas of the neuron.

14. An imaging system for adaptive illumination, comprising: an excitation source having a modulator, which generates a pulse intensity pattern having a first wavelength when the excitation source receives a modulation pattern; wherein the modulation pattern is a data sequence of one or more regions of interest in a structural image of a sample; an amplifier configured to receive and amplify the pulse intensity pattern from the modulator; a frequency shift mechanism which shifts the first wavelength of the pulse intensity pattern to a second wavelength; and a laser scanning microscope, which receives the pulse intensity pattern having the second wavelength.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In the drawings, like reference characters generally refer to the same parts throughout the different views. Also, the drawings are not necessarily to scale, emphasis instead generally being placed upon illustrating the principles of the invention.

[0021] FIG. 1 is a schematic representation of an imaging system for adaptive illumination, in accordance with an embodiment.

[0022] FIG. 2 is a graph of results from wavelength tuning using SSDFS, in accordance with an embodiment.

[0023] FIG. 3 is a chart of a method of multiplane imaging, in accordance with an embodiment.

[0024] FIG. 4A is a pre-EDFA graph and post-EDFA graph showing pulse energy of output pulses over time.

[0025] FIG. 4B is the pre-EDFA graph and post-EDFA graph from FIG. 4A with the data pre-shaped.

[0026] FIG. 5 is a diagram of a feedback loop for flattening the output pulse train when it is passed from the EOM to the EDFA, in accordance with an embodiment.

[0027] FIG. 6 is a first data set of images resulting from use of the imaging system and method for adaptive illumination, in accordance with an embodiment.

[0028] FIG. 7 is a second data set of images resulting from use of the imaging system and method for adaptive illumination, in accordance with an embodiment.

[0029] FIG. 8 is a third data set of images resulting from use of the imaging system and method for adaptive illumination, in accordance with an embodiment.

[0030] FIG. 9 is a fourth data set of images resulting from use of the imaging system and method for adaptive illumination, in accordance with an embodiment.

[0031] FIG. 10 is a fifth data set of images resulting from use of the imaging system and method for adaptive illumination, in accordance with an embodiment.

[0032] FIG. 11 is a sixth data set of images resulting from use of the imaging system and method for adaptive illumination, in accordance with an embodiment.

DETAILED DESCRIPTION

[0033] The present disclosure is directed to embodiments of a method and system for adaptive illumination.

[0034] According to one embodiment, the disclosure is directed to an imaging system for adaptive illumination. Referring first to FIG. 1, there is shown a schematic representation of an imaging system 100 for adaptive illumination, in accordance with an embodiment. The system 100 comprises a laser scanning microscope 102 for raster scanning of a sample. Raster scanning of the sample produces a high-resolution structural image 104. Image processing is then performed on the high-resolution structural image 104 to determine regions of interest (ROIs) 106. For recording the activity of neurons, the ROIs 106 are defined by the regions of the somas of the neurons. After determining the ROIs 106, the ROI 106 information is converted to a digital sequence, a calculated modulation pattern. The calculated modulation pattern is then used to control an arbitrary waveform generator (AWG) 108, which drives an excitation source 110.

[0035] In the depicted embodiment, the excitation source 110 is an all-fiber chirped pulse amplifier (CPA). Conventional CPAs are widely known and used for amplifying fs pulses to high energies without the associated nonlinear effects, such as self-focusing, photodissociation, and UV-light production. However, conventional CPAs use a pulse picker to uniformly remove the pulses and obtain a desired repetition rate. The embodiment of the CPA 110 in FIG. 1 comprises a fiber optic electro-optic modulator (EOM) 112, which imposes a desired pulse intensity pattern on a low power fiber seed laser 114 having a high repetition rate (e.g., >20 MHz). Thus, the EOM 110 encodes the pulse intensity pattern in time.

[0036] Still referring to FIG. 1, the CPA 110 also comprises a fiber amplifier 116. In the depicted embodiment, the fiber amplifier 116 is an Erbium-doped-fiber amplifier (EDFA). The EDFA 116 receives and amplifies the resulting modulated pattern (i.e., pulse train). As the EDFA 116 has a long excited state lifetime, intensity modulation at 100s of KHz will have no impact on the average output power of the EDFA 116, which is how fiber amplifiers are utilized in telecommunications to amplify data impulses. In some aspects, the proposed excitation source 110 behaves like a telecommunications transmitter transmitting the image of the ROIs 106. The amplified pulses typically range from 1 to 4 μJ/pulse depending on the effective repetition rate, and are then compressed back to approximately 400 fs in a compressor.

[0037] The imaging system 100 of FIG. 1 also comprises a soliton self-frequency shift (SSFS) 118. The SSFS shifts the wavelength from 1550 nm to approximately 1700 nm. In the depicted embodiment, the SSFS 118 is shown in a large mode area photonic crystal (PC) rod 118, which is achieved by controlling the launch power. In the embodiment shown, the SSFS 118 is capable of generating wavelength tunable solitons from 1600 nm to close to 2000 nm, as shown as (c) in FIG. 1. The images (a) and (b) in FIG. 1 are images resulting from a conventional laser source (i.e., uniform pulse train) and the imaging system 100 described herein, respectively. The solitons have short pulse width and high pulse energy for 3PE. For 2PM with adaptive illumination as described herein, the pulses can be frequency doubled using SHG crystals, such as BiBo and KNBO, to the range of 800 nm to 1000 nm.

[0038] The pulse intensity pattern is then transmitted from the SSFS 118 to the laser scanning microscope 102, a multiphoton microscope (MPM) 102 in the depicted embodiment. The synchronization of scanning and the pulse intensity pattern ensure that the excitation beam will only illuminate the somas. Thus, all the permissible laser power is focused on the ROIs 106 and the laser is essentially “turned off” outside the ROIs 106. Through the laser power concentration, the signal generation and resulting imaging speed is increased by the inverse of the volume fraction of the ROI 106. Localized labeling of the cells further enhances the increased speed. An example of a localized label is a nucleus label, which provides more precise laser power concentration, increasing the imaging speed. This method can be used to simultaneously image thousands of neurons in deep cortical layers of subjects, such as mice and rats, without degrading the temporal resolution and signal-to-noise ratio (SNR). In addition, the method can be used to simultaneously image tens of thousands of neurons through the depth of a single cortical column with simultaneous 2PM (shallow) and 3PM (deep).

[0039] The above embodiments describe 3PM at the 1700 nm spectral window and 2PM at 800 nm-1000 nm. In an alternative embodiment, the method is executed at the 1300 nm window. Because the proposed pulse modulation scheme is compatible with all amplified laser systems, fiber or solid state, the method may be translated to the 1300 nm window. In the 1300 nm embodiment, a 1300 nm excitation source 102 may utilize a CPA 110 to pump an optical parametric amplifier (OPA). A nearly identical modulation scheme may be used to control the pulse pattern of the CPA 110, which subsequently generates the desired pulse train at the 1300 nm in the OPA.

[0040] In another embodiment, “unwanted” pulses are removed before the final power amplifier (i.e., the EDFA 116). While modulation of laser power using an EOM 112 or AOM has been done routinely in the past, e.g., for beam blanking or enhancing dynamic range, the modulation is always performed after the laser output (i.e., outside the laser). By placing the modulator (EOM) 112 inside the imaging system 100, the imaging system 100 not only allows a high speed, low power (i.e., only required to handle <100 mW), and low cost (˜$1k) fiber optic modulator (EOM) 112 to perform the intensity modulation, but also enables the entire output power of the fiber amplifier (EDFA) 116 to be used. It is estimated that an EDFA 116 with ˜5 W output power is sufficient for the subsequent SSFS 118. Such an amplifier is commercially available and can be routinely made in a university lab. In contrast, placing the modulator (EOM) 112 outside the imaging system 100 would necessitate an EDFA 116 with ˜50 W output (e.g., neurons occupy ˜10% of the volume of the mouse brain). Such a high power amplifier would be a major challenge to the current state-of-the-art, and totally impractical for neuroscience applications.

[0041] In an alternative embodiment, each ROI 106 may also comprise a guard band. The guard band is built around the ROI 106 and used to eliminate motion artifact resulting from imaging awake or behaving animals (or other samples). The size of the guard band may be determined based at least in part on the range of motion measured during the image session. That way, the size of guard band is tailored to the RMS value of the motion to eliminate the vast majority of the motion artifact without significantly comprising the efficiency of the illumination. In one embodiment, the guard band may also be used to easily detect systematic drift. Thus, the microscope may receive real-time feedback of the sample drift, which allows it to track the sample drift.

[0042] Referring now to FIG. 2, there is shown a graph of results from wavelength tuning using SSFS 118, in accordance with an embodiment. Adaptive wavelength tuning may be executed using SSFS 118 (as shown in FIG. 1). Using SSFS 118 allows for faster wavelength tuning than conventional tuning using a mode-locked laser (e.g., single box Ti:S laser). Wavelength tuning across the 1700 nm spectral window for 3PE can be achieved on a pulse-to-pulse time scale using the EO intensity modulator. The combination of adaptive intensity modulation and wavelength tuning ensures the most efficient illumination for samples with multiple fluorescent markers with significant spectral separations.

[0043] Referring now to FIG. 3, there is shown a chart of a method of multiplane imaging, in accordance with an embodiment. In the depicted embodiment, concurrently 2PE and 3PE (two-photon excitation microscopy and three-photon excitation microscopy) and are used to simultaneously measure neural activity from both shallow and deep cells. In one example, deep cells include scattering tissue, which is approximately 1 mm in a mouse brain. Through simultaneous activity recording with 2PE and 3PE, the number of neurons studied in each region may be maximized. In the embodiment shown in FIG. 3, time division multiplexing 2PE and 3PE laser beams, which are spatially offset to have two separate focal planes after the objective, permit simultaneous activity recording.

[0044] In the depicted embodiment, the 2PE beam is switched to the “on” position only during the time between two adjacent 3PE pulses. The 2PE beam is switched to the “on” position by intensity modulation with an EOM 112, which is driven with an RF signal derived from the 3PE pulse train using a photodetector, as shown. A sample is alternatively (within approximately 1 μs) illuminated either by the 2PE or the 3PE beam. The 2- and 3-photon excited fluorescence signals are separated in time according to the recorded 3PE laser pulse train with a post-processing software.

[0045] For imaging deep cells (e.g., into scattering tissue), the optimum repetition rate for a 3PE source is approximately 1 MHz due to the high peak power needs. The optimal repetition rate for 2PM at shallower depths is approximately 10-50 times higher. The significant difference in pulse repetition rates allows parallel 2- and 3-photon imaging without the need to synchronize the excitation pulse trains. Synchronizing the excitation pulse trains is often difficult and thus, the method is more efficient without it. Still referring to FIG. 3, through removal of occasional “conflicting” pulses (e.g., using a pulse picker) from the 2PE source, which comprise approximately 1-10% of the pulses, a minimum offset between excitation pulses of tens of ns can be ensured, which is sufficient for more fluorophores (e.g., the fluorescence lifetime for many GCaMPs is approximately 3 ns). Therefore, simultaneous 2- and 3-photon imaging can be achieved for the same fluorophore just by correlating the detected fluorescence signal with the appropriate excitation source (as shown in FIG. 3). Once again, such an intensity modulation can be done before the final power amplifier stage, enabling efficient and robust operation. One of the benefits of simultaneous use of 2- and 3-photon excitation comes when the lasers are focused in different planes. Simply controlling the divergence of the beam entering the objective back aperture can provide hundreds of micrometers of focal plane shift without significant imaging performance degradation (e.g., FOV and resolution).

[0046] Referring now to FIGS. 4A-11, there are shown various graphs and charts illustrating the proof of concept for the imaging system for adaptive illumination. Referring first to FIGS. 4A and 4B, there is shown the pulse energy of output pulses over time. When a uniform pulse train is sent through the EDFA 116, the output pulses have different pulse energy due to the EDFA 116 gain transient. In FIG. 4A, the pre-EDFA uniform pulse train is shown on the left and the post-EDFA output pulses are shown on the right. It has been proved in the field of telecommunications and fiber laser source that this problem can be solved by pre-shaping the input, as shown in our measured data. The data shown in FIG. 4A has been pre-shaped, resulting in the graphs in FIG. 4B. Specifically, in FIG. 4B, the pre-shaped uniform pulse train pre-EDFA is shown on the left and the pre-shaped output pulses post-EDFA is shown on the right. However, when the input pulse train has a complicated temporal distribution, the definition of pre-shaping waveform will be impossible.

[0047] In order to facilitate pre-shaping with an input pulse train having a complicated temporal distribution, a feedback loop configuration can flatten the output pulse train. In an exemplary embodiment, shown in FIG. 5, the input pulse train is flattened each time it is passed from the EOM 112 to the EDFA 116. The input pulse train will be flattened a 1st round, a 2nd round, and an Nth round until the output pulse train is flat. In one embodiment, the flattening process may require several tens of rounds.

[0048] Referring now to FIG. 6, there is shown a first data set of images resulting from use of the imaging system 100 and method for adaptive illumination described herein. The results shown in FIG. 6 were produced with fluorescein dye pool and an Olympus microscope. Image (A) in FIG. 6 is an image produced according to conventional method (i.e., without the imaging system 100 and method for adaptive illumination disclosed herein). Images (B), (C), and (D) were obtained using the disclosed imaging system 100 and method for adaptive illumination (AI). As can be derived from the measurements shown in Tables 1 below, AI resulted in a power improvement factor of approximately 10.

TABLE-US-00001 TABLE 1 Comparative Imaging Measurements Uniform Non-uniform (without AI) (with AI) Power on sample 1.7 mW 0.16 mW Effective repetition rate 1 MHz ~0.1136 MHz Brightness (mean pixel value) 872.914 854.455 Calculated Measured Improvement Improvement Power (assuming same brightness) ~8.8 10.5 Brightness (assuming same average power) ~77.5 ~110.5

[0049] Referring now to FIG. 7, there is shown a second data set of images resulting from use of the imaging system 100 and method for adaptive illumination described herein. The results shown in FIG. 7 were produced with fluorescein dye pool and an Olympus microscope. Image (A) in FIG. 7 is an image produced according to conventional method (i.e., without the imaging system 100 and method for AI disclosed herein). Images (B), (C), and (D) were obtained using the disclosed imaging system 100 and method for AI. As can be derived from the measurements shown in Tables 2 below, AI resulted in a power improvement factor of approximately 6.

TABLE-US-00002 TABLE 2 Comparative Imaging Measurements Uniform Non-uniform (without AI) (with AI) Power on sample 1.9 mW 0.48 mW Effective repetition rate 1 MHz ~0.2 MHz Brightness (mean pixel value) 545 1272 Calculated Measured Improvement Improvement Power (assuming same brightness) ~5 6 Brightness (assuming same average power) ~25 ~36

[0050] Referring now to FIG. 8, there is shown a third data set of images resulting from use of the imaging system 100 and method for adaptive illumination described herein. The results shown in FIG. 8 were produced with fluorescein dye pool and an Olympus microscope. Image (A) in FIG. 8 is an image produced according to conventional method (i.e., without the imaging system 100 and method for AI disclosed herein). Images (B), (C), and (D) were obtained using the disclosed imaging system 100 and method for AI. As can be derived from the measurements shown in Tables 3 below, AI resulted in a brightness improvement factor of approximately 23, 41, and 100, respectively.

TABLE-US-00003 TABLE 3 Comparative Imaging Measurements Uniform Non-uniform 1 Non-uniform 2 Non-uniform 3 (without AI) (with AI) (with AI) (with AI) Power on sample 0.32 mW 0.32 mW 0.32 mW 0.32 mW Laser repetition rate 1 MHz 1 MHz 1 MHz 1 MHz Effective repetition rate 1 MHz 0.25 MHz 0.156 MHz 0.094 MHz Pixel value (mean value of the 9.619 222.6 402.272 1042.111 bright area) Measured improvement factor 23 42 100 Theoretical improvement factor 16 41 114

[0051] Referring now to FIG. 9, there is shown a fourth data set of images resulting from use of the imaging system 100 and method for adaptive illumination described herein. The results shown in FIG. 9 were produced with neurons (YFP) and an Olympus microscope. Images (A) and (C) are at a depth of 338 μm, while images (B) and (D) are at a depth of 320 μm. Images (A) and (B) were produced according to conventional method (i.e., without the imaging system 100 and method for AI disclosed herein) with 4.32 mW on the sample. Images (C) and (D) were obtained using the disclosed imaging system 100 and method for AI with 0.88 mW on the sample. The use of AI resulted in a power improvement factor of approximately 4.36.

[0052] Referring now to FIG. 10, there is shown a fifth data set of images resulting from use of the imaging system 100 and method for adaptive illumination described herein. The results shown in FIG. 10 were produced with neurons (YFP) and an Olympus microscope. Images (A) and (C) are at a depth of 306 μm, while images (B) and (D) are at a depth of 649 μm. Images (A) and (B) were produced according to conventional method (i.e., without the imaging system 100 and method for AI disclosed herein) with 3.84 mW on the sample. Images (C) and (D) were obtained using the disclosed imaging system 100 and method for AI with 0.56 mW on the sample. The use of AI resulted in a power improvement factor of approximately 6.85.

[0053] Finally, turning to FIG. 11, there is shown a sixth data set of images resulting from use of the imaging system 100 and method for adaptive illumination described herein. The results shown in FIG. 11 were produced with activity imaging (GCaMP6) and a home-made microscope. Image (A) and Image (B) in FIG. 11 are both 62×62 μm, 256×256 pixels, at a depth of 498 μm taken at 15 frames per second. The power provided to Image (A) was 64 mW, while the power provided to the sample in Image (B) was 7 mW. Image (A2) was produced according to conventional method (i.e., without the imaging system 100 and method for AI disclosed herein). Image (B2) was obtained using the disclosed imaging system 100 and method for AI. As can be derived from the measurements shown in Tables 4 below, AI resulted in a power improvement factor of approximately 9.1.

TABLE-US-00004 TABLE 4 Comparative Imaging Measurements Uniform Non-uniform (without AI) (with AI) Power on sample 64 mW 7 mW Effective repetition rate 4 MHz ~0.25 MHz Brightness when neuron not fire 120 120 (mean pixel value) Calculated Measured Improvement Improvement Power (assuming same brightness) ~8 9.1 Brightness (assuming same average power) ~64 ~83

[0054] While various embodiments have been described and illustrated herein, those of ordinary skill in the art will readily envision a variety of other means and/or structures for performing the function and/or obtaining the results and/or one or more of the advantages described herein, and each of such variations and/or modifications is deemed to be within the scope of the embodiments described herein. More generally, those skilled in the art will readily appreciate that all parameters, dimensions, materials, and configurations described herein are meant to be exemplary and that the actual parameters, dimensions, materials, and/or configurations will depend upon the specific application or applications for which the teachings is/are used. Those skilled in the art will recognize, or be able to ascertain using no more than routine experimentation, many equivalents to the specific embodiments described herein. It is, therefore, to be understood that the foregoing embodiments are presented by way of example only and that, within the scope of the appended claims and equivalents thereto, embodiments may be practiced otherwise than as specifically described and claimed. Embodiments of the present disclosure are directed to each individual feature, system, article, material, and/or method described herein. In addition, any combination of two or more such features, systems, articles, materials, and/or methods, if such features, systems, articles, materials, and/or methods are not mutually inconsistent, is included within the scope of the present disclosure.

[0055] The claims should not be read as limited to the described order or elements unless stated to that effect. It should be understood that various changes in form and detail may be made by one of ordinary skill in the art without departing from the spirit and scope of the appended claims. All embodiments that come within the spirit and scope of the following claims and equivalents thereto are claimed.