AUTOMATED SYSTEM FOR WIDE-FIELD MULTIPHOTON MICROSCOPE
20210341720 · 2021-11-04
Assignee
Inventors
- Rachel GRANGE (Delémont, CH)
- Maria TIMOFEEVA (Wallisellen, CH)
- Lukas LANG (Ruggell, LI)
- Claude RENAUT (Chérisey, FR)
Cpc classification
G02B21/365
PHYSICS
International classification
G02B21/16
PHYSICS
Abstract
A method of investigating a sample using a wide-field multiphoton microscope with tunable excitation wavelength (λ) is disclosed. The sample is illuminated at some excitation wavelength. Images of the sample are acquired at wavelengths different from the excitation wavelength, e.g., at half the excitation wavelength for SHG microscopy, using an image sensor. Based on the obtained images, an autoalignment procedure is carried out for optimizing the position of the sample relative to the illumination and/or collection beam paths. An image or spectrum that has been obtained for the optimal relative position is stored. The procedure is repeated for multiple excitation wavelengths. The autoalignment procedure can comprise an autofocusing subprocedure to automatically optimize the position of the sample relative to a focal plane of at least one objective of the multiphoton microscope along a direction that is perpendicular to the focal plane, and an in-plane repositioning subprocedure for automatically optimizing the position of the sample within the focal plane.
Claims
1. A method of investigating a sample using a wide-field multiphoton microscope with tunable excitation wavelength, the method comprising: (a) operating a light source of the multiphoton microscope to create excitation light at an excitation wavelength; (b) focusing the excitation light to a focal region in or on the sample, using an illumination system of the multiphoton microscope, the illumination system defining an illumination beam path; (c) collecting light from the focal region, using a collection system of the multiphoton microscope, the collection system defining a collection beam path, the collection system comprising a wavelength separator to suppress light at the excitation wavelength; (d) directing collected light that has passed the wavelength separator to an image sensor; (e) based on the obtained images, carrying out an autoalignment procedure for optimizing a position of the sample relative to the illumination and/or collection beam paths; (f) storing an image and/or spectrum obtained from the sample in the optimized relative position; and (g) repeating steps (a) to (f) for a plurality of excitation wavelengths.
2. The method of claim 1, wherein the autoalignment procedure of step (e) comprises at least one of the following subprocedures: (e1) an autofocusing subprocedure for automatically optimizing the position of the sample relative to a focal plane of at least one objective of the multiphoton microscope along a direction that is perpendicular to the focal plane; and (e2) an in-plane repositioning subprocedure for automatically optimizing the position of the sample within the focal plane.
3. The method of claim 1, wherein the excitation wavelength is changed systematically in a stepwise manner across an excitation wavelength range.
4. The method of claim 1, wherein the autoalignment procedure comprises optimizing a focus score that is calculated from the obtained images of the sample, the focus score preferably being an intensity measure.
5. The method of claim 1, wherein the autoalignment procedure comprises a coarse alignment subprocedure followed by a fine alignment subprocedure.
6. The method of claim 5, wherein coarse alignment subprocedure comprises: obtaining images of the sample, using the image sensor, while stepwise changing a relative position between the sample and the illumination and/or collection beam paths along a direction of change; calculating a focus score from each image; and determining a range of relative positions along the direction of change where the focus score is near an optimum.
7. The method of claim 5, wherein the coarse alignment subprocedure comprises: determining an indicator whether the direction of change of the relative position should be reversed, and if the indicator indicates that the direction of change should be reversed, reversing the direction of change.
8. The method of claim 5, wherein the coarse focusing subprocedure comprises computing running averages of focus scores for different relative positions, and wherein the range of relative positions is determined based on the running averages.
9. The method of claim 5, wherein the fine alignment subprocedure comprises systematically scanning the range of relative positions that was determined by the coarse alignment subprocedure in steps that are smaller than the during the coarse alignment subprocedure.
10. The method of claim 1, wherein the illumination system comprises an illumination objective, wherein the collection system comprises a collection objective, and wherein the multiphoton microscope comprises a first translation stage for moving the illumination objective along the illumination beam path and a second translation stage for moving the collection objective along the collection beam path, and wherein the autoalignment procedure comprises operating at least one of the first and second translation stages to change a distance between the sample and the illumination and/or collection objectives.
11. The method of claim 1, wherein the illumination system and the collection system comprise a common objective, wherein the multiphoton microscope comprises a translation stage for moving the common objective along a beam path portion that is common to the illumination and collection beam paths, and wherein the autoalignment procedure comprises operating the translation stage to change a distance between the sample and the common objective along the common beam path portion.
12. The method of claim 1, wherein the multiphoton microscope comprises a sample holder mounted on a pair of translation stages for moving the sample holder along two mutually orthogonal transverse directions that are transverse to the illumination and detection beam paths, and wherein the autoalignment procedure comprises operating the pair of translation stages so as to move the sample holder relative to the illumination and collection beam paths along the transverse directions.
13. A wide-field multiphoton microscope, comprising: a tunable light source; an illumination system for focusing light from the light source to a focal region on or in a sample, the illumination system defining an illumination beam path; a collection system for collecting light from the focal region, the collection system defining a collection beam path, the collection system comprising a wavelength separator to suppress light at the excitation wavelength; and an image sensor for obtaining images from the light that has passed the wavelength separator; a positioning system for changing a position of the sample relative to the illumination and/or collection beam paths; and a controller configured to carry out a method of investigating a sample using the wide-field multiphoton microscope, the method comprising: (a) operating the tunable light source to create excitation light at an excitation wavelength; (b) focusing the excitation light to the focal region in or on the sample using the illumination system; (c) collecting light from the focal region using the collection system; (d) directing collected light that has passed the wavelength separator to the image sensor; (e) based on the obtained images, carrying out an autoalignment procedure for optimizing a position of the sample relative to the illumination and/or collection beam paths using the positioning system; (f) storing an image and/or spectrum obtained from the sample in the optimized relative position; and (g) repeating steps (a) to (f) for a plurality of excitation wavelengths.
14. The wide-field multiphoton microscope of claim 13, wherein the illumination system comprises an illumination objective having a first magnification, wherein the collection system comprises a collection objective having a second magnification, and wherein the second magnification is larger than the first magnification.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
[0039] Preferred embodiments of the invention are described in the following with reference to the drawings, which are for the purpose of illustrating the present preferred embodiments of the invention and not for the purpose of limiting the same. In the drawings,
[0040]
[0041]
[0042]
[0043]
[0044]
[0045]
[0046]
[0047]
[0048]
[0049]
[0050]
[0051]
[0052]
[0053]
DESCRIPTION OF PREFERRED EMBODIMENTS
[0054] Exemplary Setups for Wide-Field Multiphoton Microscopy
[0055] (i) Transmission Mode
[0056]
[0057] Light from light source 1 is reflected by a mirror 21 and enters an illumination objective 22 that is mounted on an automated translation stage allowing the position of the illumination objective 22 to be changed along the illumination beam path. The optical axis of illumination objective 22 extends along the z1 axis, which in the present example is a horizontal axis.
[0058] A sample 2 is mounted on automated translation stages allowing the position of sample 2 to be changed along two mutually orthogonal directions perpendicular to the z1 axis, denoted the x and y axes. Illumination objective 22 focuses the excitation light to a focal region on or in sample 2.
[0059] Light originating from the focal region is collected by a collection objective 23, which is also placed on an automated translation stage allowing the position of collection objective 23 to be changed along an axis designated the z2 axis. In the present embodiment, the z1 and z2 axes coincide.
[0060] It is preferred that illumination objective 22 has a lower magnification than collection objective 23 to perform wide-field excitation, thus allowing the investigation of the sample without scanning. For instance, illumination objective 22 may have a magnification of 10×, whereas collection objective 23 may have a magnification of 100×. Of course other combinations if magnifications are possible.
[0061] Collection objective 23 is followed by an optional beam splitter 25, whose function will become apparent from the discussion of
[0062] In order to selectively observe light at desired detection wavelengths (e.g., SHG light at half the excitation wavelength), filter wheel 31 is set such that only light at the desired detection wavelengths passes through filter wheel 31, while light at other wavelengths (in particular, light at the excitation wavelength) is rejected. In more general terms, filter wheel 31 may be considered an example of a wavelength separator for separating light at the excitation wavelength from light at the detection wavelengths. Instead of filter wheel 31, for instance a prism or grating may be used for suppressing light at the excitation wavelength.
[0063] A controller 3 receives data from power meters 14 and/or 24, camera 32 and any other detectors that might be present. Controller 3 carries out all necessary control tasks. For instance, in one aspect, controller 3 controls wavelength, output power and polarization of light source 1. In another aspect, controller 3 controls the acquisition of images by camera 32. In yet another aspect, controller 3 receives image data from camera 32, calculates a focus score from the acquired images, and carries out an autoalignment procedure based on the focus score, as will be described in more detail below in conjunction with
[0064] (ii) “Angled” Reflection Mode
[0065]
[0066] (iii) “Straight” Reflection Mode
[0067]
[0068] (iv) Combining Transmission and Reflection Mode
[0069]
Exemplary Embodiment of Mechanical Setup
[0070]
[0071] Focus Score
[0072] In the present disclosure, image-based methods are employed for autoalignment in a wide-field multiphoton microscope. Such methods use actual image data acquired by the image sensor and calculate a quantitative figure of merit, called the focus score, from the image data. Autoalignment is achieved by optimizing (e.g. maximizing or minimizing) the focus score while changing the relative position of the sample and the optical elements used for focusing.
[0073] Preferably, the focus score is an intensity measure, for instance a (possibly weighted) sum over the intensities (or squared intensities or any other power of intensities) of all pixels or of selected pixels of the image sensor. However, the focus score can be calculated by any other known algorithm for calculating focus scores, including derivative-based, statistical, intuitive or histogram-based algorithms as disclosed, for example, in Yu Sun et al., “Autofocusing in Computer Microscopy: Selecting the Optimal Focus Algorithm”, Microscopy Research and Technique 65:139-149 (2004), DOI 10.1002/jemt.20118. These algorithms include: [0074] Variance (Groen et al., 1985; Yeo et al., 1993), Statistical Algorithm:
F.sub.max=max(i(x,y)) [0081] Edge Filter plus Variance, combining Derivative-Based and Statistical Algorithms. The edge filter reveals high pixel intensity changes. It acts similarly to the derivative method as it compares pixel intensity differences, but it returns an image, enabling the application of another algorithm to it.
[0082] Some algorithms for determining a focus score require the calculation of a threshold. In order to be able to calculate such a threshold, a reference score is needed, which may for instance correspond to the maximum intensity to be expected in a single pixel. In order to determine the reference score, it may be necessary to carry out a quick, coarse scan of the search range, taking images for a plurality of sample positions relative to the optical components of the microscope, and compute the highest single-pixel score from each image. This may be done at the beginning of the autoalignment subprocedure.
[0083] Considerations Concerning Wavelength Tuning
[0084]
Embodiment of Proposed Method
[0085]
[0086] The method starts with setting an initial excitation wavelength λ in step 51.
[0087] An image acquisition procedure 60 is carried out at excitation wavelength λ. The image acquisition procedure comprises the following steps: In step 61, excitation light at excitation wavelength λ is created by operating the light source at this wavelength. In step 62, the light is focused to a focal region on or in the sample using the optical components of the illumination beam path, in particular, the illumination objective. In step 63, light from the focal region is collected, using the optical components of the collection beam path, in particular, the collection objective (which in “straight” reflection mode may be identical with the illumination objective). In step 64, the collected light passes through a wavelength separator that suppresses light at the excitation wavelength while allowing light having different wavelengths, in particular, approximately half the excitation wavelength, to pass the wavelength separator. In step 65, the light that has passed the wavelength separator is directed to the image sensor, and an image of the sample is obtained by the image sensor.
[0088] Image acquisition procedure 60 is followed by autoalignment procedure 70. In the autoalignment procedure, the images obtained in the image acquisition procedure 60 are analyzed, and based on this analysis it is determined in step 71 whether the position of the sample relative to the optical components in the illumination and collection beam paths is optimal. In particular, it is determined whether the illumination and/or collection objectives are properly focused to a region of interest of the sample (i.e., whether the focal plane of the objective(s) contains the region of interest), and whether the region of interest is properly aligned to the illumination and collection beam paths within the focal plane. If this is not the case, the relative position of the sample and at least one of the objectives is changed in step 72, and the image acquisition procedure 60 is repeated until proper alignment has been achieved. To this end, a focus score is computed for each relative position of sample and objective(s), and the focus score is optimized (e.g., maximized or minimized, depending on the type of focus score) by changing the relative position of the sample and the objective(s).
[0089] In step 80, an image obtained in the optimized position is stored, or a spectrum is obtained in this position and stored.
[0090] This sequence of events is repeated as often as possible until a wavelength sweep across a desired wavelength range has been completed. To this end, wavelength sweep procedure 90 includes a step 91 of determining whether the wavelength sweep has been completed. If not, the excitation wavelength λ is changed in step 92, and procedures 60 to 80 are repeated until the wavelength sweep has been completed. The method then ends at step 100.
[0091] Additional Considerations Concerning Autoalignment
[0092] (i) Autofocusing in Transmission and Angled Reflection Mode
[0093] If two separate objectives are used for illumination and collection, both the illumination objective 22 and the collection objective 23 should be properly focused onto the sample region of interest. Different strategies can be pursued to achieve this goal. In one strategy, first the optical distance between the illumination and collection objectives is optimized so as to cause the focal planes of both objectives to coincide, and only thereafter the position of the sample along the z2 axis is optimized while keeping the optical distance between the objectives fixed. This sequence can be repeated as often as necessary until the focal planes of the objectives coincide and the sample region of interest is properly positioned within the focal plane. However, other strategies are conceivable, for instance, strategies wherein the position of each objective relative to the sample along the illumination and collection beam paths is optimized separately.
[0094] (ii) Autofocusing in “Straight” Reflection Mode
[0095] In “straight” reflection mode (as in
[0096] (iii) Optimization of in Plane Position
[0097] Further to autofocusing, the position of the sample within the focal plane should be optimized. The same or different algorithms for calculating a focus score may be employed for autofocusing and for in-plane positioning.
[0098] (iv) Coarse/Fine Alignment Subprocedures
[0099] In order to speed up autoalignment without compromising on the result of autoalignment, for each alignment axis the autoalignment procedure may be split into two subprocedures. This is schematically illustrated in the flowchart of
[0100] The coarse alignment subprocedure can use the so-called three-point method for determining the slope and curvature of a focus score as a function of the relative position of the sample and the objective(s) along a particular direction and for determining whether the focus score has reached a maximum.
[0101] The three-point method is illustrated in
[0102] Once the maximum has been found, a range of relative positions around the maximum is defined, and the fine alignment subprocedure is carried out only in this range of positions. In the present example, the range of positions between B1 and B3 would be employed for the fine alignment subprocedure.
[0103] In order to reduce the likelihood that the three-point method converges to a local optimum that is not the global optimum, a running average of the focus scores for at least two, preferably at least three successive relative positions is determined, and the three-point method is applied to the running averages. This amounts to a kind of low-pass filtering of the focus score with respect to position before applying the three-point method.
[0104] The effect of this kind of averaging is illustrated in
[0105] The coarse alignment subprocedure is illustrated in the form of a flow chart in
[0106] The fine alignment subprocedure is schematically illustrated in
[0107] In the fine alignment subprocedure, in step 141, the translation stages are set to a relative position within the range determined by the coarse alignment subprocedure. In step 142, an image is taken, and the corresponding focus score is determined. In step 143, it is checked whether the whole range has already been evaluated. If not, the next position is evaluated. Once the whole range has been evaluated, it is determined which relative position has achieved the highest focus score, and the translation stages are moved to that relative position (step 144). This ends the autoalignment procedure for this axis.
[0108] The fine alignment subprocedure is diagrammatically illustrated in
[0109] Each known focus score algorithm has a particular dependence on relative position.
[0110] Although the invention has been described with reference to particular embodiments, this description is not meant to be construed in a limiting sense, and various modifications and alternative embodiments are possible. For instance, while in the above-described embodiments a half-wave plate together with a polarizing beam splitter is used for controlling power of the excitation light, any other means for controlling power may be employed, as it is well known in the art. Instead of filter wheel 31, any other wavelength separator may be used. If no spectral information is desired, mirror 27, filter 28 and spectrometer 29 can be omitted. Instead of camera 32, any other type of digital image sensor that is capable of recording 2D images in the desired wavelength range can be used, and instead of lens 30, any other means of focusing light onto the image sensor can be employed. In embodiments without power control, power meters 14, 24 can be omitted. The invention is not limited to the particular setups of