SYSTEM AND METHOD FOR THREE-DIMENSIONAL IMAGING OF UNSTAINED SAMPLES USING BRIGHT FIELD MICROSCOPY
20230184683 · 2023-06-15
Assignee
Inventors
Cpc classification
G02B21/365
PHYSICS
G02B21/367
PHYSICS
G02B21/008
PHYSICS
G02B21/16
PHYSICS
International classification
Abstract
A system and method for optical sectioning in bright field microscopy (OSBM). The system includes a bright field optical microscope having automated change of focus, a substage condenser fitted with an adjustable aperture iris diaphragm, a digital camera that records the microscope image of samples, and one or more digital computers to perform digital image processing. The OSBM method comprises operating the microscope to Kohler illumination, using the iris diaphragm of the condenser to generate contrast in images, acquiring a Z-stack of images of the unstained sample, and applying a sequence of digital image processing filters to the Z-stack, resulting in optical sections from where the final three-dimensional (3D) image of the sample can be reconstructed by computational device. The final 3D images produced by this invention present quality comparable to that of available optical sectioning techniques that require sample labeling, such as light sheet fluorescence microscopy.
Claims
1. A method for three-dimensional imaging using bright field light microscopy, the method comprising the steps of: a) providing an unstained sample held by an appropriate mount and situated for observation in an optical microscope having automated change of focus, a substage condenser fitted with an adjustable aperture iris diaphragm, a digital camera that records the images of said unstained sample created by said microscope, and one or more digital computers that store and process the images of said unstained sample captured by said digital camera; b) operating said optical microscope in bright field mode; c) adjusting said optical microscope to Kohler illumination and operating said optical microscope under Kohler illumination; d) adjusting the aperture of said condenser iris diaphragm of said optical microscope so that contrast is generated in the images of said unstained sample captured by said digital camera of said optical microscope; e) acquiring a Z-stack of images of said unstained sample; f) transferring the outputs of said digital camera to said computer or computers; g) producing a stack of optical section images of said unstained sample by applying to said Z-stack of images a set of digital image processing filters; and h) reporting the result of step (f) in the form of digital image stacks, matrices of numbers, projection images or three-dimensional rendering images.
2. The method according to claim 1, wherein said unstained sample comprises organic or inorganic material, prokaryotic cells, eukaryotic cells, archaea cells, tissue, polymers, biological polymers, biofilms, transparent materials, or semi-transparent materials.
3. The method according to claim 1, wherein said unstained sample is free of staining agents and/or free of fluorescent agents.
4. The method according to claim 1, wherein said unstained sample contains staining and/or fluorescent agents but where these agents allow light transmission in bright field microscopy.
5. The method according to claim 1, wherein said appropriate mount comprises optically transparent surfaces, chambers, microchambers, cuvettes, or vessels.
6. The method according to claim 1, wherein said Z-stack of images comprises a collection of images corresponding to different planes of focus within said unstained sample.
7. The method according to claim 1, wherein said Z-stack of images is acquired by using said automated change of focus to image a set of different planes of focus within said unlabeled sample and using said digital camera to record a bright field image of said sample at each of said different planes of focus.
8. The method according to claim 1, wherein said set of digital image processing filters rejects out-of-focus background in the images of said Z-stack of images.
9. The method according to claim 1, wherein said set of digital image processing filters are applied to said Z-stack of images by said digital computer or computers.
10. The method according to claim 1, wherein said set of digital image processing filters comprises frequency filters that suppress low spatial frequencies in the XY, XZ and YZ slices of said Z-stacks of images, filters that subtract background noise from images of said Z-stacks of images, filters that smooth images of said Z-stacks of images, filters that perform pairwise frame subtraction to said Z-stacks of images, Top Hat filters, Mean filters, and Median filters.
11. An optical sectioning bright field microscopy system, comprising: an optical microscope having automated change of focus, a substage condenser fitted with an adjustable aperture iris diaphragm, and a digital camera, said digital camera recording the images of a sample produced by said optical microscope and having an output to provide the information contained in said images of a sample, said optical microscope configured to allow operation in bright field mode and under Kohler illumination, said optical microscope configured to allow adjustment of said iris diaphragm, said optical microscope configured to allow acquisition of Z-stacks of images of said sample by using said automated change of focus of said optical microscope to image a set of different planes of focus within said sample and using said digital camera to record a bright field image representative of each of said different planes of focus; one or more digital computers configured to receive the output of said digital camera and recognize said output as said Z-stacks of images of said sample, said one or more digital computers configured to apply a digital process to said Z-stacks of images of said sample resulting in optical section images of the said sample; and a reporting apparatus in communication with said one or more digital computers, said reporting apparatus configured to provide a report of said optical section images in the form of digital image stacks, matrices of pixel values, projection images or three-dimensional rendered images.
12. The optical sectioning bright field microscopy system of claim 11, wherein said adjustment of said iris diaphragm allows the generation of contrast in the images of said sample captured by said digital camera of said optical microscope.
13. The optical sectioning bright field microscopy system of claim 11, wherein said digital process consists of a set of digital image processing filters configured to remove out-of-focus background in the images comprising said Z-stacks of images of said sample.
14. The optical sectioning bright field microscopy system of claim 11, wherein said set of digital image processing filters comprise frequency filters that suppress low spatial frequencies in the XY, XZ and YZ slices of said Z-stacks of images of said sample, filters that subtract background noise from images of said Z-stacks of images of said sample, filters that smooth images of said Z-stacks of images of said sample, filters that perform pairwise frame subtraction, White Top Hat filters, Mean filters, Median filters.
Description
BRIEF DESCRIPTION OF THE FIGURES
[0018]
[0019]
[0020]
[0021]
[0022]
[0023]
[0024]
[0025]
[0026]
[0027]
[0028]
[0029]
[0030]
DETAILED DESCRIPTION OF THE INVENTION
[0031] The invention presented herein is related to optical bright field (BF) microscopy as an alternative to perform three-dimensional (3D) imaging of unstained samples. The herein optical sectioning in bright field microscopy method (henceforth referred to as OSBM) is based on acquiring a Z-stack of raw images using a microscope operating in bright field mode, whose illumination is adjusted to optimize image contrast, and applying a set of digital filters to the Z-stack of raw images. The procedure yields a stack of processed images corresponding to optical sections of the unstained samples from where 3D image reconstruction can be performed with a computer.
[0032] As used in this application, the term “bright field microscopy”, “bright field microscope”, “optical sectioning microscopy”, and “optical section” have the meanings given hereinbelow.
[0033] Bright field microscopy is an optical microscopy modality where light illuminating the sample is diffracted by the sample and image contrast is generated by the interference of diffracted light by the sample with undiffracted light.
[0034] Bright field microscope is a microscope that uses bright field microscopy to perform imaging of samples and has means to implement Kohler illumination.
[0035] Optical sectioning microscopy consists in imaging thin slices of a thick specimen by removal of out-of-focus background in each image plane.
[0036] Optical section is the image of a thin slice within a thick specimen, where such image contains minimal contribution from out-of-focus background.
[0037] The general strategy of the OSBM method is to achieve optical sectioning by performing axial localization of the scattering structures comprising the unstained sample.
[0038] In the regime of coherent illumination, the BF image of a point source (
[0039] The method of the present invention can be disclosed as follows:
[0040] 1. Support a sample on an optically transparent surface or contain the sample inside a microchamber or cuvette or similar; and surround the sample by air or by an optically transparent liquid or solid. Said sample comprising organic or inorganic material, free of staining agents or free of fluorescent agents or containing staining and/or fluorescent agents but where these agents allow light transmission in bright field microscopy.
[0041] 2. Place said sample in the specimen plane of an optical microscope having automated change of focus, a substage condenser fitted with an adjustable aperture iris diaphragm, a digital camera that records the image of said sample created by said microscope, and one or more digital computers that store the images captured by said digital camera of said sample and which are used to apply digital filters to said images. In some embodiments said digital camera having a CCD (Charged Coupled Device) or CMOS (Complementary metal-oxide-semiconductor) as the light-sensing device.
[0042] 3. Configure said optical microscope to operate in bright field mode.
[0043] 4. Configure said optical microscope to operate under Kohler illumination.
[0044] 5. Adjust the aperture of the said condenser iris diaphragm such that illumination so that contrast is generated in the images of said unstained sample captured by said digital camera of said optical microscope.
[0045] 6. Acquire a Z-stack of raw images of said unstained sample by using said automated change of focus to image a set of different planes of focus within said unlabeled sample and using said digital camera to record a bright field image of said sample at each of said different planes of focus.
[0046] 7. Transfer the outputs of said camera to said computer or computers for digital storage and processing.
[0047] 8. Produce a stack of optical section images of said unstained sample by applying to said Z-stack of raw images a set of digital image processing filters that reject out-of-focus background in the images of said Z-stack of raw images. In one embodiment:
[0048] a) Apply to said Z-stack of raw images digital image processing filters that remove the low spatial frequencies contained both in “frontal view” (XY) and “side views” (YZ and XZ) of the stack. In one embodiment of this step a fast Fourier transform (FFT) bandpass filter is used. In another embodiment of this step the image stack is “resliced” to generate YZ or XZ view, the high-pass filter is applied, and the stack is “resliced” back to XY view. This filtering step removes from the image stack the structures that extend over large axial distances away from scattering points, thus contributing to achieve out-of-focus background rejection.
[0049] b) Apply background subtraction to the resulting image stack of step (a). This filtering step produces images with uniform background.
[0050] c) Apply a smoothing filter to the resulting image stack of step (b). This filtering step reduces noise and contributes to the axial localization of scattering structures in images.
[0051] d) Apply pairwise frame subtraction to the resulting image stack of step (c). This filtering step reduces noise and contributes to the axial localization of scattering structures in images.
[0052] e) Apply a White Top Hat filter in 3D whose structuring element is a Z-line. This filtering step reduces noise and contributes to the axial localization of scattering structures in images.
[0053] In one embodiment said computer or computers are used to apply said set of digital image processing filters.
[0054] In one embodiment of this method, said set of digital image processing filters are implemented using software available such as FIJI, ImageJ, MATLAB, Python, Java, C++.
[0055] In another embodiment, said set of digital image processing filters are implemented with variation in order and/or by removing some of the filters and/or by adding filters that contribute to rejection of out-of-focus background from the images in said Z-stack of raw images.
[0056] This invention can be applied to a wide variety of optically transparent or translucid samples, such as prokaryotic cells, eukaryotic cells, archaea cells, polymers, biological polymers, biological tissues, biofilms, or transparent thin layer samples. These kinds of samples are named herein as the unstained samples or as the optically thin samples and are free of staining agents or free of fluorescent agents or contain staining and/or fluorescent agents but where these agents allow light transmission through the sample in bright field microscopy.
[0057] Is an embodiment that the result of the OSBM method is a stack of processed images containing optical sections of said sample, from where the final 3D image of said sample can be reconstructed by digital means.
[0058] The present OSBM invention is a different approach to achieve optical sectioning microscopy and three-dimensional microscopy. The advantages of the OSBM method are:
[0059] The method is suitable for samples free of staining or fluorescent agents.
[0060] The method is suitable for samples that are optically transparent or partially transparent.
[0061] The method does not use high-intensity laser sources that cause photodamage to the specimen.
[0062] The method is suitable to be implemented using bright field microscopy without introducing any other optical imaging technique.
[0063] The method is suitable to be implemented in bright field microscopes.
[0064] The method does not require required use of specialized interferometric instrumentation.
[0065] The method does not require theoretical or computer modelling.
[0066] The method does not require a process of computer training.
[0067] The method achieves optical sectioning comparable to that of available optical sectioning techniques such as light sheet fluorescence microscopy, allowing the proper identification of sample morphology in 3D.
EXAMPLES
[0068] The following examples disclose the best way to practice the invention, without limiting the scope of the same. In light of this disclosure, those of skill in the art will recognize that variations, modifications or substitutions may be applied to the system of this invention, to the method of this invention, to the steps of the method, or to the sequence of steps of the method described herein without departing from the concept, spirit and scope of the invention.
Example 1: Axial Localization of Scattering Structures in BF and Application of the OSBM Method to an Unlabeled Fungal Sample
[0069] In
[0070] To prepare the Trichoderma atroviride sample (hereafter called the fungal sample), 700 .Math.L of Vogel’s medium was left to solidify at the bottom of a polymethacrylate fluorimeter cuvette. Conidia of Trichoderma atroviride wild-type strain (IMI 206040) were inoculated at the surface of the medium and left to incubate during 36 h at 28° C. Next, 2 mL of liquid Vogel’s medium at 1% was poured in the cuvette, over the fungal colony. After an incubation over 24 h at 28° C., the sample was taken to the BF microscope for measurements.
[0071] The microscope used is a home-made instrument (hereafter called the home-made microscope) with the dual capability of operating in BF modality or in Light Sheet Fluorescence Microscopy (LSFM) modality. The LSFM modality uses fluorescence to enable three-dimensional visualization of fluorescent samples (Stelzer, E. H. K. (2014) “Light-sheet fluorescence microscopy for quantitative biology”. Nat Methods 12:23-26).
[0072] The BF modality of the microscope is used to implement the OSBM method, whereas the LSFM modality is used to validate the results of OSBM. The construction and testing of the home-made instrument have been described in detail (Gutierrez-Medina, B., Vazquez Villa, A. “Visualizing three-dimensional fungal growth using light sheet fluorescence microscopy”. Fung. Genet. Biol. 150, 103549 (2021)). In summary, the microscope comprises an illumination laser beam (λ = 488 nm) focused onto the sample by a 4X/0.13 (magnification/numerical aperture) microscope objective (N4X-PF, Nikon).
[0073] The laser beam is scanned using twin galvanometric mirrors and shaped using telescope and scanning lenses, such that a light-sheet 5.6-.Math.m thick (twice the beam waist, 2ω.sub.0) and 100-.Math.m deep (twice the Rayleigh length) is formed at the sample plane. The detection arm is formed by a 10X/0.28 long-working distance (LWD = 34 mm) microscope objective (MY10X-803, Mitutoyo), a GFP 525/39 emission filter (MF525-39 Thorlabs), a tube lens TL2 (TTL200, Thorlabs) and a sCMOS camera (OrcaFlash 4.0-LT, Hamamatsu).
[0074] An 800-.Math.m travel range piezo electric stage (P-628.1CD, PI) displaces the detection objective axially, concomitantly with the light sheet. Instrument control is achieved using a multifunction data acquisition system (DAQ) (USB-6341, National Instruments) operated by a dual Xeon gold 5122 (3.60 GHz) workstation (Precision 7920, Dell). Automated control and acquisition are achieved by custom-made routines developed in LabView (National Instruments).
[0075] The BF arm of the home-made microscope consists of a high-power light emitting diode (LED) illumination source (λ = 445 nm, SOLIS-445C, Thorlabs), followed by a collector lens (f = 60 mm) and a LWD air condenser lens (0.78 NA, CSC2001, Thorlabs), all mounted on a rigid microscope body (CFB1500, Thorlabs). To provide space for sample manipulation, a 2f relay lens was used to image the condenser illumination plane onto the specimen plane.
[0076] A sample of Trichoderma atroviride was placed in the home-made microscope, the BF modality of the microscope was aligned to Kohler illumination, and the condenser diaphragm was adjusted to limit the numerical aperture of the condenser to (NA).sub.condenser = 0.15. A Z-stack of images was acquired using parameters: 200 ms exposure time, 4 .Math.m z-step, 260 frames in the stack. Images were acquired at 16-bit depth, 2048×2048 px.sup.2, and LED illumination intensity was adjusted such that background counts in images were ~40,000 counts. Acquired images were stored in the computer operating the microscope for off-line processing.
[0077] To apply digital image processing filters on the acquired BF Z-stack of images, the software FIJI (Schindelin, J., Arganda-Carreras, I., Frise, E. et al. “Fiji: an open-source platform for biological-image analysis”. Nat Methods 9, 676-682 (2012)) was used. Custom-made functions called “macros” and available procedures called “plugins” within FIJI were used. The following image processing steps were implemented in sequence:
[0078] STEP 1. The Bandpass Filter plugin with parameters (filter_large = 1 px; filter_small = 0 px) was applied to each x-y image of the Z-stack, yielding the image stack called I.sub.FFTxy.
[0079] The Reslice plugin was applied to image stack I.sub.FFTxy from top and left, resulting in image stacks I.sub.FFT.sub.xy-xz and I.sub.FFT.sub.xy-yz, respectively. The Bandpass Filter plugin with parameters (filter_large = 1 px; filter_small = 0 px) was applied to each I.sub.FFT.sub.xy-xz and I.sub.FFT.sub.xy-yz, and the resulting were resliced back (using the Reslice plugin) to X-Y views and averaged, resulting in stack called .sub.1.
[0080] STEP 2. Background subtraction was performed on a frame-by-frame basis onto stack l.sub.1, by computing the mean count value of a selected region-of-interest where no sample is present and subtracting this value from all pixel-counts in the frame. The result is image stack called l.sub.2.
[0081] STEP 3. The image stack l.sub.2 was binned by a factor of 2 in X and Y, yielding a Z-stack of 1024×1024 px.sup.2. The result is image stack called ls.
[0082] STEP 4. A smoothing filter was applied to l.sub.3, using the plugin Gaussian Blur 3D with parameters (x = 2, y = 2, z = 1.5). The result is image stack called l.sub.4.
[0083] STEP 5. Pairwise frame subtraction (Δ = 4 frames) is performed onto image stack l.sub.4, by duplicating frames 5-260 and frames 1-256 of stack l.sub.4 (calling the duplicates l.sub.45-260 and l.sub.41-254, respectively) and performing the subtraction: l.sub.4.sub.1-254 - l.sub.45-260. The result is image stack called l.sub.5.
[0084] STEP 6. A White Top Hat filter along the z-axis is applied to image stack l.sub.5, available in the plugin Morphological Filters (3D) with parameters “operation=[White Top Hat] element=Z-Line x-radius=1 y-radius=1 z-radius=2”. The result is image stack called l.sub.6.
[0085] STEP 7. A final filter is applied to l.sub.6 to remove small particles (below an area of 40 p.sub.X.sup.2), helping to reduce background noise, by creating a binary mask of l.sub.6 that retained image features larger than 40 p.sub.X.sup.2 (using the plugin Analyze Particles) in area and using this mask to filter out spots smaller than 40 p.sub.X.sup.2 form images of the l.sub.6 stack. No further filters are applied to the resulting images. The result of the entire image processing filter process is called lFINAL-FUNGUS.
[0086] In
[0087] To validate the results shown in
[0088]
[0089] In
Example 2: Application of the OSBM Method to a Cleared Tissue Sample
[0090] In
[0091] The said home-made microscope described in Example 1 was used for the experiment and results pertaining to Example 2.
[0092] The cleared tissue sample was placed in the home-made microscope, the BF modality of the microscope was aligned to Kohler illumination, and the condenser diaphragm was adjusted to limit the numerical aperture of the condenser to (NA).sub.condenser = 0.05. A Z-stack of images was acquired using parameters: 200 ms exposure time, 4 .Math.m z-step, 290 frames in the stack. Images were acquired at 16-bit depth, 1024×1024 px.sup.2, and LED illumination intensity was adjusted such that background counts in images were ~40,000 counts. Acquired images were stored in the computer operating the microscope for off-line processing.
[0093] The sequence of digital image processing filters applied to the Z-stack of BF images of the cleared tissue sample were the same as for Example 1, with the following modifications: (i) In STEP 1 the Bandpass Filter plugin was used with parameters (filter_large = 2 px; filter_small = 0 px); (ii) STEP 3 was not performed; (iii) In STEP 5 pairwise frame subtraction is performed by duplicating frames 6-290 and frames 1-285 of the stack resulting from STEP 4; (vi) STEP 7 was not performed. No further filters are applied to the resulting images. The result of the entire image processing filter process is called lFINAL-TISSUE.
[0094] To validate the results of the OSBM method for the cleared tissue sample, the same tissue sample was imaged using LSFM. A Z-stack of images was acquired employing the LSFM modality of the said home-made microscope, using the following imaging parameters: laser intensity 1 mW, exposure time 200 ms, z-step between frames 4 .Math.m. The autofluorescence of tissue was used to perform LSFM imaging.
[0095]
Example 3: Application of the OSBM Method to an Onion Skin Sample
[0096] In
[0097] The said home-made microscope described in Example 1 was used for the experiment and results pertaining to Example 3.
[0098] The unlabeled onion sample was placed in the home-made microscope, the BF modality of the microscope was aligned to Kohler illumination, and the condenser diaphragm was adjusted to limit the numerical aperture of the condenser to (NA).sub.condenser = 0.05. A Z-stack of images was acquired using parameters: 200 ms exposure time, 4 .Math.m z-step, 200 frames in the stack. Images were acquired at 16-bit depth, 1024×1024 px.sup.2, and LED illumination intensity was adjusted such that background counts in images were ~40,000 counts. Acquired images were stored in the computer operating the microscope for off-line processing.
[0099] The sequence of digital image processing filters applied to the Z-stack of BF images of the onion sample are the same as for Example 1, with the following modifications: (i) In STEP 1 the Bandpass Filter plugin was used with parameters (filter_large = 2 px; filter_ small= 0 px); (ii) STEP 3 was not performed; (iii) In STEP 5 pairwise frame subtraction is performed by duplicating frames 5-200 and frames 1-295 of the stack resulting from STEP 4; (vi) STEP 7 was not performed. No further filters are applied to the resulting images. The result of the entire image processing filter process is called I.sub.FINAL-ONION.
[0100]
Example 4: Application of the OSBM Method to a Plant Root Sample
[0101] The OSBM method was applied to a plant root sample, which presents significant optical opacity. In
[0102] The said home-made microscope described in Example 1 was used for the experiment and results pertaining to Example 4.
[0103] The plant root sample was placed in the home-made microscope, the BF modality of the microscope was aligned to Kohler illumination, and the condenser diaphragm was adjusted to limit the numerical aperture of the condenser to (NA).sub.condenser = 0.05. A Z-stack of images was acquired using parameters: 200 ms exposure time, 4 .Math.m z-step, 200 frames in the stack. Images were acquired at 16-bit depth, 2048×2048 px.sup.2, and LED illumination intensity was adjusted such that background counts in images were ~40,000 counts. Acquired images were stored in the computer operating the microscope for off-line processing.
[0104] The sequence of digital image processing filters applied to the Z-stack of BF images of the plant root sample are the same as for Example 1, with the following modification: STEP 7 was not performed. No further filters are applied to the resulting images. The result of the entire image processing filter process is an image stack of optical sections called l.sub.FINAL-ROOT.
[0105] To validate the results of the OSBM method for the plant root sample, the same root sample was imaged using LSFM. A Z-stack of images was acquired employing the LSFM modality of the said home-made microscope, using the following imaging parameters: laser intensity 1 mW, exposure time 200 ms, z-step between frames 4 .Math.m. The autofluorescence of tissue was used to perform LSFM imaging.
[0106] For the plant root sample
[0107] These results with the plant sample demonstrate that even in partially optically opaque samples OSBM achieves accurate optical sectioning of regions with good light transmission.