Full-Color Three-Dimensionnal Optical Sectioning Microscopic Imaging System and Method Based on Structured Illumination

Abstract

The present invention provides a full-color three-dimensional optical sectioning microscopic imaging system and method based on structured illumination, includes an illumination source, a dichroic prism positioned at the illumination optical path, a structured light generator positioned at the reflected optical path of the dichroic prism, a lens positioned at the transmitted optical path of the dichroic prism, a beam splitter positioned at the optical path of the lens, an objective lens and a sample stage positioned at the upper optical path of the beam splitter, a reflector mirror and a tube lens positioned at the lower optical path of the beam splitter and a CCD camera positioned behind the tube lens. The illumination source is an incoherent monochrome LED or a white light LED The structured light generator is a DMD (Digital Micro-mirror Device).

Claims

1-5. (canceled)

6. A full-color three-dimensional optical sectioning microscopic imaging system, comprising: an illumination source, a dichroic prism positioned in the illumination optical path, a structured light generator positioned in the reflected optical path of the dichroic prism, a lens positioned in the transmitted optical path of the dichroic prism, a beam splitter positioned in the optical path of the lens, an objective lens and a sample stage positioned in the upper optical path of the beam splitter, and a reflector mirror and a tube lens positioned in the lower optical path of the beam splitter.

7. The imaging system of claim 6, further comprising a CCD camera positioned behind the tube lens.

8. The imaging system of claim 7, wherein the CCD camera is a color CCD camera.

9. The imaging system of claim 6, wherein the illumination source is an incoherent monochrome LED or a white light LED.

10. The imaging system of claim 6, wherein the structured light generator is a DMD (Digital Micro-mirror Device).

11. The imaging system of claim 6, wherein the beam splitter is a long-pass dichroic or a broad band beam-splitter.

12. A method for obtaining a full-color three-dimensional optical sectioning microscopic image, comprising: Step. 1) Generating structured illumination light pattern: Use a monochrome LED or a white light LED as the illumination source, and combine with a DMD to generate three structured illumination patterns with same orientation but different phases. The structured illumination patterns illuminate the sample placed on the sample stage through the objective lens. Step. 2) Collecting images by a color CCD camera: Corresponding to three structured illumination patterns with different phases (0, 120, and) 240, the color CCD camera collect three two-dimensional color images I.sub.0(RGB), I.sub.120(RGB) and I.sub.240(RGB), respectively. Step. 3) Image processing: Step. 3.1) Convert the three 2D color images I.sub.0(RGB), I.sub.120(RGB) and I.sub.240(RGB) from RGB color space to HSV color space according to the following conversion equation (1), and then get three 2D images with different phases in HSV color space: I.sub.0(HSV), I.sub.120(HSV) and I.sub.240(HSV), $\begin{array}{cc}H=\{\begin{array}{cc}0.Math.,& \mathrm{if}.Math..Math.\max =\min \\ 60.Math.\frac{G-R}{\max -\min }+0.Math.,& \mathrm{if}.Math..Math.\max =R,\mathrm{and}.Math..Math.GB\\ 60.Math.\frac{G-R}{\max -\min }+360.Math.,& \mathrm{if}.Math..Math.\max =R,\mathrm{and}.Math..Math.G<B\\ 60.Math.\frac{B-R}{\max -\min }+120.Math.,& \mathrm{if}.Math..Math.\max =G\\ 60.Math.\frac{B-R}{\max -\min }+240.Math.,& \mathrm{if}.Math..Math.\max =B\end{array}.Math..Math.S=\{\begin{array}{cc}0,& \mathrm{if}.Math..Math.\max =0\\ 1-\frac{\min }{\max },& \mathrm{if}.Math..Math.\max =B\end{array}.Math..Math.V=\max & \left(1\right)\end{array}$ where max=max{R, G, B}, min=min{R, G, B}. Step. 3.2) According to equation (2), three wide-field images I.sub.wide(i) in H channel, S channel and V channel are calculated respectively, where i=H, S, V; According to equation (3), three sectioned images I.sub.z(i) in H channel, S channel and V channel are calculated respectively, where i=H, S, V. $\begin{array}{cc}{I}_{\mathrm{wide}}\left(i\right)=\frac{1}{3}\left[I_0\left(i\right)+I_{120}\left(i\right)+I_{240}\left(i\right)\right]& \left(2\right)\\ I_z\left(i\right)=\sqrt{{\left[I_0\left(i\right)-I_{120}\left(i\right)\right]}^2+{\left[I_{120}\left(i\right)-I_{240}\left(i\right)\right]}^2+{\left[I_{240}\left(i\right)-I_0\left(i\right)\right]}^2}& \left(3\right)\end{array}$ Step. 3.3) Combine the three sectioned images got in Step. 3.2) to a single sectioned image and reconvert it from HSV color space to RGB color space; Combine the three wide-filed images got in Step. 3.2) to a single wide-field image without fringe patterns and reconvert it from HSV color space to RGB color space; Converting from HSV color space to RGB color space according to conversion equation (4), where H.sub.i=H/60. $\begin{array}{cc}R=\{\begin{array}{cc}V,& \mathrm{if}.Math..Math.H_i=0.Math..Math.\mathrm{or}.Math..Math.H_i=5\\ V\left[1-S\left(\frac{H}{60.Math.}-H_i\right)\right],& \mathrm{if}.Math..Math.H_i=1\\ V\left(1-S\right),& \mathrm{if}.Math..Math.H_i=2.Math..Math.\mathrm{or}.Math..Math.H_i=3\\ V\left[1-S\left(1-\frac{H}{60.Math.}+H_i\right)\right],& \mathrm{if}.Math..Math.H_i=4\end{array}.Math..Math.G=\{\begin{array}{cc}V\left[1-S\left(1-\frac{H}{60.Math.}+H_i\right)\right],& \mathrm{if}.Math..Math.H_i=0\\ V,& \mathrm{if}.Math..Math.H_i=1.Math..Math.\mathrm{or}.Math..Math.H_i=2\\ V\left[1-S\left(\frac{H}{60.Math.}-H_i\right)\right],& \mathrm{if}.Math..Math.H_i=3\\ V\left(1-S\right),& \mathrm{if}.Math..Math.H_i=4.Math..Math.\mathrm{or}.Math..Math.H_i=5\end{array}.Math..Math.B=\{\begin{array}{cc}V\left(1-S\right),& \mathrm{if}.Math..Math.H_i=0.Math..Math.\mathrm{or}.Math..Math.H_i=1\\ V\left[1-S\left(1-\frac{H}{60.Math.}+H_i\right)\right],& \mathrm{if}.Math..Math.H_i=2\\ V,& \mathrm{if}.Math..Math.H_i=3.Math..Math.\mathrm{or}.Math..Math.H_i=4\\ V\left[1-S\left(\frac{H}{60.Math.}-H_i\right)\right],& \mathrm{if}.Math..Math.H_i=5\end{array}& \left(4\right)\end{array}$ Step. 3.4) Normalize the sectioned image I.sub.z(RGB) got in Step. 3.3) to get the normalized sectioned image I.sub.z.sub._.sub.Norm(RGB), and then multiply it with the wide-field image I.sub.wide(RGB) to get the two-dimensional color sectioned image I.sub.z.sub._.sub.result(RGB) of this layer. Step. 4) Depending on the thickness of the sample, repeat Step. 2) and Step. 3) in required times, to obtain a series of two-dimensional color sectioned images along Z-direction. Finally, the completely three-dimensional color image of the sample I.sub.result(RGB) can be obtained.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0033] FIG. 1 is the schematic diagram of the full-color three-dimensional optical sectioning microscopic imaging system based on structured illumination;

[0034] FIG. 2 is the flowchart diagram of technical solution 3;

[0035] FIG. 3 is the 3D reconstructed images of mixed pollen grain specimen. The color comes from the auto-fluorescence of the pollen grain under the excitation of blue LED In which FIG. 3(a) is the 3D color reconstructed image and FIG. 3(b) is the 3D monochrome reconstructed image using an conventional SIM system (for comparison);

[0036] FIGS. 4(a) and 4(b) are the 3D reconstructed images of pollen grains with different colors and shapes. In which FIG. 4(a) is the 3D color reconstructed image and FIG. 4(b) is the 3D monochrome reconstructed image using an conventional SIM system (for comparison);

[0037] FIGS. 5(a) and 5(b) are the 3D reconstructed images of a micro circuit chip. In which FIG. 5(a) is the 3D color reconstructed image, the color comes from the reflection of the metallic surface illuminated by a white light LED and FIG. 5(b) is the 3D monochrome reconstructed image using an conventional SIM system (for comparison).

EMBODIMENTS OF THE INVENTION

[0038] The present invention is a full-color 3D optical sectioning microscopic imaging system based on structured illumination. As shown in FIG. 1, it includes an illumination source 1, a dichroic prism 2 positioned at the illumination optical path, a structured light generator 3 positioned at the reflected optical path of 2, a lens 4 positioned at the transmitted optical path of 2, a beam splitter 5 positioned at the optical path of 4, an objective lens 6 and a sample stage 7 positioned at the upper optical path of 5, a reflector mirror 8 and a tube lens 9 positioned at the lower optical path of 5 and a CCD camera 10 positioned behind 9. The CCD camera 10 is a color CCD camera. The illumination source 1 is an incoherent monochrome LED or a white light LED. The structured light generator 3 is a DMD (Digital Micro-mirror Device).

[0039] The present invention can be applied to either fluorescently labeled specimens or non-fluorescent reflective specimens.

[0040] 1. Reconstruction of Three-Dimensional Color Image of Mixed Pollen Grains

[0041] Step. 1) A 450 nm wavelength LED enters the dichroic prism and irradiates the DMD chip perpendicularly, the reflective light transmits the dichroic prism and enters the collimate lens, then illuminates the pollen grain sample through the objective lens.

[0042] Step. 2) Lie the mixed pollen grain sample in the structured light field and place it on the sample stage.

[0043] Step. 3) Control the DMD load three structured fringe patterns with different phases (0, 120, and 240), color CCD camera respectively collect three two-dimensional color images I.sub.0(RGB), I.sub.120(RGB) and I.sub.240(RGB), which are stored in the computer. Through the imaging processing algorithm mentioned in technical solutions 3, the color sectioned image of this layer will be obtained. FIG. 2 is the specific flow chart diagram of technical solutions 3.

[0044] Step. 4) Move the motorized stage along Z direction and repeat Step. 3), two-dimensional sectioned images of other layers of the sample will be obtained, and finally get the complete 3D color image.

[0045] FIG. 3 is the 3D color image of the mixed pollen grains obtained under the use of the system described in the present invention. The color comes from the auto-fluorescence of the pollen grains. A 20/NA 0.45 objective lens and 450 nm blue LED are used in the experiment. For each image, the exposure time is 0.215 s, and three sub-images are needed for each layer. FIG. 3(a) is the reconstructed 3D color image of the mixed pollen grains using the system of the present invention. As a comparison, FIG. 3(b) is the reconstructed monochrome image using the conventional SIM system.

[0046] Different pollen grains have different shapes and volumes, it also can be observed that they emit auto-fluorescence with different wavelengths. FIG. 4s show pollen grains with different shapes and colors. FIG. 4(a) is the reconstructed 3D color image using the system of the present invention. As a comparison, FIG. 4(b) is the reconstructed monochrome image using the conventional SIM system.

[0047] 2. Reconstruction of Three-Dimensional Color Image of a Micro Circuit Chip

[0048] Step. 1) Use white light LED as the illumination source, replace the blue LED used for imaging the mixed pollen grains.

[0049] Step. 2) In order to collect the reflected light from the metal surface of the micro circuit chip, the long-pass dichroic mirror 5 used for fluorescent imaging is replaced by a 50:50 broad band beam-splitter.

[0050] Step. 3) Repeat Step. 2) to Step. 4) in the embodiment of Reconstruction of three-dimensional color image of mixed pollen grains.

[0051] FIG. 5 is the 3D color image of the micro circuit chip obtained after 25 layers imaging. A 20/NA 0.45 objective lens and a white light LED are used in the experiment. For each image, the exposure time is 0.027 s, and three sub-images are needed for each layer. FIG. 5(a) is the reconstructed 3D color image of the micro circuit chip, the color comes from the reflected light of the chip's surface. As a comparison, FIG. 3(b) is the reconstructed monochrome image using the conventional SIM system.