A SINGLE-SHOT DIFFERENTIAL PHASE CONTRAST QUANTITATIVE PHASE IMAGING METHOD BASED ON COLOR MULTIPLEXED ILLUMINATION
20220366552 · 2022-11-17
Assignee
Inventors
- Qian Chen (Nanjing, CN)
- Yao FAN (Nanjing, CN)
- Chao Zuo (Nanjing, CN)
- Jiasong Sun (Nanjing, CN)
- Xiangpeng PAN (Nanjing, CN)
- Shijie Feng (Nanjing, CN)
- Yuzhen Zhang (Nanjing, CN)
- Guohua Gu (Nanjing, CN)
- Jiaji LI (Nanjing, CN)
- Jialin ZHANG (Nanjing, CN)
Cpc classification
G06T2207/10101
PHYSICS
G02B21/367
PHYSICS
International classification
Abstract
A single-shot differential phase contrast quantitative phase imaging method based on color multiplexing illumination. A color multiplexing illumination solution is used to realize single-shot differential phase contrast quantitative phase imaging. In the single-shot color multiplexing illumination solution, three illumination wavelengths of red, green, and blue are used to simultaneously illuminate a sample, and the information of the sample in multiple directions is converted into intensity information on different channels of a color image. By performing channel separation on this color image, the information about the sample at different spatial frequencies can be obtained. Such a color multiplexing illumination solution requires only one acquired image, thus enhancing the transfer response of the phase transfer function of single-shot differential phase contrast imaging in the entire frequency range, and achieving real-time dynamic quantitative phase imaging with a high contrast, a high resolution, and a high stability. In addition, an alternate illumination strategy is provided, so that a completely isotropic imaging resolution at the limit acquisition speed of the camera can be achieved.
Claims
1. A single-shot DPC QPI method based on color multiplexed illumination, characterized in the imaging process steps as follows: step 1, image acquisition under color multiplexed illumination: computer-controlled high-contrast LCD TFT-LCD or high-density programmable LED array display color multiplexed illumination pattern irradiated samples, while sending illumination control to generate a synchronous trigger signal to the color camera to acquire a color sample image, recorded as I.sub.c; step 2, image color channel separation and correction: the color sample image I.sub.c is separated in a single channel and corrected for color leakage to obtain the sample intensity image I.sub.r,corr, I.sub.g,corr, I.sub.b,corr corresponding to the three channels of red r, green g and blue b; step 3, differential phase lining image spectrum generation: fourier transform of the sample intensity images of the three channels I.sub.r,corr, I.sub.g,corr, I.sub.b,corr, to obtain the spectral distribution of the three images, the zero frequency of the three spectra is set to 0, to eliminate the effect of the background term, to obtain the DPC image spectral distribution of the three channels of samples expressed as ,
,
; step 4, PTF calculation: Based on the weak phase approximation condition, the PTF PTF.sub.r(ρ,θ), PTF.sub.g(ρ,θ), PTF.sub.b(ρ,θ), corresponding to different wavelengths is calculated according to the parameters of the illumination function and the objective lens; step 5, sample quantitative phase recovery: according to the PTF PTF.sub.r(ρ,θ), PTF.sub.g(ρ,θ), PTF.sub.b(ρ,θ), of different wavelengths and the DPC image spectrum
,
,
of the sample, the Tikhonov criterion is used for the inverse convolution calculation to obtain the high- resolution spectrum of the sample phase, and the inverse Fourier transform is performed on the high-resolution spectrum to obtain the quantitative phase distribution ϕ of the sample:
2. A single-shot DPC QPI method based on color multiplexed illumination, characterized in that the imaging process steps are as follows: step 1, image acquisition under color multiplexed illumination: computer-controlled high-contrast liquid crystal display TFT-LCD or high-density programmable LED array display color multiplexed illumination pattern irradiated samples, while sending illumination control to generate a synchronous trigger signal to the color camera to capture a color sample image, recorded as I.sub.c; using an alternating illumination strategy, the illumination pattern is rotated 90° in any direction to illuminate the sample to acquire a second color sample image, denoted as I.sub.c,⊥; step 2, image color channel separation and correction: the two captured color sample images I.sub.c, I.sub.c,⊥, respectively, channel separation and correction, to obtain two images corresponding to the three channels of the sample intensity image I.sub.r,corr, I.sub.g,corr, I.sub.b,corr, I.sub.r,⊥,corr, I.sub.g,⊥,corr, I.sub.b,⊥,corr; step 3, DPC image spectrum generation: the sample intensity image I.sub.r,corr, I.sub.g,corr, I.sub.b,corr, I.sub.r,⊥,corr, I.sub.g,⊥,corr, I.sub.b,⊥,corr of a single channel corresponding to the two acquired images will be solved for the spectrum and the zero frequency of the spectrum will be removed, and the background term will be eliminated; step 4, PTF calculation: solve for the PTF.sub.r(ρ,θ), PTF.sub.g(ρ,θ), PTF.sub.b(ρ,θ), PTF.sub.r,⊥(ρ,θ), PTF.sub.g,⊥(ρ,θ), PTF.sub.b,⊥(ρ,θ), corresponding to the two illumination patterns; step 5, sample quantitative phase recovery: the two illumination patterns corresponding to the sample image spectrum and transfer function are brought into the deconvolution calculation, then the quantitative phase distribution ϕ of the sample is solved by the formula:
3. The method according to claim 1 wherein in step one, the illumination pattern is a semi-annular illumination of three colors of red r, green g, and blue b with an asymmetric axis difference of 120° , wherein the NA of the semi-annular illumination is equal to the NA of the objective, and the illumination intensity is distributed sinusoidally, assuming that, the NA of the objective is expressed as NA.sub.obj ;the illumination function for color multiplexing expressed in polar coordinates is:
S.sub.r(ρ,θ)=δ(ρ—NA.sub.obj)sin(θ+θ.sub.r) S.sub.g(ρ,θ)=δ(ρ—NA.sub.obj)sin(θ+θ.sub.g) S.sub.b(ρ,θ)=δ(ρ—NA.sub.obj)sin(θ+θ.sub.b) ( θ.sub.y+θ.sub.g—120 ° , θ.sub.b=θ.sub.g+120° ) where S.sub.r(ρ,θ), S.sub.g(ρ,θ), S.sub.b(ρ,θ) denotes the illumination functions corresponding to the three wavelengths of red r, green g. and blue b, respectively, ρ, θ denotes the radius and polar angle of the polar coordinate system, respectively, θ.sub.r, θ.sub.g, θ.sub.b, is the angle of the asymmetry axis of the illumination patterns of red r, green g, and blue b, respectively, and δ(ρ—NA.sub.obj) denotes the shape of the illumination pattern is an annulus where the NA of the illumination coincides with the NA of the objective.
4. The method according to claim 1 wherein in step 2, a color leakage correction method is used to represent the detector signal measured in the color channel as the sum of light of the desired color and light of other colors, i.e., the measured signal in the red r, green g, and blue b channels as:
5. The method according to claim 1 wherein in step 4 by solving the PTF for illumination in a single direction, and obtaining the solution expression for PTF as:
PTF.sub.r(ρ,θ)=sin(α.sub.r)sin(θ+θ.sub.r) PTF.sub.g(ρ,θ)=sin(α.sub.g)sin(θ+θ.sub.g) PTF.sub.b(ρ,θ)=sin(α.sub.b)sin(θ+θ.sub.b) ( θ.sub.r=θ.sub.g−120° ,θ.sub.b=θ.sub.g+120° ) β.sub.r,β.sub.g,β.sub.b is determined by the NA.sub.obj of the objective lens and the illumination wavelength λ.sub.r,λ.sub.g,λ.sub.b which is obtained by solving for the following equation:
6. The method according to claim 2, wherein in step one, the illumination pattern is a semi-annular illumination of three colors of red r, green g, and blue b with an asymmetric axis difference of 120° , wherein the NA of the semi-annular illumination is equal to the NA of the objective, and the illumination intensity is distributed sinusoidally, assuming that the NA of the objective is expressed as NA.sub.obj ;the illumination function for color multiplexing expressed in polar coordinates is:
S.sub.r(ρ,θ)=δ(ρ—NA.sub.obj)sin(θ+θ.sub.r) S.sub.g(ρ,θ)=δ(ρ—NA.sub.obj)sin(θ+θ.sub.g) S.sub.b(ρ,θ)=δ(ρ—NA.sub.obj)sin(θ+θ.sub.b) ( θ.sub.y+θ.sub.g−120 ° , θ.sub.b=θ.sub.g+120 ° ) where S.sub.r(ρ,θ), S.sub.g(ρ,θ), S.sub.b(ρ,θ) denotes the illumination functions corresponding to the three wavelengths of red r, green g. and blue b, respectively, ρ, θ denotes the radius and polar angle of the polar coordinate system, respectively, θ.sub.r,θ.sub.g,θ.sub.b is the angle of the asymmetry axis of the illumination patterns of red r, green g, and blue b, respectively, and δ(ρ—NA.sub.obj) denotes the shape of the illumination pattern is an annulus where the NA of the illumination coincides with the NA of the objective.
7. The method according to claim 2, wherein in step 2, a color leakage correction method is used to represent the detector signal measured in the color channel as the sum of light of the desired color and light of other colors, i.e., the measured signal in the red r, green g, and blue b channels as:
8. The method according to claim 2, wherein in step 4 by solving the PTF for illumination in a single direction, and obtaining the solution expression for PTF as:
PTF.sub.r(ρ,θ)=sin(α.sub.r)sin(θ+θ.sub.r) PTF.sub.g(ρ,θ)=sin(α.sub.g)sin(θ+θ.sub.g) PTF.sub.b(ρ,θ)=sin(α.sub.b)sin(θ+θ.sub.b) ( θ.sub.r=θ.sub.g−120° ,θ.sub.b=θ.sub.g+120° ) β.sub.r,β.sub.g,β.sub.b of the objective lens and the illumination wavelength λ.sub.r,λ.sub.g,λ.sub.b which is obtained by solving for the following equation:
Description
Figures
[0012]
[0013]
[0014]
[0015]
[0016]
[0017]
Specific Implementation
[0018] The experimental platform of the present invention for DPC quantitative phase microscopy imaging method based on color multiplexed illumination can be built based on any commercial microscope system by simply adding a light source modulation module color LCD display illumination or programmable color LED array to the optical path. The schematic diagram of the microscope system is shown in
[0019] If LED arrays are used for system illumination, the drive implementation circuit to light up each of the point sources can be implemented using (but not limited to) existing technologies such as microcontrollers, ARM, or programmable logic devices, and the specific implementation methods can be found in the references (Baozeng Guo, Chunmiao Deng: FPGA-based LED display control system design [J]. Liquid Crystal and Display, 2010, 25(3):424-428). If the LCD display is used for system illumination, the LCD is used to replace the aperture diaphragm under the condenser lens in the original microscope. The illumination pattern of the invention is used as a spatial light filter. The technology used in the driving circuit is basically the same as that of the LED array, and the specific implementation method can be found in the references (Lin, F., Zhang, W. W.: Rheinberg illumination microscopy principle and system based on programmable LCD. Design. Journal of Optics, 2016, 8:237-243).
[0020] Combined with
[0021] Step 1, image acquisition under color multiplexed illumination: a high-contrast LCD TFT-LCD or a high-density programmable color LED array is controlled by a computer through a serial port to display a color multiplexed illumination pattern illumination sample, as shown in
[0022] Assuming that the NA of the objective is expressed as NA.sub.obj, the illumination function for color multiplexing expressed in polar coordinates is:
S.sub.r(ρ,θ)=δ(ρ—NA.sub.obj)sin(θ+θ.sub.r) S.sub.g(ρ,θ)=δ(ρ—NA.sub.obj)sin(θ+θ.sub.g) S.sub.b(ρ,θ)=δ(ρ—NA.sub.obj)sin(θ+θ.sub.b) ( θ.sub.r+θ.sub.g−120 ° , θ.sub.b=θ.sub.g+120 ° )
where S.sub.r(ρ,θ) , S.sub.g(ρ,θ) , S.sub.b(ρ,θ) denotes the illumination functions corresponding to the three wavelengths of red, green, and blue, respectively. ρ, θ denotes the radius and polar angle of the polar coordinate system, respectively, and δ(ρ—NA.sub.obj) is the angle of the asymmetry axis of the illumination pattern of the three colors of red, green, and blue, respectively. The shape of the illumination pattern is expressed as a annulus of illumination NA matched with the NA of the objective lens. From this illumination function, it can be seen that the design of the present invention is satisfied as long as the angle of the three wavelength illumination patterns is 120° . A synchronous trigger signal is generated to the color camera while sending the illumination control, and then a color sample image is captured, as shown in
[0023] If the alternating illumination strategy is used, a second color sample image will be acquired by rotating the illumination pattern by 90° in any direction after the above acquisition process is completed, noted as I.sub.c,⊥. A single shot of color multiplexed illumination pattern is used as pattern 1, which is rotated by 90° in any direction as pattern 2. Two images are acquired using these two alternating illuminations. Phase recovery can be achieved using these two images to obtain a completely isotropic PTF and achieve a completely isotropic imaging resolution.
[0024] Step 2, image color channel separation and correction: The color sample images are separated to three channels and corrected for color leakage to obtain the sample intensity images corresponding to the red, green, and blue channels I.sub.r,corr, I.sub.g,corr, I.sub.b,corr, as shown in
[0025] If the alternating illumination strategy is adopted, the two color sample images are acquired and the channel separation and correction are performed separately to obtain the sample intensity images of red, green and blue channels corresponding to the two images respectively I.sub.r,corr, I.sub.g,corr, I.sub.b,corr, I.sub.r,⊥,corr, I.sub.g,⊥,corr, I.sub.b,⊥,corr.
[0026] Since color LCDs or LEDs usually have a wide emission spectrum, and for most color image sensors, the spectral response of different color channels cannot be completely isolated. Therefore, the light of one color in the illumination may leak into other color channels and be detected by other color channels of the camera, which means that the single-channel image of a color sensor is actually a mixture of different channels. In color multiplexed illumination, the illumination light with three channels simultaneously illuminate the sample to acquire a color image, color leakage becomes more apparent due to the overlap of emission spectra (part of the spectral response of green light overlaps spectrally with the blue and red channels). Directly using the image after separating the channels for DPC phase recovery, the color leakage will lead to severe phase estimation errors. To alleviate the phase error due to color leakage, the present invention employs a color leakage correction method that represents the detector signal measured in the color channel as the sum of the light of the desired color and the light of other colors. In other words, the measured signals in the red, green, and blue channels can be written as:
[0027] where I.sub.r, I.sub.g, I.sub.b is the signal intensity of the red, green, and blue channels measured by the camera sensor, i.e., the intensity images of the red, green, and blue channels obtained by direct channel separation. I.sub.r,corr, I.sub.g,corr, I.sub.b,corr is the light intensity of the red, green, and blue channels incident on the camera sensor, i.e., the intensity of the image that should be brought into phase recovery after correction. The element R.sup.n .sub.m represents the detection response of the m(m =r,g,b) color channel of the camera to LED light of color n(n=r,g,b).
[0028] The purpose of the color leakage correction is to obtain the value of each R.sup.n.sub.m so that I.sub.r,corr, I.sub.g,corr, I.sub.b,corr can be obtained from the image I.sub.c acquired by the camera .The specific correction scheme is to use a single color l(l=r,g,b) LED for illumination without placing the sample, respectively, and use the color camera to acquire the corresponding placement of the color image I.sub.c,l′. This image I.sub.c,l′ can be separated to obtain three images with different channels I.sub.l,r′, I.sub.l,g′, I.sub.l,b′. The mean values of the intensity of the three channels of the image are calculated separately, and the mean value of illumination l are used as a standard to normalize the mean values of other two channels, obtaining R.sup.l.sub.m(m=r,g,b). Implementing this process for all three channels, and we will get all R.sup.n.sub.m values. Once the acquisition image with the sample is obtained, the color image can be separated to obtain I.sub.r, I.sub.g, I.sub.b. Then, the corrected light intensity image I.sub.r,corr, I.sub.g,corr, I.sub.b,corr for each wavelength can be obtained according to the following equation:
[0029] The three images are used to calculate the spectral response matrix of the camera, which can be used to subsequently correct the images and effectively address the phase reconstruction errors caused by color leakage. This spectral response matrix reconfiguration is calculated only once for the same imaging system.
[0030] Step 3, DPC image spectrum generation: Fourier transform is performed on the three channels of I.sub.r,corr, I.sub.g,corr, I.sub.b,corr to obtain the spectrum distribution of the three images. In order to eliminate the influence of the background term, the value at zero frequency of the three spectrum is set as 0 to obtain the spectrum distribution of the DPC image of the sample under three channels, they are expressed as ,
,
.
[0031] If the alternating illumination strategy is used, the single channel sample intensity image I.sub.r,corr, I.sub.g,corr, I.sub.b,corr , ,
,
, corresponding to the two acquired images will be solved for their spectrum separately and the zero frequency of the spectrum will be removed for the elimination of the background term.
[0032] Step 4, PTF calculation: Based on the weak phase approximation condition, the PTF PTF.sub.r(ρ,θ) ,PTF.sub.g(ρ,θ) , PTF.sub.b(ρ,θ) corresponding to different wavelengths are calculated according to the parameters of the illumination function and the objective lens.
[0033] If the alternating illumination strategy is used, the PTFs corresponding to both illumination patterns need to be solved PTF.sub.r(ρ,θ) , PTF.sub.g(ρ,θ) , PTF.sub.b(ρ,θ), PTF.sub.r,⊥(ρ,θ) ,PTF.sub.g,⊥(ρ,θ) , PTF.sub.b,⊥(ρ,θ)
[0034] As shown in
[0035] According to this calculated expression, the illumination function is brought into the calculation of red, green, blue three wavelengths corresponding to the transfer function, to obtain:
PTF.sub.r(ρ,θ)=sin(α.sub.r)sin(θ+θ.sub.r) PTF.sub.g(ρ,θ)=sin(α.sub.g)sin(θ+θ.sub.g) PTF.sub.b(ρ,θ)=sin(α.sub.b)sin(θ+θ.sub.b) ( θ.sub.r=θ.sub.g −120° ,θ.sub.b=θ.sub.g+120° )
[0036] Here α.sub.r, α.sub.g, α.sub.b, is determined by the NA.sub.obj of the objective and the illumination wavelength λ.sub.r, λ.sub.g, λ.sub.b which can be obtained by solving for the following equation:
[0037] Step 5, sample quantitative phase recovery: according to PTF.sub.r(ρ,θ) , PTF.sub.g(ρ,θ) , PTF.sub.b(ρ,θ) of different wavelengths and the DPC image spectrum ,
,
, the Tikhonov criterion is used for the inverse convolution calculation to obtain the high-resolution spectrum of the sample phase. Then, the inverse Fourier transform is performed on this high-resolution spectrum to obtain the quantitative phase distribution ϕof the sample.
[0038] Here k denotes different wavelength channels, red, green, blue. PTF.sup..Math..sub.k(ρ,θ) denotes the conjugate distribution of PTF.sub.k(ρ,θ) . λ.sub.k/λ denotes the wavelength normalization coefficient. Because the phase and wavelength are inversely proportional, so in the color multiplexed illumination, it is necessary to normalize the wavelength to get a uniform phase distribution, here λ denotes the normalized wavelength, which can be chosen as any wavelength. The blue illumination wavelength is chosen as the normalized wavelength. β is the normalization parameter, generally choose a smaller value, such as 0.01.
[0039] If the alternating illumination strategy is used, the sample image spectrum and transfer function corresponding to the two illumination patterns should be brought into the deconvolution calculation, then the quantitative phase distribution ϕ of the sample is solved by:
[0040] To compare the imaging performance of the present invention,
[0041] In order to verify the high resolution, high stability, and high contrast of the single-shot DPC QPI results based on color multiplexed illumination of the present invention, a comparison experiment was conducted using a standard USAF phase resolution target as a sample with uniform circular illumination and the method of the present invention. The experimental results are shown in
[0042]
[0043] In order to verify the real-time dynamic imaging performance of the single-shot DPC QPI based on color multiplexed illumination of the present invention, a living cell dynamic experiment was performed on cervical cancer (Hela) cells cultured in vitro for 1.65h. The cells were placed in a suitable culture environment, and the reconstructed experimental phase results are shown in