Quantitative phase imaging method based on differential phase contrast with optimal lighting pattern design
11487096 · 2022-11-01
Assignee
Inventors
- Qian Chen (Nanjing, CN)
- Chao Zuo (Nanjing, CN)
- Yao FAN (Nanjing, CN)
- Jiasong Sun (Nanjing, CN)
- Jiaji LI (Nanjing, CN)
- Shijie Feng (Nanjing, CN)
- Yuzhen Zhang (Nanjing, CN)
Cpc classification
G02B21/365
PHYSICS
H04N23/74
ELECTRICITY
International classification
G02B21/36
PHYSICS
Abstract
The patent discloses a differential phase contrast (DPC) quantitative phase microscopy method based on the optimal illumination pattern design. Firstly, the optimal illumination pattern corresponding to the isotropic phase transfer function of DPC quantitative phase imaging is derived, which is determined as a semi-annular illumination pattern with the illumination numerical aperture NA.sub.ill equal to the numerical aperture NA.sub.obj of the objective lens. The illumination intensity distribution varies with the cosine of the illumination angle, and it can be expressed as S(θ)=cos(θ). This patent effectively compensates for the frequency loss of phase transfer, not only the high-frequency responses of PTF are enhanced, but also the transfer responses of low-frequency phase information is significantly improved. As a result, the optimal illumination scheme ensures the correctness and achieves high resolution phase reconstruction, while the number of illuminations is reduced to a minimum of two, which greatly increases the imaging speed, allowing for real-time dynamic, high-correctness, high-resolution phase imaging results.
Claims
1. A differential phase contrast (DPC) quantitative phase imaging (QPI) method based on an optimal lighting pattern design, comprising: a step 1 of designing the optimal lighting pattern, comprising the step of: deriving a lighting pattern corresponding to an isotropic phase transfer function (PTF) in the DPC QPI to be the optimal lighting pattern, wherein the optimal lighting pattern is a semi-annular lighting with a lighting numerical aperture (NA) NA.sub.ill equal to an objective NA NA.sub.obj, and wherein when the NA of an objective is expressed as NA.sub.obj a distance between a single LED on a lighting annulus and a light axis is d, and an angle between a light emitted by the single LED and an optical axis is a.sub.i, then the lighting NA.sub.ill obtained by the single LED on an annular pattern satisfies the following equation:
S(θ)=cos(θ) where the angle θ increases in a clockwise direction; a step 2 of capturing original images under optimal lighting, comprising: using computer-controlled LCD or high-density programmable LED arrays to display four optimal lighting patterns under a two-axis lighting generating synchronous trigger signals to a camera to acquire an image under each of the four optical lighting patterns; and recording a total of four original images I.sub.l, I.sub.r, I.sub.u, and I.sub.d, wherein I.sub.l and I.sub.r represent two images in left and right axis directions, and I.sub.u and I.sub.d represent two images in up and down axis directions; a step 3 of calculating, according to a formula for DPC imaging, a phase gradient image of a specimen in a left-right direction I.sub.lr=(I.sub.l−I.sub.r)/(I.sub.l+I.sub.r) and an up-down direction I.sub.ud=(I.sub.u−I.sub.d)/(I.sub.u+I.sub.d) under optimal lighting; a step 4 of solving a phase transfer function (PTF) of the optimal lighting pattern, wherein the PTF of optimal lighting pattern PTF.sub.lr(u) and PTF.sub.ud (u) are isotropic, and wherein in the polar coordinate system, when r.sub.a is a distance from any point to a center zero frequency in a PTF distribution, and r.sub.NA is a distance from a cut-off frequency to the center zero frequency in a frequency domain under a partially coherent imaging, then the PTF in the left-right direction is obtained as follows:
2. The method according to claim 1, wherein a process of determining the intensity distribution of the optimal light pattern in the step 1 is characterized as follows: in polar coordinates, the PTF is defined as PTF(r,θ), where r denotes a distance from a point to a center point of the PTF distribution, θ denotes an angle between the point and an axis direction; achieving an isotropic PTF requires a minimum of two axes of lighting which are required to satisfy:
|PTF.sub.lr(r,θ)|.sup.2+|PTF,.sub.ud(r,θ)|.sup.2=C.sub.r, where PTF.sub.lr(r,θ) and PTF.sub.ud(r, θ) denote the PTFs in the left-right and up-down axis directions, respectively, C.sub.r is a constant only related to r; in polar coordinates, the PTF is a periodic function and thus can perform a Fourier series expansion to yield the following expression:
PTF.sub.lr(r,θ)=a.sub.n cos(nθ) or PTF.sub.lr(r,θ)=b.sub.n sin(nθ), wherein the two lighting axes of DPC are perpendicular to each other, and the PTFs thereof differ by π/2rad; and wherein when PTF.sub.lr(r,θ)=a.sub.n cos(nθ), PTF.sub.ud(r,θ)=b.sub.n sin(nθ), and when PTF.sub.lr(r,θ)=b.sub.n sin(nθ), PTF.sub.ud(r,θ) cos(nθ); when using a light source with a lighting NA equal to the objective NA.sub.obj, an intensity distribution of the light source is expressed as S(θ) which, in the polar coordinate system, is required to satisfy three constraints: (a) S(θ) is an even function about the direction axis; (b) S(θ) is a periodic function about the angle θ; (c) S(θ) is symmetric about a point (π/2,0), and the intensity distribution of the light source is obtained by:
S(θ)=k.sub.n cos(nθ) where k.sub.n, is an arbitrary constant, and the optimal lighting pattern is an annular lighting whose lighting intensity varies with the cosine of the lighting angle; and in a process of phase recovery, considering a minimum number of over-zero points, an intensity distribution of the optimal lighting pattern is determined as:
S(θ)=cos(θ), wherein the angle θ increases in a clockwise direction.
3. The method according to claim 1, wherein the step 2 further comprises using a LCD or high-density LED array to generate the optimal lighting pattern, wherein in a LCD-based system, a lighting system consists of the light source and the LCD, and the LCD is used to adjust the lighting pattern and wherein in a LED-based system, the LED array is considered as the lighting source to directly display optimal lighting pattern by computer control.
4. The method according to claim 1, wherein the derivation of the optimal lighting in the step 4 further comprises: imaging a thin pure phase specimen with a complex transmittance function of t(r)=e.sup.−a(r)+iϕ(r) which is illuminated by a single angle tilted lighting of S(u.sub.j), where r=(x, y) indicates coordinates of a specimen plane, ϕ(r) represents a phase of the specimen, u.sub.j represents a phase shift corresponding to a single angular lighting in the frequency domain; introducing a weak phase approximation in order to analyze the specimen phase and intensity independently, and to simplify a complex amplitude distribution of the specimen as t(r)=1+a(r)+iϕ(r), wherein Fourier transform of the complex amplitude distribution in a camera plane can be expressed as:
W.sub.j(u)=√{square root over (S(u.sub.j))}[δ(u−u.sub.j)−A(u−u.sub.j)+iΦ(u−u.sub.j)]P(u) where u denotes a frequency component in the frequency domain, A(u) denotes an amplitude distribution of the specimen in the frequency domain, Φ(u) denotes a phase distribution of the specimen in the frequency domain; P(u) is a pupil function of an objective lens; obtaining a spectral distribution of an intensity image acquired by the camera under a single angular tilt lighting
I.sub.j(u)=W.sub.j(u).Math.W.sub.j*(u)=S(u.sub.j)δ(u)|P(u)|.sup.2−S(u.sub.j)A(u)[P*(u.sub.j)P(u+u.sub.j)+P(u.sub.j)P*(u+u.sub.j)]+iS(u.sub.j)Φ(u)[P*(u.sub.j)P(u+u.sub.j)−P(u.sub.j)P*(u+u.sub.j)] calculating the intensity distribution in the frequency domain under complex lighting, wherein the intensity spectrum distribution can be divided into three terms (ignoring higher-order convolution terms of a computational process):
I(u)=Bδ(u)+A(u)ATF(u)+iΦ(u)PTF(u) where B represents a background term B=∫∫S(u.sub.j)|P(u.sub.j)|.sup.2d.sup.2u.sub.j; ATF(u) denotes an amplitude transfer function, and PTF(u) denotes the PTF; wherein a general expression for the PTF under complex lighting, applicable to all lighting patterns light pupil functions, is expresses as follows:
PTF(u)=∫∫S(u.sub.j)[P*(u.sub.j)P(u+u.sub.j)−P(u.sub.j)P*(u−u.sub.j)]d.sup.2u.sub.j wherein DPC uses asymmetric lighting patterns in up-down and left-right axis directions to illuminate the specimen, such that a tilted lighting introduces a phase factor that converts a invisible specimen phase into a measurable intensity, wherein a simple differential calculation procedure is used to highlight the phase contrast of the specimen, and the corresponding PTF is expressed as:
5. The method according to claim 1, wherein the step 5 further comprises a phase recovery under optimal lighting by performing a deconvolution (Tikhonov criterion) using two-axis PTF including PTF.sub.lr(u) and PTF.sub.ud(u), and corresponding phase gradient images to obtain the final reconstructed quantitative phase results:
6. The method according to claim 5, wherein in the phase recovery under optimal lighting, when solving a quantitative phase of the specimen under the Tikhonov criterion, a regularization parameter β is a zero value.
Description
(1)
(2)
(3)
(4)
(5) SPECIFIC IMPLEMENTATION
(6) The actual hardware platform of this invention is a microscopic imaging system based on a high-density programmable LED array or LCD display. The entire system includes an industrial camera for image acquisition, a microscope objective, a sample, a carrier table, and a programmable LED array or LCD display as the microscope lighting source. The LED array or LCD display is positioned under the carrier table at a spacing H typically between 20-100 mm and centered on the optical axis of the microscope system. The LED array or LCD display includes a number of point light sources, which are regularly aligned to form a two-dimensional matrix. Each point source can be illuminated in red, green, and blue, with typical wavelengths of 635 nm for red, 525 nm for green, and 475 nm for blue. Each point source has a typical center spacing d of 1-10 mm, and the LED array or LCD display is fixed in position by a fixed substrate.
(7) If LED arrays are used for lighting, the implementation circuit to drive the LED arrays to light each of the point sources can be used existing technologies such as microcontrollers, ARM, or programmable logic devices. The specific implementation method can be found in related paper (Guo, Bao-Zeng, and Chun-Miao Deng. “Design of LED Display Control System Based on FPGA.” Chinese Journal of Liquid Crystals and Displays 25.3 (2010): 424-428).
(8) If the LCD display is used for lighting, it requires the microscope's own halogen lamp and other light sources at the bottom of the LCD display to provide raw lighting, and the LCD is introduced to replace the aperture diaphragm of condenser lens. By displaying different patterns as the spatial light filter, the lighting is modulated to different shapes and color distribution. The technology used in the driving circuit of LCD is basically the same as the LED array, and the specific implementation method can be referred to the related literature 201510631692.4.
(9) From
(10) Step 1, design optimal lighting pattern
(11) The design of the optimal lighting pattern for DPC QPI is to derive the lighting pattern of the isotropic PTF. It can be divided into the following two major steps, that is, for the determination of the optimal lighting shape and the lighting intensity distribution.
(12) (1) The shape of the optimal lighting pattern is determined by the following derivation process:
(13) First, the outer diameter of the optimal lighting pattern is determined, i.e., the maximum lighting angle. In order to research the relationship between the lighting shape and the PTF, the PTF under different aperture lightings are simulated.
(14) Next, the inner diameter of the optimal lighting pattern is further determined by analyzing the PTF. In
(15) Based on the above analysis, it can be determined that the shape of the optimal lighting pattern in DPC should be semi-annular lighting, and its NA of the lighting NA.sub.ill should be equal to the NA of the objective lens NA.sub.obj. The transfer response of any lighting intensity distribution will be enhanced when such a semi-annular lighting is adopted. Thus, it can be determined that if the NA of the objective used for DPC QPI is NA.sub.obj, the radius of the optimal lighting annulus is d, the distance between the LEDs and the specimen is h, and the angle between the light emitted by a single LED and the optical axis is α.sub.i. The optimal lighting pattern should be satisfy:
(16)
(2) The process of determining the intensity distribution of the optimal lighting pattern is as follows:
(17) In polar coordinates, the PTF is defined as PTF (r,θ), where r denotes the distance from a point to the center point of the PTF distribution, θ denotes the angle between the point and the axis direction. Achieving an isotropic PTF requires a minimum of two axes of lighting, and they are required to satisfy:
|PTF.sub.lr(r,θ)|.sup.2+|PTF.sub.ud(r,θ)|.sup.2C.sub.r
where PTF.sub.lr(r,θ) and PTF.sub.ud(r,θ) denote the PTFs in the left-right and up-down axis directions, respectively. C.sub.r is a constant only related to r.
(18) In polar coordinates, the PTF is a periodic function, so it can perform the Fourier series expansion to yield the following expression:
(19)
where a.sub.n and b.sub.n in the coefficient terms independent of r and θ, and n is an arbitrary positive integer. To satisfy |PTF.sub.lr(r,θ)|.sup.2+|PTF.sub.ud(r,θ)|.sup.2C.sub.r, |PTF.sub.lr(r,θ)|.sup.2 and |PTF.sub.ud(r,θ)|.sup.2 cannot produce any cross term, then the expression of PTF.sub.lr(r,θ) can be obtained as follows:
PTF.sub.lr(r,θ)=a.sub.n cos(nθ) or PTF.sub.lr(r,θ)=b.sub.n sin(nθ)
The two lighting axes of DPC are perpendicular to each other, so their PTFs differ by π/2 rad. Therefore, when PTF.sub.lr(r,θ)=a.sub.n cos(nθ), we will get PTF.sub.ud(r,θ)=b.sub.n sin(nθ), and when PTF.sub.lr(r,θ)=b.sub.n sin(nθ), we will get PTF.sub.ud(r,θ)=a.sub.n cos(nθ).
(20) When using a light source with a lighting NA equal to the objective NA.sub.obj, the intensity distribution of the light source is expressed as S(θ). In the polar coordinate system, S(θ) is required to satisfy three constraints: (a) S(θ) is an even function about the direction axis; (b) S(θ) is a periodic function about the angle θ; (c) S(θ) is symmetric about the point (π/2,θ).
(21) The PTF response of any point can be obtained by integrating the lighting intensity corresponding to the red arc in
PTF.sub.lr(r,θ)=∫.sub.θ−α.sup.θ+αS(θ)dθ
α denotes half of the circular angle corresponding to the red arc. It can be obtained that:
S(θ)=k.sub.n cos(nθ)
where k.sub.n is an arbitrary constant. From this equation, the optimal lighting pattern is an annular lighting whose lighting intensity varies with the cosine of the lighting angle.
(22) In the process of phase recovery, considering the minimum number of over-zero points, the intensity distribution of the optimal lighting pattern should be determined as:
S(θ)=cos(θ)
here the angle β increases in a clockwise direction.
(23) Step 2, capture original images under optimal lighting:
(24) computer-controlled LCD or high-density programmable LED arrays are used to display four optimal lighting patterns under two-axis lighting, and generate synchronous trigger signals to the camera to acquire an image under each pattern, thus a total of four original images are recored, i.e., I.sub.l, I.sub.r, I.sub.u, and I.sub.d. Here, I.sub.l and I.sub.r represent two images in the left and right axis directions, and I.sub.u and I.sub.d represent two images in the up and down axis directions.
(25) LCD or high-density LED array can be used to generate the optimal lighting pattern. In the LCD-based system, the lighting system consists of the light source and LCD. The LCD is used to adjust the lighting pattern. In the LED-based system, the LED array is considered as the lighting source to directly display optimal lighting pattern by computer control.
(26) Step 3: calculate phase gradient image:
(27) According to the formula for DPC imaging, the phase gradient image of the specimen in the left-right direction I.sub.lr=(I.sub.l−U.sub.r)/(I.sub.l+I.sub.r) and the phase gradient image of the specimen in the up-down direction I.sub.ud=(I.sub.u−I.sub.d/(I.sub.u+I.sub.d) are calculated under optimal lighting.
(28) Step 4, solve the phase transfer function (PTF) of the optimal lighting pattern:
(29) Based on the weak phase approximation, the PTFs in the two axis directions in the optimal lighting pattern DPC PTF.sub.lr(u) and PTF.sub.ud(u) are derived. Consider imaging a thin pure phase specimen with a complex transmittance function of t(r)=e.sup.−a(r)+iϕ(r), it is illuminated by a single angle tilted lighting of S(u.sub.j), where r=(x, y) indicates the coordinates of the specimen plane, ϕ(r) represents the phase of the specimen. u.sub.j represents the phase shift corresponding to a single angular lighting in the frequency domain. In order to analyze the specimen phase and intensity independently, the weak phase approximation is introduced to simplify the complex amplitude distribution of the specimen as t(r)=1+a(r)+iΦ(r). Then, the Fourier transform of the complex amplitude distribution in the camera plane can be expressed as:
W.sub.j(u)=√{square root over (S(u.sub.j))}[δ(u−u.sub.j)−A(u−u.sub.j)+iΦ(u−u.sub.j)]P(u)
u denotes the frequency component in the frequency domain, A(u) denotes the amplitude distribution of the specimen in the frequency domain, Φ(u) denotes the phase distribution of the specimen in the frequency domain. P(u) is the pupil function of the objective. The spectral distribution of the intensity image acquired by the camera under a single angular tilt lighting is obtained by convolving it with its conjugate term:
I.sub.j(u)=W.sub.j(u).Math.W.sub.j*(u)=S(u.sub.j)δ(u)|P(u)|.sup.2−S(u.sub.j)A(u)[P*(u.sub.j)P(u+u.sub.j)+P(u.sub.j)P*(u+u.sub.j)]+iS(u.sub.j)Φ(u)[P*(u.sub.j)P(u+u.sub.j)−P(u.sub.j)P*(u+u.sub.j)]
(30) Calculating the intensity distribution in the frequency domain under complex lighting, the intensity spectrum distribution can be divided into three terms (ignoring the higher-order convolution terms of the computational process):
I(u)=Bδ(u)+A(u)ATF(u)+iΦ(u)PTF(u)
B represents the background term B=∫∫S(u.sub.j)|P(u.sub.j)|.sup.2d.sup.2u.sub.j; ATF(u) denotes the amplitude transfer function, and PTF(u) denotes the PTF. Where
PTF(u)=∫∫S(u.sub.j)[P*(u.sub.j)P(u+u.sub.j)−P(u.sub.j)P*(u−u.sub.j)]d.sup.2u.sub.j
This is a general expression for the PTF under complex lighting, applicable to all lighting patterns light pupil functions.
(31) DPC uses asymmetric lighting patterns in the up-down and left-right axis directions to illuminate the specimen, such that the tilted lighting introduces a phase factor that converts the invisible specimen phase into a measurable intensity. A simple differential calculation procedure is used to highlight the phase contrast of the specimen, and the corresponding PTF is expressed as:
(32)
Where PTF.sub.lr(u) denotes the PTF in the left-right axis directions, S.sub.lr(u,) denotes the lighting in the left-right direction. The PTF in the up-down direction PTF.sub.ud(u) is derived as above process.
(33) In the optimal lighting pattern, the PTF of DPC imaging is required be isotropic. In the polar coordinate system, let r.sub.a be the distance from any point to the center zero frequency in the PTF distribution, and r.sub.NA be the distance from the cut-off frequency to the center point in the frequency domain under the partially coherent imaging, then the PTF in the left-right direction is obtained as follows:
(34)
The PTF in the up-down direction is
(35)
Their intensity distribution is obtained as
(36)
(37) 5. Step 5, solve the quantitative phase of the specimen:
(38) Perform deconvolution (Tikhonov criterion) using the obtained two-axis PTF PTF.sub.lr(u) and PTF.sub.ud(u), and the corresponding phase gradient mage to obtain the final reconstructed quantitative phase results:
(39)
ϕ(r) is the reconstructed quantitative phase of the sample. The regularization parameter β in this equation is used to suppress the error caused by excessive amplification by noise. In the optimal lighting pattern, since the PTF is greatly enhanced in entire partially coherent imaging range (from 0 to 2 NA.sub.obj), the Tikhonov criterion can be calculated without considering the regularization parameter to obtain the anti-noise reconstructed quantitative phase. Thus, the regularization parameter β is usually set as a very small or zero value.
(40) In order to verify the imaging performance of DPC QPI under the optimal lighting pattern, we simulated the PTFs under four different lighting patterns. As shown in
(41) To further simulate the imaging performance of the optimal lighting patterns in a real system, we added the same level of Gaussian noise to the raw images to simulate the imaging in a real situation. The regularization parameter β is used to suppress the system noise during the deconvolution with the Tikhonov criterion.
(42) Finally, we verified the imaging performance of DPC QPI in optimal lighting pattern by an experiment on unstained Hela cells cultured in vitro. Raw experimental data were collected using an objective with 10×, 0.4 NA, and the recovered quantitative phase is shown in