HIGH-ILLUMINATION NUMERICAL APERTURE-BASED LARGE FIELD-OF-VIEW HIGH-RESOLUTION MICROIMAGING DEVICE AND A METHOD FOR ITERATIVE RECONSTRUCTION
20210112187 · 2021-04-15
Assignee
Inventors
- Qian Chen (Nanjing, CN)
- Chao Zuo (Nanjing, CN)
- Jiasong Sun (Nanjing, CN)
- Shijie Feng (Nanjing, CN)
- Yuzhen Zhang (Nanjing, CN)
- Guohua Gu (Nanjing, CN)
Cpc classification
G02B21/33
PHYSICS
G02B21/367
PHYSICS
International classification
G02B21/33
PHYSICS
G02B21/36
PHYSICS
Abstract
A high-illumination numerical aperture-based large field-of-view high-resolution microimaging device, and a method for iterative reconstruction, the device comprising an LED array (1), a stage (2), a condenser (3), a microscopic objective (5), a tube lens (6), and a camera (7), the LED array (1) being arranged on the forward focal plane of the condenser (3). Light emitted by the i-th lit LED unit (8) of the LED array (1) passes through the condenser (3) and converges to become parallel light illuminating a specimen (4) to be examined, which is placed on the stage (2); part of the diffracted light passing through the specimen (4) is collected by the microscopic objective (5), converged by the tube lens (6), and reaches the imaging plane of the camera (7), forming an intensity image recorded by the camera (1). The present device and method ensure controllable programming of the illumination direction, while also ensuring an illumination-numerical-aperture up to 1.20 and thus achieving a reconstruction resolution up to 0.15 μm.
Claims
1. A high-illumination numerical aperture-based large field-of-view high-resolution microimaging device, wherein the device comprises an LED array (1), a stage (2), a condenser (3), a microscopic objective (5), a tube lens (6), and a camera (7), the LED array (1) being arranged on the forward focal plane of the condenser (3), and the center of the LED array (1) is on the optical axis of the microscope objective (5); The back focal plane of the microscope objective (5) coincides with the forward focal plane of the tube lens (6), and the imaging plane of the camera (7) is placed on the back focal plane of the tube lens (6), and during imaging, a specimen (4) to be examined on the stage (2) is adjusted to the front focal plane position of the microscope objective (5) to form an infinity-corrected imaging system; Light emitted by the i-th lit LED unit (8) of the LED array (1) passes through the condenser (3) and converges to become parallel light illuminating a specimen (4) to be examined, which is placed on the stage (2); part of the diffracted light passing through the specimen (4) is collected by the microscopic objective (5), converged by the tube lens (6), and reaches the imaging plane of the camera (7), forming an intensity image recorded by the camera (7).
2. According to claim 1, the high-illumination numerical aperture-based large field-of-view high-resolution microimaging device is characterized in that microscope objective (5) has a numerical aperture of NA.sub.obj and satisfies
3. According to claim 1, the high-illumination numerical aperture-based large field-of-view high-resolution microimaging device is characterized in that a camera (7) is a color or monochrome camera, and if it is a monochrome camera, when imaging, each LED unit (8) of the LED array (1) emits red or green or blue monochromatic light respectively, recording all monochrome images sequentially by the monochrome camera; if it is a color camera, when imaging, each of the LED units (8) of the LED array (1) simultaneously emits red, green and blue light, recording all color images by the color camera.
4. According to claim 1, the high-illumination numerical aperture-based large field-of-view high-resolution microimaging device is characterized in that a microscope oil is disposed between the condenser (3) and the specimen (4) to be examined.
5. According to claim 1, a high-illumination numerical aperture-based large field-of-view high-resolution micro-imaging iterative reconstruction method is characterized in that the steps are as follows: step one, the brightness of the LED unit (8) is calibrated, using the LED array (1) as an illumination source of a high-illumination numerical aperture-based large field-of-view high-resolution microimaging device, sequentially lighting up each LED unit (8) of the LED array (1), and after illuminating the blank specimen (4), using the high magnification objective lens to collect the corresponding images and calculating the normalized brightness correction coefficient corresponding to each LED unit (8) in the three channels of red, green and blue; step two, the position of the LED array (1) is calibrated, using the resolution board as the specimen (4) to be examined, using the LED array (1) as the illumination source of a high-illumination numerical aperture-based large field-of-view high-resolution microimaging device, sequentially lighting up each LED unit (8) of the LED array (1), after illuminating the specimen (4) to be examined, images in focus and images with defocus distance h are respectively collected, and then the illumination angle corresponding to each of the three channels of red, green and blue for each LED unit (8) is calculated by using a sub-pixel registration algorithm, then determining the position of the LED array by using nonlinear regression. step three, raw images are captured, using the LED array (1) as an illumination source of a large field-of-view high-resolution micro-imaging device, sequentially lighting up each LED unit (8) of the LED array (1), after illuminating the specimen (4) to be examined, the corresponding low-resolution raw images are collected; step four, raw images are pre-processed, including threshold denoising and brightness correction, firstly using the average value of the dark current noise of the camera (7) as a threshold, performing threshold denoising on the captured low-resolution raw images, obtaining denoised low-resolution images, and then dividing each denoised low-resolution image by the normalized brightness correction coefficient obtained in step one, brightness-corrected low-resolution images are obtained; step five, high-resolution images are initialized, adding and averaging all the low-resolution bright-field images among the brightness-corrected low-resolution images, and then initializing the amplitude and phase of high-resolution images by up-sampling; step six, iterative reconstruction, all the brightness corrected low-resolution images are subjected to synthetic aperture calculation one by one in the frequency domain using the pixel binning based Fourier ptychographic iterative reconstruction method and gradually reducing the updating coefficient; The cost function value is used as a criterion, and when the cost function value is less than a given threshold, the iteration is stopped, and at this moment, the amplitude and phase of the high-resolution image are the final reconstructed large field-of-view high-resolution microscopic image.
6. According to claim 5, the high-illumination numerical aperture-based large field-of-view high-resolution microimaging iterative reconstruction method is characterized by step one: the entire LED array (1) comprises a total of N LED units, N>261, each LED unit (8) emits monochromatic light of red, green and blue, respectively, and a total of 3N low-resolution images are captured, and the image of the blank specimen lighted by the i-th LED unit with color c is captured and marked as I.sub.i,c.sup.kb(r), where i=1, 2, . . . , N, c=r, g, b, r is two-dimensional coordinates of real space r=(x,y), and then the average intensity of each image is calculated
7. According to claim 5, the high-illumination numerical aperture-based large field-of-view high-resolution microimaging iterative reconstruction method is characterized by step two: First, after the LED array (1) illuminating the specimen (4) to be examined, in-focus image I.sub.i,c.sup.focus and defocus image I.sub.i,c.sup.defocus with defocus distance h are respectively collected; according to angular spectrum diffraction theory, numerically propagating in-focus image I.sub.l,c.sup.focus corresponding to LED unit (8) in the center of LED array (1) along optical axis with numerical value h distance, thereby obtaining numerical defocus image I.sub.l,c.sup.pro; and then calculating the offset value (PY.sub.x,PY.sub.y) of each defocus image I.sub.i,c.sup.defocus relative to the numerical defocus image I.sub.l,c.sup.pro by using sub-pixel registration algorithm, the spatial frequency vector of the illumination light corresponding to the i-th LED unit is [ . . . ] indicates that the nonlinear regression operation is performed, and d is the spacing between two adjacent LED units (8) of the LED array (1), (x.sub.i, y.sub.i) represents the spatial position coordinates of the i-th LED unit, λ is the wavelength of the illumination light, and (m.sub.i, n.sub.i) is the row number and the column number corresponding to the i-th LED unit.
8. According to claim 5, the high-illumination numerical aperture-based large field-of-view high-resolution microimaging iterative reconstruction method is characterized by step four: first, sequentially lighting up the LED units and illuminating a specimen (4) to be examined by using the monochromatic light of red, green and blue, then the captured original low-resolution images are marked as I.sub.i,c; turning off all the LED units and the captured dark current noise image I.sub.dark; the average value of the dark current noise of the camera is used as a threshold, then performing threshold denoising on the captured original low-resolution images, the formula is
I.sub.i,c.sup.uni=I.sub.i,c.sup.dn/R.sub.i,c.sup.bri where I.sub.i,c.sup.uni is the low-resolution image after brightness correction, I.sub.i,c.sup.dn is the low-resolution image after performing threshold denoising, R.sub.i,c.sup.bri is the normalized brightness correction coefficient obtained in step one.
9. According to claim 5, the high-illumination numerical aperture-based large field-of-view high-resolution iterative reconstruction method is characterized by step six: the formula based on the pixel binning based Fourier ptychographic iterative method is as follows:
10. According to claim 1, a high-illumination numerical aperture-based large field-of-view high-resolution microimaging iterative reconstruction method is characterized in that the steps of reconstructing a true color image are as follows: step one, the brightness of the LED unit (8) is calibrated, using the LED array (1) as an illumination source of a high-illumination numerical aperture-based large field-of-view high-resolution microimaging device, sequentially lighting up each LED unit (8) of the LED array (1), and after illuminating the blank specimen (4), using the high magnification objective lens to collect the corresponding image and calculating the normalized brightness correction coefficient corresponding to each LED unit (8) in the three channels of red, green and blue; step two, the position of the LED array (1) is calibrated, using the resolution board as the specimen (4) to be examined, using the LED array (1) as the illumination source of a high-illumination numerical aperture-based large field-of-view high-resolution microimaging device, sequentially lighting up each LED unit (8) of the LED array (1), after illuminating the specimen (4) to be examined, in-focus images and defocus images with defocus distance h are respectively collected, and then the illumination angle corresponding to each of the three channels of red, green and blue for each LED unit (8) is calculated by using a sub-pixel registration algorithm, then determining the position of the LED array by using nonlinear regression; step three, raw images are captured, using the LED array (1) as an illumination source of a high-illumination numerical aperture-based large field-of-view high-resolution micro-imaging device, sequentially lighting up each LED unit (8) of the LED array (1), after illuminating the specimen (4) to be examined, the corresponding low-resolution raw images are collected; step four: raw images are pre-processed, including threshold denoising and brightness correction, firstly using the average value of the dark current noise of the camera (7) as a threshold, performing threshold denoising on the captured low-resolution raw images to obtain denoised low-resolution images, and then dividing each denoised low-resolution image by the normalized brightness correction coefficient obtained in step one, then brightness-corrected low-resolution images are obtained; step five, high-resolution images are initialized, adding and averaging all the low-resolution bright-field images among the brightness-corrected low-resolution images, and then initializing the amplitude and phase of the high-resolution images by up-sampling; step six, iterative reconstruction; each of the brightness corrected low-resolution images is subjected to synthetic aperture calculation one by one in the frequency domain by using pixel binning based Fourier ptychographic recovery method and the updating coefficient is gradually reduced; the cost function value is used as a criterion, and when the cost function value is less than a given threshold, the iteration is stopped. and at this moment, the amplitude and phase of the high-resolution images are the final reconstructed large field-of-view high-resolution microscopic images; step seven, color fusion; repeating step five and step six, and respectively reconstructing high-resolution images of red, green and blue channels, then three reconstructed high-resolution images are synthesized as the red, green and blue channels respectively of the final true color image.
11. According to claim 2, a high-illumination numerical aperture-based large field-of-view high-resolution micro-imaging iterative reconstruction method is characterized in that the steps are as follows: step one, the brightness of the LED unit (8) is calibrated, using the LED array (1) as an illumination source of a high-illumination numerical aperture-based large field-of-view high-resolution microimaging device, sequentially lighting up each LED unit (8) of the LED array (1), and after illuminating the blank specimen (4), using the high magnification objective lens to collect the corresponding images and calculating the normalized brightness correction coefficient corresponding to each LED unit (8) in the three channels of red, green and blue; step two, the position of the LED array (1) is calibrated, using the resolution board as the specimen (4) to be examined, using the LED array (1) as the illumination source of a high-illumination numerical aperture-based large field-of-view high-resolution microimaging device, sequentially lighting up each LED unit (8) of the LED array (1), after illuminating the specimen (4) to be examined, images in focus and images with defocus distance h are respectively collected, and then the illumination angle corresponding to each of the three channels of red, green and blue for each LED unit (8) is calculated by using a sub-pixel registration algorithm, then determining the position of the LED array by using nonlinear regression. step three, raw images are captured, using the LED array (1) as an illumination source of a large field-of-view high-resolution micro-imaging device, sequentially lighting up each LED unit (8) of the LED array (1), after illuminating the specimen (4) to be examined, the corresponding low-resolution raw images are collected; step four, raw images are pre-processed, including threshold denoising and brightness correction, firstly using the average value of the dark current noise of the camera (7) as a threshold, performing threshold denoising on the captured low-resolution raw images, obtaining denoised low-resolution images, and then dividing each denoised low-resolution image by the normalized brightness correction coefficient obtained in step one, brightness-corrected low-resolution images are obtained; step five, high-resolution images are initialized, adding and averaging all the low-resolution bright-field images among the brightness-corrected low-resolution images, and then initializing the amplitude and phase of high-resolution images by up-sampling; step six, iterative reconstruction, all the brightness corrected low-resolution images are subjected to synthetic aperture calculation one by one in the frequency domain using the pixel binning based Fourier ptychographic iterative reconstruction method and gradually reducing the updating coefficient; The cost function value is used as a criterion, and when the cost function value is less than a given threshold, the iteration is stopped, and at this moment, the amplitude and phase of the high-resolution image are the final reconstructed large field-of-view high-resolution microscopic image.
12. According to claim 3, a high-illumination numerical aperture-based large field-of-view high-resolution micro-imaging iterative reconstruction method is characterized in that the steps are as follows: step one, the brightness of the LED unit (8) is calibrated, using the LED array (1) as an illumination source of a high-illumination numerical aperture-based large field-of-view high-resolution microimaging device, sequentially lighting up each LED unit (8) of the LED array (1), and after illuminating the blank specimen (4), using the high magnification objective lens to collect the corresponding images and calculating the normalized brightness correction coefficient corresponding to each LED unit (8) in the three channels of red, green and blue; step two, the position of the LED array (1) is calibrated, using the resolution board as the specimen (4) to be examined, using the LED array (1) as the illumination source of a high-illumination numerical aperture-based large field-of-view high-resolution microimaging device, sequentially lighting up each LED unit (8) of the LED array (1), after illuminating the specimen (4) to be examined, images in focus and images with defocus distance h are respectively collected, and then the illumination angle corresponding to each of the three channels of red, green and blue for each LED unit (8) is calculated by using a sub-pixel registration algorithm, then determining the position of the LED array by using nonlinear regression. step three, raw images are captured, using the LED array (1) as an illumination source of a large field-of-view high-resolution micro-imaging device, sequentially lighting up each LED unit (8) of the LED array (1), after illuminating the specimen (4) to be examined, the corresponding low-resolution raw images are collected; step four, raw images are pre-processed, including threshold denoising and brightness correction, firstly using the average value of the dark current noise of the camera (7) as a threshold, performing threshold denoising on the captured low-resolution raw images, obtaining denoised low-resolution images, and then dividing each denoised low-resolution image by the normalized brightness correction coefficient obtained in step one, brightness-corrected low-resolution images are obtained; step five, high-resolution images are initialized, adding and averaging all the low-resolution bright-field images among the brightness-corrected low-resolution images, and then initializing the amplitude and phase of high-resolution images by up-sampling; step six, iterative reconstruction, all the brightness corrected low-resolution images are subjected to synthetic aperture calculation one by one in the frequency domain using the pixel binning based Fourier ptychographic iterative reconstruction method and gradually reducing the updating coefficient; The cost function value is used as a criterion, and when the cost function value is less than a given threshold, the iteration is stopped, and at this moment, the amplitude and phase of the high-resolution image are the final reconstructed large field-of-view high-resolution microscopic image.
13. According to claim 4, a high-illumination numerical aperture-based large field-of-view high-resolution micro-imaging iterative reconstruction method is characterized in that the steps are as follows: step one, the brightness of the LED unit (8) is calibrated, using the LED array (1) as an illumination source of a high-illumination numerical aperture-based large field-of-view high-resolution microimaging device, sequentially lighting up each LED unit (8) of the LED array (1), and after illuminating the blank specimen (4), using the high magnification objective lens to collect the corresponding images and calculating the normalized brightness correction coefficient corresponding to each LED unit (8) in the three channels of red, green and blue; step two, the position of the LED array (1) is calibrated, using the resolution board as the specimen (4) to be examined, using the LED array (1) as the illumination source of a high-illumination numerical aperture-based large field-of-view high-resolution microimaging device, sequentially lighting up each LED unit (8) of the LED array (1), after illuminating the specimen (4) to be examined, images in focus and images with defocus distance h are respectively collected, and then the illumination angle corresponding to each of the three channels of red, green and blue for each LED unit (8) is calculated by using a sub-pixel registration algorithm, then determining the position of the LED array by using nonlinear regression. step three, raw images are captured, using the LED array (1) as an illumination source of a large field-of-view high-resolution micro-imaging device, sequentially lighting up each LED unit (8) of the LED array (1), after illuminating the specimen (4) to be examined, the corresponding low-resolution raw images are collected; step four, raw images are pre-processed, including threshold denoising and brightness correction, firstly using the average value of the dark current noise of the camera (7) as a threshold, performing threshold denoising on the captured low-resolution raw images, obtaining denoised low-resolution images, and then dividing each denoised low-resolution image by the normalized brightness correction coefficient obtained in step one, brightness-corrected low-resolution images are obtained; step five, high-resolution images are initialized, adding and averaging all the low-resolution bright-field images among the brightness-corrected low-resolution images, and then initializing the amplitude and phase of high-resolution images by up-sampling; step six, iterative reconstruction, all the brightness corrected low-resolution images are subjected to synthetic aperture calculation one by one in the frequency domain using the pixel binning based Fourier ptychographic iterative reconstruction method and gradually reducing the updating coefficient; The cost function value is used as a criterion, and when the cost function value is less than a given threshold, the iteration is stopped, and at this moment, the amplitude and phase of the high-resolution image are the final reconstructed large field-of-view high-resolution microscopic image.
14. According to claim 2, a high-illumination numerical aperture-based large field-of-view high-resolution microimaging iterative reconstruction method is characterized in that the steps of reconstructing a true color image are as follows: step one, the brightness of the LED unit (8) is calibrated, using the LED array (1) as an illumination source of a high-illumination numerical aperture-based large field-of-view high-resolution microimaging device, sequentially lighting up each LED unit (8) of the LED array (1), and after illuminating the blank specimen (4), using the high magnification objective lens to collect the corresponding image and calculating the normalized brightness correction coefficient corresponding to each LED unit (8) in the three channels of red, green and blue; step two, the position of the LED array (1) is calibrated, using the resolution board as the specimen (4) to be examined, using the LED array (1) as the illumination source of a high-illumination numerical aperture-based large field-of-view high-resolution microimaging device, sequentially lighting up each LED unit (8) of the LED array (1), after illuminating the specimen (4) to be examined, in-focus images and defocus images with defocus distance h are respectively collected, and then the illumination angle corresponding to each of the three channels of red, green and blue for each LED unit (8) is calculated by using a sub-pixel registration algorithm, then determining the position of the LED array by using nonlinear regression; step three, raw images are captured, using the LED array (1) as an illumination source of a high-illumination numerical aperture-based large field-of-view high-resolution micro-imaging device, sequentially lighting up each LED unit (8) of the LED array (1), after illuminating the specimen (4) to be examined, the corresponding low-resolution raw images are collected; step four: raw images are pre-processed, including threshold denoising and brightness correction, firstly using the average value of the dark current noise of the camera (7) as a threshold, performing threshold denoising on the captured low-resolution raw images to obtain denoised low-resolution images, and then dividing each denoised low-resolution image by the normalized brightness correction coefficient obtained in step one, then brightness-corrected low-resolution images are obtained; step five, high-resolution images are initialized, adding and averaging all the low-resolution bright-field images among the brightness-corrected low-resolution images, and then initializing the amplitude and phase of the high-resolution images by up-sampling; step six, iterative reconstruction; each of the brightness corrected low-resolution images is subjected to synthetic aperture calculation one by one in the frequency domain by using pixel binning based Fourier ptychographic recovery method and the updating coefficient is gradually reduced; the cost function value is used as a criterion, and when the cost function value is less than a given threshold, the iteration is stopped. and at this moment, the amplitude and phase of the high-resolution images are the final reconstructed large field-of-view high-resolution microscopic images; step seven, color fusion; repeating step five and step six, and respectively reconstructing high-resolution images of red, green and blue channels, then three reconstructed high-resolution images are synthesized as the red, green and blue channels respectively of the final true color image.
15. According to claim 3, a high-illumination numerical aperture-based large field-of-view high-resolution microimaging iterative reconstruction method is characterized in that the steps of reconstructing a true color image are as follows: step one, the brightness of the LED unit (8) is calibrated, using the LED array (1) as an illumination source of a high-illumination numerical aperture-based large field-of-view high-resolution microimaging device, sequentially lighting up each LED unit (8) of the LED array (1), and after illuminating the blank specimen (4), using the high magnification objective lens to collect the corresponding image and calculating the normalized brightness correction coefficient corresponding to each LED unit (8) in the three channels of red, green and blue; step two, the position of the LED array (1) is calibrated, using the resolution board as the specimen (4) to be examined, using the LED array (1) as the illumination source of a high-illumination numerical aperture-based large field-of-view high-resolution microimaging device, sequentially lighting up each LED unit (8) of the LED array (1), after illuminating the specimen (4) to be examined, in-focus images and defocus images with defocus distance h are respectively collected, and then the illumination angle corresponding to each of the three channels of red, green and blue for each LED unit (8) is calculated by using a sub-pixel registration algorithm, then determining the position of the LED array by using nonlinear regression; step three, raw images are captured, using the LED array (1) as an illumination source of a high-illumination numerical aperture-based large field-of-view high-resolution micro-imaging device, sequentially lighting up each LED unit (8) of the LED array (1), after illuminating the specimen (4) to be examined, the corresponding low-resolution raw images are collected; step four: raw images are pre-processed, including threshold denoising and brightness correction, firstly using the average value of the dark current noise of the camera (7) as a threshold, performing threshold denoising on the captured low-resolution raw images to obtain denoised low-resolution images, and then dividing each denoised low-resolution image by the normalized brightness correction coefficient obtained in step one, then brightness-corrected low-resolution images are obtained; step five, high-resolution images are initialized, adding and averaging all the low-resolution bright-field images among the brightness-corrected low-resolution images, and then initializing the amplitude and phase of the high-resolution images by up-sampling; step six, iterative reconstruction; each of the brightness corrected low-resolution images is subjected to synthetic aperture calculation one by one in the frequency domain by using pixel binning based Fourier ptychographic recovery method and the updating coefficient is gradually reduced; the cost function value is used as a criterion, and when the cost function value is less than a given threshold, the iteration is stopped. and at this moment, the amplitude and phase of the high-resolution images are the final reconstructed large field-of-view high-resolution microscopic images; step seven, color fusion; repeating step five and step six, and respectively reconstructing high-resolution images of red, green and blue channels, then three reconstructed high-resolution images are synthesized as the red, green and blue channels respectively of the final true color image.
16. According to claim 4, a high-illumination numerical aperture-based large field-of-view high-resolution microimaging iterative reconstruction method is characterized in that the steps of reconstructing a true color image are as follows: step one, the brightness of the LED unit (8) is calibrated, using the LED array (1) as an illumination source of a high-illumination numerical aperture-based large field-of-view high-resolution microimaging device, sequentially lighting up each LED unit (8) of the LED array (1), and after illuminating the blank specimen (4), using the high magnification objective lens to collect the corresponding image and calculating the normalized brightness correction coefficient corresponding to each LED unit (8) in the three channels of red, green and blue; step two, the position of the LED array (1) is calibrated, using the resolution board as the specimen (4) to be examined, using the LED array (1) as the illumination source of a high-illumination numerical aperture-based large field-of-view high-resolution microimaging device, sequentially lighting up each LED unit (8) of the LED array (1), after illuminating the specimen (4) to be examined, in-focus images and defocus images with defocus distance h are respectively collected, and then the illumination angle corresponding to each of the three channels of red, green and blue for each LED unit (8) is calculated by using a sub-pixel registration algorithm, then determining the position of the LED array by using nonlinear regression; step three, raw images are captured, using the LED array (1) as an illumination source of a high-illumination numerical aperture-based large field-of-view high-resolution micro-imaging device, sequentially lighting up each LED unit (8) of the LED array (1), after illuminating the specimen (4) to be examined, the corresponding low-resolution raw images are collected; step four: raw images are pre-processed, including threshold denoising and brightness correction, firstly using the average value of the dark current noise of the camera (7) as a threshold, performing threshold denoising on the captured low-resolution raw images to obtain denoised low-resolution images, and then dividing each denoised low-resolution image by the normalized brightness correction coefficient obtained in step one, then brightness-corrected low-resolution images are obtained; step five, high-resolution images are initialized, adding and averaging all the low-resolution bright-field images among the brightness-corrected low-resolution images, and then initializing the amplitude and phase of the high-resolution images by up-sampling; step six, iterative reconstruction; each of the brightness corrected low-resolution images is subjected to synthetic aperture calculation one by one in the frequency domain by using pixel binning based Fourier ptychographic recovery method and the updating coefficient is gradually reduced; the cost function value is used as a criterion, and when the cost function value is less than a given threshold, the iteration is stopped. and at this moment, the amplitude and phase of the high-resolution images are the final reconstructed large field-of-view high-resolution microscopic images; step seven, color fusion; repeating step five and step six, and respectively reconstructing high-resolution images of red, green and blue channels, then three reconstructed high-resolution images are synthesized as the red, green and blue channels respectively of the final true color image.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
[0015]
[0016]
[0017]
[0018]
DESCRIPTION OF THE PREFERRED EMBODIMENT
[0019] Referring to
[0020] The LED array (1) includes a plurality of (at least 261) LED units (8) which are equally spaced to form a two-dimensional array. Each of the LED units is a three-color unit with the colors of red, green and blue and its typical wavelength is red light 633 nm, green light 525 nm and blue light 465 nm. The center-to-center spacing d between each LED unit is typically 1-4 mm. The LED array (1) does not need to be processed separately and is generally commercially available for purchase. Table 1 shows the product parameters of a commercially available LED array. In this LED array, the LED unit has 32 rows and 32 columns, 1024 in total, and the brightness of each LED unit is above 2000 cd/m.sup.2.
[0021] Light emitted by the i-th lit LED unit (8) of the LED array (1) passes through the condenser (3) and converges to become parallel light illuminating a specimen (4) to be examined, which is placed on the stage (2); part of the diffracted light passing through the specimen (4) is collected by the microscopic objective (5), converged by the tube lens (6), and reaches the imaging plane of the camera (7), forming an intensity image recorded by the camera (7). Each LED unit in LED array (1) can be individually lighted up, and the specific method of lighting up LED units is a prior art, and the implementation circuit can be achieved by adopting (but is not limited to) a prior art such as microcontrollers, an ARM, or a programmable logic device and the like; for the specific implementation methods, the relevant references (Guo Baozeng, Deng Yumiao: FPGA-based LED display control system design [J]. LCD and Display, 2010, 25(3): 424-428) can be referred to.
TABLE-US-00001 TABLE 1 Physical parameters of the LED array items parameters wavelength of LED unit red 633 nm, green 525 nm, blue 475 nm number of LED units 32 × 32 spacing of LED units 1.67 mm light emitting surface size 150 μm of LED units brightness of LED units 2000 cd/m.sup.2 array dimensions 55 mm × 55 mm × 17 mm weight 170 g cone angle of LED unit 150° power 5 v current the maximum 2A (all light up)
[0022] In order to meet the minimum frequency domain sampling rate needed by the reconstruction method of the present invention, the microscope objective (5) has a numerical aperture of NA.sub.obj and satisfies
wherein f is the focal length of the condenser (3), d is the spacing between adjacent LED units (8). In order to ensure the quality and accuracy of the reconstructed image, it is necessary to ensure that d is smaller than 2 mm, that is, an LED array with LED unit distance of less than 2 mm must be used. At the same time, in order to meet the minimum spatial sampling rate required by the reconstruction algorithm, the illumination wavelength of the LED array (1) is λ, the magnification of microscopic objective (5) is Mag, the size of pixel is Δx.sub.cam, which satisfies
[0023] The camera (7) is a color or monochrome camera, and if it is a monochrome camera, when imaging, each LED unit (8) of the LED array (1) emits red or green or blue monochromatic light respectively, recording all monochrome images sequentially by the monochrome camera; if it is a color camera, when imaging, each of the LED units (8) of the LED array (1) simultaneously emits red, green and blue light, recording all color images by the color camera.
[0024] In the invention, microscopic oil can be added between the condenser (3) and the specimen (4) to be examined for obtaining a higher illumination numerical aperture. Generally, if no microscopic oil is added between the condenser (3) and the specimen (4) to be examined, the illumination numerical aperture is up to 0.95. After the addition of microscopic oil, the illumination numerical aperture can reach to more than 1.20.
[0025] Referring to
and becomes the average brightness of each LED unit in three channels of red, green and blue, where N.sub.pixel is the total number of pixels in one image I.sub.i,c.sup.kb(r), and the average brightness corresponding to the LED unit (8) in the center of the LED array (1) is B.sub.i,c.sup.kb, then the normalized brightness correction coefficient R of each LED unit corresponding to three channels of red, green and blue is R.sub.i,c.sup.bri=B.sub.i,c.sup.kb/B.sub.l,c.sup.kb.
[0028] Step two, the position of the LED array (1) is calibrated, using the resolution board as the specimen (4) to be examined, using the LED array (1) as the illumination source of a high illumination-numerical-aperture based large field-of-view high-resolution microscopic imaging device, sequentially lighting up each LED unit (8) of the LED array (1), after illuminating the specimen (4) to be examined, images in focus and images with defocus distance h are respectively collected, and then the illumination angle corresponding to each of the three channels of red, green and blue for each LED unit (8) is calculated by using a sub-pixel registration algorithm, then determining the position of the LED array by using nonlinear regression.
[0029] In step two, using the resolution board as the specimen (4) to be examined, using the LED array (1) as the illumination source, sequentially lighting up each LED unit (8). First, after the LED array (1) illuminating the specimen (4) to be examined, in-focus image I.sub.i,c.sup.focus and defocus image I.sub.i,c.sup.defocus with defocus distance h (typical value of h ranges from 10 microns to 30 microns) are respectively collected; according to angular spectrum diffraction theory, numerically propagating in-focus image I.sub.l,c.sup.focus corresponding to LED unit (8) in the center of LED array (1) along optical axis with h distance, thereby obtaining numerical defocus image I.sub.l,c.sup.pro;
[0030] and then calculating the offset value (PY.sub.x,PY.sub.y) of each defocus image I.sub.i,c.sup.defocus relative to the numerical defocus image I.sub.l,c.sup.pro by using sub-pixel registration algorithm, the spatial frequency vector of the illumination light corresponding to the i-th LED unit is
where (u.sub.x,u.sub.y) is the spatial frequency along the x,y direction, and λ is the wavelength of the illumination light; finally, the position of the LED array is determined by nonlinear regression, the formulas are:
where Q( . . . ) is the objective function of the nonlinear regression method, (θ, Δx, Δy, f) are the updated four position parameters of the LED array, which are respectively the rotation error, the translation error in the x direction, the translation error in the y direction, and the focusing error, (θ.sup.0, Δx.sup.0, Δy.sup.0, f.sup.0) are the initialized LED array position parameters, [ . . . ] indicates that the nonlinear regression operation is performed, and d is the spacing between two adjacent LED units (8) of the LED array (1), (x.sub.i, y.sub.i) represents the spatial position coordinates of the i-th LED unit, λ is the wavelength of the illumination light, and (m.sub.i, n.sub.i) is the row number and the column number corresponding to the i-th LED unit.
[0031] Step three, raw images are acquired, using the LED array (1) as an illumination source of a high-illumination numerical aperture-based large field-of-view high-resolution micro-imaging device, sequentially illuminating each LED unit (8) of the LED array (1), after irradiating the specimen (4) to be examined, the corresponding low-resolution raw images are collected.
[0032] Step four, raw images are pre-processed, including threshold denoising and brightness correction, firstly using the average value of the dark current noise of the camera (7) as a threshold, performing threshold denoising on the captured low-resolution raw images, obtaining denoised low-resolution images (including low-resolution bright field images and low-resolution dark field images), and then dividing each denoised low-resolution image by the normalized brightness correction coefficient obtained in step one, brightness-corrected low-resolution images are obtained;
[0033] In step four: first, sequentially lighting up the LED units and illuminating a specimen (4) to be examined by using the monochromatic light of red, green and blue, then the captured original low-resolution images are marked as I.sub.i,c; turning off all the LED units and the captured dark current noise image I.sub.dark; the average value of the dark current noise of the camera is used as a threshold, then performing threshold denoising on the captured original low-resolution images, the formula is
where I.sub.i,c.sup.dn represents the low-resolution images after the threshold denoising, mean ( . . . ) indicates evaluating average gray value of the image; then dividing each image by the normalized brightness correction coefficient obtained in step one to accomplish the image brightness correction process, wherein the formula is:
I.sub.i,c.sup.uni=I.sub.i,c.sup.dn/R.sub.i,c.sup.bri
where I.sub.i,c.sup.uni is the low-resolution image after brightness correction, I.sub.i,c.sup.dn is the low-resolution image after performing threshold denoising, R.sub.i,c.sup.bri is the normalized brightness correction coefficient obtained in step one.
[0034] Step five, high-resolution images are initialized, adding and averaging all the low-resolution bright-field images among the brightness-corrected low-resolution images, and then initializing the amplitude and phase of high-resolution images by up-sampling. The formula for high-resolution image initialization is:
where o.sub.0.sup.ini is the initialized high-resolution complex amplitude image, UP[ . . . ] indicates the up-sampling nearest-neighbor interpolation and N.sub.b is the total number of bright-field images.
[0035] Step six: iterative reconstruction, all the brightness corrected low-resolution images are subjected to synthetic aperture calculation one by one in the frequency domain using the pixel binning based Fourier ptychographic iterative reconstruction method and gradually reducing the updating coefficient; The cost function value is used as a criterion, and when the cost function value is less than a given threshold (the typical value of the threshold can be set as 0.01 and can also be adjusted empirically), the iteration is stopped, and at this moment, the amplitude and phase of the high-resolution image are the final reconstructed large field-of-view high-resolution microscopic image.
[0036] In step six: the formula based on the pixel binning based Fourier ptychographic iterative method is as follows:
where F{ . . . } indicates that the Fourier transform is performed, F.sup.−1{ . . . } indicates that the inverse Fourier transform is performed, UP[ . . . ] indicates that the up-sampling nearest-neighbor interpolation is performed, DOWN[ . . . ] indicates the down-sampling pixel binning process; O.sub.i.sup.k the high-resolution spectrum of the specimen to be examined, k represents the k-th iteration, P.sub.i.sup.k is the spectrum aperture function of the microscopic objective, and Ψ.sub.i.sup.k is the updated local spectrum of the specimen to be examined, γ is a constant to ensure that the denominator is not zero, the typical value is 0.001; | . . . | represents the modulus of the two-dimensional complex matrix, | . . . |.sub.max represents the maximum value in the modulus of the two-dimensional complex matrix; α.sup.k is the updated coefficient of the spectrum of the specimen to be examined in the k-th iteration, β.sup.k is the updated coefficient of the aperture function of the microscopic objective in the k-th iteration, COST.sup.k is the cost function. When the cost function COST.sup.k is less than a certain fixed threshold ε (the typical value of ε is 0.001) at the end of the k-th iteration, the iteration is considered to be convergent and the iteration is stopped, the amplitude and the phase of the high-resolution image at the moment are the finally reconstructed large field-of-view high-resolution microscopic image.
[0037] The above reconstruction process is only applicable to reconstructing a monochrome image, and the present invention utilizes a high-illumination numerical aperture-based large field-of-view high-resolution microimaging iterative reconstruction method of the above device, the steps of reconstructing a true color image are as follows: [0038] step one, the brightness of the LED unit (8) is calibrated, using the LED array (1) as an illumination source of a high illumination-numerical-aperture based large field-of-view high-resolution microscopic imaging device, sequentially lighting up each LED unit (8) of the LED array (1), and after illuminating the blank specimen (4), using the high magnification objective lens to collect the corresponding image and calculating the normalized brightness correction coefficient corresponding to each LED unit (8) in the three channels of red, green and blue; [0039] step two, the position of the LED array (1) is calibrated, using the resolution board as the specimen (4) to be examined, using the LED array (1) as the illumination source of a high illumination-numerical-aperture based large field-of-view high-resolution microscopic imaging device, sequentially lighting up each LED unit (8) of the LED array (1), after illuminating the specimen (4) to be examined, in-focus images and defocus images with defocus distance h are respectively collected, and then the illumination angle corresponding to each of the three channels of red, green and blue for each LED unit (8) is calculated by using a sub-pixel registration algorithm, then determining the position of the LED array by using nonlinear regression; [0040] step three, raw images are captured, using the LED array (1) as an illumination source of a high illumination-numerical-aperture based large field-of-view high-resolution microscopic imaging device, sequentially lighting up each LED unit (8) of the LED array (1), after illuminating the specimen (4) to be examined, the corresponding low-resolution raw images are collected; [0041] step four: raw images are pre-processed, including threshold denoising and brightness correction, firstly using the average value of the dark current noise of the camera (7) as a threshold, performing threshold denoising on the captured low-resolution raw images to obtain denoised low-resolution images, and then dividing each denoised low-resolution image by the normalized brightness correction coefficient obtained in step one, then brightness-corrected low-resolution images are obtained; [0042] step five, high-resolution images are initialized, adding and averaging all the low-resolution bright-field images among the brightness-corrected low-resolution images, and then initializing the amplitude and phase of the high-resolution images by up-sampling; [0043] step six, iterative reconstruction; each of the brightness corrected low-resolution images is subjected to synthetic aperture calculation one by one in the frequency domain by using pixel binning based Fourier ptychographic recovery method and the updating coefficient is gradually reduced; the cost function value is used as a criterion, and when the cost function value is less than a given threshold, the iteration is stopped. and at this moment, the amplitude and phase of the high-resolution images are the final reconstructed large field-of-view high-resolution microscopic images; step seven, color fusion; repeating step five and step six, and respectively reconstructing high-resolution images of red, green and blue channels, then three reconstructed high-resolution images are synthesized as the red, green and blue channels respectively of the final true color image.
[0044] In order to test the iterative reconstruction method of large field-of-view high-resolution microimaging, the present invention selects the 1951 USAF resolution test board for imaging test. In the experiment, the used LED array comprises 261 LED units, and these 261 LED units are used to generate 261 illumination light with different angles. The distance between LED units is 1.67 mm, the center wavelength of the emitted red light is 632.8 nm, and the spectral bandwidth is about 20 nm. The microscope objective used in the system has a numerical aperture of 0.4 and a magnification of 10×. At the same time, the low-resolution image captured when simultaneously lighting up all brightfield LED units in the LED array to illuminate the 1951 USAF resolution test board specimen is shown in
TABLE-US-00002 TABLE 2 Physical parameters of the 1951 USAF resolution test board group id lp/mm Elements −2 −1 0 1 2 3 4 5 6 7 8 9 10 11 1 0.250 0.500 1.00 2.00 4.00 8.00 16.0 32.0 64.0 128 256 512 1024 2048 2 0.280 0.561 1.12 2.24 4.49 8.98 18.0 36.0 71.8 144 287 575 1149 2299 3 0.315 0.630 1.26 2.52 5.04 10.1 20.2 40.3 80.6 261 323 645 1290 2580 4 0.353 0.707 1.41 2.83 5.66 11.3 22.6 45.3 90.5 181 362 724 1448 2896 5 0.397 0.793 1.59 3.17 6.35 12.7 25.4 50.8 102 203 406 813 1625 3251 6 0.445 0.891 1.78 3.56 7.13 14.3 28.5 57.0 114 228 456 912 1825 3649