Rapid autofocus method for stereo microscope
09851549 · 2017-12-26
Assignee
Inventors
- Mei Yu (Zhejiang, CN)
- Yi Liu (Zhejiang, CN)
- Li Cui (Zhejiang, CN)
- Shengli Fan (Zhejiang, CN)
- Yigang Wang (Zhejiang, CN)
Cpc classification
G02B21/367
PHYSICS
International classification
G02B21/36
PHYSICS
Abstract
A rapid autofocus method for a stereo microscope includes steps of: calculating a disparity of each of stereo microscopic images in a stereo microscopic calibration image sequence; extracting a clear stereo microscopic image sequence from the stereo microscopic calibration image sequence; then, finding out a largest disparity and a smallest disparity among the disparities of all the stereo microscopic images in the clear stereo microscopic image sequence; at a chosen magnification, arbitrarily acquiring a stereo microscopic image; finally, determining a disparity range according to the disparity of the acquired stereo microscopic image, the largest disparity and the smallest disparity, and realizing an autofocus of a target object in the acquired stereo microscopic images. The disparity range is obtained via once off-line calibration at the same magnification, and applicable to the autofocus at an arbitrary timing.
Claims
1. A rapid autofocus method for a stereo microscope, comprising steps of: {circle around (1)} acquiring a stereo microscopic calibration image sequence having M stereo microscopic images, wherein: the M stereo microscopic images in the stereo microscopic calibration image sequence are orderly arranged from vague to clear and then from clear to vague, and contain the same target object; an i th stereo microscopic image in the stereo microscopic calibration image sequence is denoted as I.sub.i, wherein M≧50 and 1≦i≦M; each of the stereo microscopic images consists of a left view image and a right view image with the same resolution; {circle around (2)} calculating a disparity of each of the stereo microscopic images in the stereo microscopic calibration image sequence, wherein the disparity of I.sub.i is denoted as d.sub.i; {circle around (3)} extracting a clear stereo microscopic image sequence out of the stereo microscopic calibration image sequence, comprising steps of: {circle around (3)}-1, calculating a value of a Tenengrad function of a left view in each of the stereo microscopic images in the stereo microscopic calibration image sequence with a Tenengrad definition operator; and {circle around (3)}-2, finding out a largest value among the M values of the Tenengrad function, denoted as F.sub.Tm.sup.L; according to a range of a depth of field at a chosen magnification, picking out m stereo microscopic images before the stereo microscopic image corresponding to F.sub.Tm.sup.L, and also m stereo microscopic images after the stereo microscopic image corresponding to F.sub.Tm.sup.L; and then, constructing the clear stereo microscopic image sequence orderly with the m stereo microscopic images before the stereo microscopic image corresponding to F.sub.Tm.sup.L, the stereo microscopic image corresponding to F.sub.Tm.sup.L and the m stereo microscopic images after the stereo microscopic image corresponding to F.sub.Tm.sup.L; wherein: a value of m is related to the magnification of an objective lens of the stereo microscope as:
2. The rapid autofocus method, as recited in claim 1, wherein the step {circle around (1)} of “acquiring a stereo microscopic calibration image sequence having M stereo microscopic images” comprises steps of: {circle around (1)}-1, providing the vertical elevator right below the objective lens of the stereo microscope; providing the object on the vertical elevator; observing imaging of the object with the stereo microscope and adjusting a position of the object, in such a manner that the stereo microscopic images after imaging have information of only one plane and contain the target object; and {circle around (1)}-2, adjusting a height of the vertical elevator to blur the stereo microscopic images after imaging; moving the vertical elevator towards the same one direction and acquiring one stereo microscopic image after imaging each time the vertical elevator moves by Δh μm, wherein: the vertical elevator moves at M times, so the M stereo microscopic images after imaging are acquired and orderly arranged from vague to clear and then from clear to vague; constructing the stereo microscopic calibration image sequence with the M stereo microscopic images after imaging, wherein: 2≦Δh≦20 and M≧50.
3. The rapid autofocus method, as recited in claim 1, wherein: the disparity d.sub.i of I.sub.i in the step {circle around (2)}, the disparity d.sub.test of I.sub.test and the disparity d.sub.test′ of I.sub.test′ in the step {circle around (5)} are all acquired through the same process; respectively defining I.sub.i, I.sub.test and I.sub.test′ as to-be-treated images, the process for acquiring the disparity of the to-be-treated images comprises steps of: a, processing the to-be-treated image with an epipolar line rectification with a quasi-Euclidean epipolar line rectification algorithm, so as to acquire parallel stereo microscopic image; and b, extracting all matching point pairs from a left view image and a right view image of the parallel stereo microscopic image with a SURF algorithm, wherein: for an arbitrary pair of matching points, the matching point pair is formed by two pixel points matched with each other in the left view image and the right view image of the parallel stereo microscopic image; calculating the disparity of each matching point pair, wherein: for an arbitrary matching point pair, the disparity of the matching point pair is acquired by subtracting a horizontal coordinate of the matching point in the right view image of the parallel stereo microscopic image from a horizontal coordinate of the matching point in the left view image of the parallel stereo microscopic image; and then, calculating a mean value of the disparities of all the matching point pairs, wherein the mean value represents the disparity of the to-be-treated image.
4. The rapid autofocus method, as recited in claim 2, wherein: the disparity d.sub.i of I.sub.i in the step {circle around (2)}, the disparity d.sub.test of I.sub.test and the disparity d.sub.test′ of I.sub.test′ in the step {circle around (5)} are all acquired through the same process; respectively defining I.sub.i, I.sub.test and I.sub.test′ as to-be-treated images, the process for acquiring the disparity of the to-be-treated images comprises steps of: a, processing the to-be-treated image with an epipolar line rectification with a quasi-Euclidean epipolar line rectification algorithm, so as to acquire parallel stereo microscopic image; and b, extracting all matching point pairs from a left view and a right view of the parallel stereo microscopic image with a SURF algorithm, wherein: for an arbitrary pair of matching points, the matching point pair is formed by two pixel points matched with each other in the left view and the right view of the parallel stereo microscopic image; calculating the disparity of each matching point pair, wherein: for an arbitrary matching point pair, the disparity of the matching point pair is acquired by subtracting a horizontal coordinate of the matching point in the right view of the parallel stereo microscopic image from a horizontal coordinate of the matching point in the left view of the parallel stereo microscopic image; and then, calculating a mean value of the disparities of all the matching point pairs, wherein the mean value represents the disparity of the to-be-treated image.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
(1)
(2)
(3)
(4)
(5)
(6)
(7)
(8)
(9)
(10)
(11)
(12)
(13)
(14)
(15)
(16)
(17)
(18)
(19)
(20)
(21)
(22)
(23)
(24)
(25)
(26)
(27)
(28)
(29)
(30)
(31)
(32)
(33)
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT
(34) The present invention is further illustrated in combination of the drawings the preferred embodiment.
(35) Referring to
(36) Step {circle around (1)}: acquiring a stereo microscopic calibration image sequence which has M stereo microscopic images, wherein: the M stereo microscopic images in the stereo microscopic calibration image sequence are orderly arranged from vague to clear and then from clear to vague, and contain the same target object; an i th stereo microscopic image in the stereo microscopic calibration image sequence is denoted as I.sub.i; a left view image and a right view image of I.sub.i are respectively denoted as I.sub.i.sup.L and I.sub.i.sup.R; M≧50. According to the preferred embodiment of the present invention, M=150; 1≦i≦M.
(37) According to the preferred embodiment of the present invention, the step of acquiring the stereo microscopic calibration image sequence comprises steps of:
(38) {circle around (1)}-1, providing a vertical elevator right below an objective lens of the stereo microscope; providing an object on the vertical elevator; observing imaging of the object with the stereo microscope and adjusting a position of the object, in such a manner that the stereo microscopic images after imaging have information of only one plane and contain the target object. Because the target object is prevented from moving beyond a field of vision of the stereo microscopic images after imaging during a whole process of calibration moving shooting, it is preferable to adjust the target object to be at a center of the whole stereo microscopic image after imaging. Because a background of the stereo microscopic images requires a single color, it is preferable to keep a relatively small brightness change in the background during moving the vertical elevator.
(39) {circle around (1)}-2, adjusting a height of the vertical elevator with a precise step motor to blur the stereo microscopic images after imaging; driving the vertical elevator to move towards the same one direction (vertically upward or vertically downward) with the precise step motor (minimal moving step of 2 μm), and acquiring one stereo microscopic image after imaging each time the vertical elevator moves by Δh μm, wherein: the vertical elevator moves at M times, so the M stereo microscopic images after imaging are acquired and orderly arranged from vague to clear and then from clear to vague; constructing the stereo microscopic calibration image sequence with the M stereo microscopic images after imaging, wherein: 2≦Δh≦20; M≧50. According to the preferred embodiment of the present invention, Δh=10; M=150.
(40)
(41) Step {circle around (2)}: calculating a disparity of each of the stereo microscopic images in the stereo microscopic calibration image sequence, wherein the disparity of I.sub.i is denoted as d.sub.i.
(42) Step {circle around (3)}: extracting a clear stereo microscopic image sequence from the stereo microscopic calibration image sequence, comprising steps of:
(43) {circle around (3)}-1, calculating a value of a Tenengrad function of the left view in each of the stereo microscopic images in the stereo microscopic calibration image sequence with a Tenengrad definition operator, wherein: the value of the Tenengrad function of I.sub.i.sup.L is denoted as F.sub.i,Tenengrad.sup.L;
(44)
wherein: (x, y) represents coordinates of a pixel point in the stereo microscopic image in the stereo microscopic calibration image sequence; W and H represent a width and a height of the stereo microscopic image in the stereo microscopic calibration image sequence; 1≦x≦W, 1≦y≦H; S.sub.X (x, y) represents a value of the pixel point with the coordinates of (x, y) in I.sub.i.sup.L acquired through a Sobel operator for detecting a horizontal edge; S.sub.Y (x, y) represents the value of the pixel point with the coordinates of (x, y) in I.sub.i.sup.L acquired through the Sobel operator for detecting a vertical edge; and
(45) {circle around (3)}-2, finding out a largest value of the Tenengrad function among the M values of the Tenengrad function, denoted as F.sub.Tm.sup.L; according to a range of a depth of field at a chosen magnification, picking out m stereo microscopic images before the stereo microscopic image corresponding to F.sub.Tm.sup.L, and also m stereo microscopic images after the stereo microscopic image corresponding to F.sub.Tm.sup.L; and then, constructing the clear stereo microscopic image sequence orderly with the m stereo microscopic images before the stereo microscopic image corresponding to F.sub.Tm.sup.L, the stereo microscopic image corresponding to F.sub.Tm.sup.L and the m stereo microscopic images after the stereo microscopic image corresponding to F.sub.Tm.sup.L; wherein:
(46) a value of m is related to the magnification of the objective lens of the stereo microscope as:
(47)
wherein: n represents a refractive index of a medium between the object provided on the vertical elevator and the objective lens of the stereo microscope; generally the medium is the air, so n=1; λ represents a light wavelength, generally 0.55; e represents a minimal physical distance distinguishable between two neighboring light sensors on an image plane of the stereo microscope, generally 14; μ represents the magnification of the objective lens of the stereo microscope, embodied as μ=8; NA represents a numerical aperture of the objective lens of the stereo microscope; NA=n×sin θ; θ represents an angle formed by a refraction light farthest from a central optical axis of the objective lens of the stereo microscope; according to the preferred embodiment of the present invention, NA=0.01; Δh represents a distance by which the vertical elevator moves each time, having a unit of μm; 2≦Δh≦12; according to the preferred embodiment of the present invention, Δh=10.
(48) Step {circle around (4)}: finding out a smallest disparity and a largest disparity from the disparities of all the stereo microscopic images in the clear stereo microscopic image sequence, respectively denoted as d.sub.Cmin and d.sub.Cmax wherein an interval [d.sub.Cmin, d.sub.Cmax] represents a disparity range of clear stereo microscopic images at the chosen magnification.
(49) Step {circle around (5)}: at the chosen magnification, arbitrarily acquiring a first stereo microscopic image after imaging by the stereo microscope, denoted as I.sub.test, wherein: the acquired stereo microscopic images after imaging are generally required to contain a target object; the target object of the acquired stereo microscopic images after imaging can be the same with or different from the target object of each of the stereo microscopic images in the stereo microscopic calibration image sequence; calculating the disparity of I.sub.test, denoted as d.sub.test, and judging the definition of I.sub.test with d.sub.test; if d.sub.test is out of [d.sub.Cmin, d.sub.Cmax], judging that I.sub.test is unclear; if d.sub.test is within [d.sub.Cmin, d.sub.Cmax], judging that I.sub.test is clear; and then realizing an autofocus of the target object in I.sub.test according to d.sub.test and [d.sub.Cmin, d.sub.Cmax], comprising steps of:
(50) {circle around (5)}-4, judging whether |d.sub.test−d.sub.mc|<T or not; if yes, determining that I.sub.test is clearest, so as to realize the autofocus of the target object in I.sub.test; if no, executing a step {circle around (5)}-2; wherein: “| |” is a symbol for counting an absolute value; d.sub.mc represents the disparity corresponding to the clearest images in [d.sub.Cmin, d.sub.Cmax];
(51)
T represents a designated judgment threshold; T∈(0,1]; according to the preferred embodiment of the present invention, T=0.5;
(52) {circle around (5)}-2, supposing that the vertical elevator moves by Δh μm each time, calculating times at which the vertical elevator needs to move for focusing the target object in I.sub.test to a clearest position, denoted as step, wherein
(53)
determining a direction towards which the vertical elevator moves for focusing the target object in I.sub.test to the clearest position according to step; if step is smaller than 0, determining the direction to be upward; if step is larger than 0, determining the direction to be downward; wherein: 2≦Δh≦20; according to the preferred embodiment of the present invention, Δh=10; d.sub.perstep represents an absolute value of a difference between the disparity of the stereo microscopic images acquired before the vertical elevator moves upward or downward at one time and the disparity of the stereo microscopic images acquired thereafter; according to the preferred embodiment of the present invention, d.sub.perstep=0.060765;
(54) {circle around (5)}-3, moving the vertical elevator at the times, step, and toward the direction for focusing the target object in I.sub.test to the clearest position; and
(55) {circle around (5)}-4 because some error exists in a linear relationship between the disparity of the stereo microscopic images and the height of the vertical elevator, it is necessary to calculate the disparity of the stereo microscopic images acquired after the vertical elevator moves, namely acquiring a second stereo microscopic image after imaging by the stereo microscope after finishing moving the vertical elevator, denoted as I.sub.test′; calculating the disparity of I.sub.test′, denoted as d.sub.test′; and then, let I.sub.test=I.sub.test′ and let d.sub.test=d.sub.test′, returning to the step {circle around (5)}-4 and executing an autofocus iteration, wherein “=” in I.sub.test=I.sub.test′ and d.sub.test=d.sub.test′ is an assignment symbol.
(56) According to the preferred embodiment of the present invention, the disparity d.sub.i of I.sub.i in the step {circle around (5)}, the disparity d.sub.test of I.sub.test and the disparity d.sub.test′ of I.sub.test′ in the step {circle around (5)} are all acquired through the same process. Respectively defining I.sub.i, I.sub.test and I.sub.test′ as to-be-treated images, a process for acquiring the disparity of the to-be-treated images comprises steps of:
(57) a, processing the to-be-treated image with an epipolar line rectification with a quasi-Euclidean epipolar line rectification algorithm, so as to acquire parallel stereo microscopic image; and
(58) b, extracting all matching point pairs from a left view image and a right view image of the parallel stereo microscopic image with a SURF algorithm, wherein: for an arbitrary pair of matching points, the matching point pair is formed by two pixel points matched with each other in the left view image and the right view image of the parallel stereo microscopic image; calculating the disparity of each matching point pair, wherein: for an arbitrary matching point pair, the disparity of the matching point pair is acquired by subtracting a horizontal coordinate of the matching point in the right view image of the parallel stereo microscopic image from a horizontal coordinate of the matching point in the left view image of the parallel stereo microscopic image; and then, calculating a mean value of the disparities of all the matching point pairs, wherein the mean value represents the disparity of the to-be-treated image.
(59) In order to illustrate validity and feasibility of the rapid autofocus method of the present invention, a circuit board is taken as an experimental shooting object for experiment verification. A test platform of the experiment comprises a digital stereo microscope, a circuit board, a vertical elevator system comprising a three-axis step motor with precise movement and a vertical elevator, a microcontroller and a computer terminal A left camera and a right camera of the digital stereo microscope both have a single-channel image definition of 720×576. The circuit board is provided on the vertical elevator; the step motor is CHUO SEIKI XA07A-R2H which has a minimal moving step of 2 μm and adjusts a moving step according to practical requirements. The microcontroller is an 80C51 microcontroller. The computer terminal sends signals of the moving step and the moving direction to the microcontroller, for controlling the step motor to drive the vertical elevator to move. The computer terminal runs in an environment of Intel Core(TM)i3 CPU 3.19 GHz, 1.74 G Memory, and adopts VS2012 and OPENCV2.4.8 for the experiment.
(60) In the experiment, four stereo microscopic images respectively of three different target objects are chosen for test.
(61) The stereo microscopic images respectively of
(62) As shown in
(63)
(64) Supposing that, under the control of the step motor, a time for the vertical elevator to vertically move by the unit step is t.sub.move, and a time for the step motor to wait for acquiring the stereo microscopic images is t.sub.wait, a time for calculating the definition function values is so far less than the above two times as to be negligible. An autofocusing time through a conventional method (continually shooting n′ stereo microscopic images to calculate out the clearest image, with an image interval nearly symmetric relative to a focused position) is (t.sub.move+t.sub.wait)×n′.
(65) As a contrast, in the method of the present invention, the disparity range of the clear image interval is calibrated merely once but applicable for a long term, and thus a time for calibrating is excluded from the autofocusing time. A time of the method of the present invention is mainly consumed by the definition judgment, iteratively moving the vertical elevator, waiting by the step motor and calculating the disparities. Compared to the time for moving the vertical elevator and the time of waiting by the step motor, the rest two times can be neglected. Since a moving range as a whole is nearly symmetric relative to the focused position, the times at which the vertical elevator moves in the method of the present invention is merely half of the times at which the vertical elevator moves in the conventional method, namely n′/2. Given the iteration times n.sub.d, the autofocusing time of the method of the present invention is
(66)
(67) A frequency of waiting by the step motor in the conventional method is far higher than a frequency of waiting by the step motor in the method of the present invention, namely n′>>n.sub.d. For the platform in the experiment, t.sub.wait=2s ; t.sub.move=1s. The autofocusing time of the method of the present invention is about ⅙ of the autofocusing time of the conventional method. Therefore, the method of the present invention has a short autofocusing time.
(68) One skilled in the art will understand that the embodiment of the present invention as shown in the drawings and described above is exemplary only and not intended to be limiting.
(69) It will thus be seen that the objects of the present invention have been fully and effectively accomplished. Its embodiments have been shown and described for the purposes of illustrating the functional and structural principles of the present invention and is subject to change without departure from such principles. Therefore, this invention includes all modifications encompassed within the spirit and scope of the following claims.