METHOD FOR DETECTING OBJECT IMAGE USING HYPERSPECTRAL IMAGING BY BAND SELECTION

Abstract

The present application related to a method for detecting image using hyperspectral imaging by band selection. Firstly, obtaining a hyperspectral imaging information according to a reference image, hereby, obtaining corresponded hyperspectral image from an input image, and obtaining corresponded feature values by band selection for operating Principal components analysis to simplify feature values. Then, obtaining feature images by Convolution kernel, and then positioning an image of an object under detected by a default box and a boundary box from the feature image. By Comparing with the esophageal cancer sample image, the image of the object under detected is classifying to an esophageal cancer image or a non-esophageal cancer image. Thus, detecting an input image from the image capturing device by the convolutional neural network to judge if the input image is the esophageal cancer image for helping the doctor to interpret the image of the object under detected.

Claims

1. A method for detecting object image using hyperspectral imaging by band selection, comprising steps of: acquiring the hyperspectral image information formed by converting a reference image to a hyperspectral reference image, said reference image including one or more object reference image and a background reference image; an image extraction unit extracting an input image to a host, said input image including one or more object-under-test image and a background image; said host converting said input image according to said hyperspectral image information for giving a hyperspectral image; said host analyzing said hyperspectral image for giving a plurality of first hyperspectral characteristic values; said host performing band selection on said plurality of first hyperspectral characteristic values according to a cell and performing principal component analysis to simplify said hyperspectral image and generate a plurality of second characteristic values correspondingly; said host performing one or more layer of convolution calculation on said plurality of second characteristic values according to a plurality of kernels to filter out said background image and give a convolution result, and giving one or more selected image according to said convolution result and said one or more object-under-test image, said plurality of kernels including a plurality of selected characteristic values and a plurality of peripheral characteristic values, said one or more object-under-test image including a plurality of peripheral images and said one or more selected image, said plurality of peripheral images surrounding said one or more selected image, said one or more selected image corresponding to said plurality of selected characteristics, and said plurality of peripheral images corresponding to said plurality of peripheral characteristic values; said host generating one or more prediction box according to the edge of said one or more selected image; said host extracting a bounding box of said input image for comparing a first central point of said prediction box with a second central point of said bounding box and giving a center displacement between said prediction box and said bounding box, and said bounding box corresponding to an edge of said input image; said host performing a regression calculation according to said center displacement and giving a regression result; said host aligning said obj ect-under-test image according to said regression result and said prediction box so that said selected image can move towards said second central point as said first central point moves towards said second central point; said host matching and comparing said aligned object-under-test image with one or more sample image for generating a comparison result; and said host classifying said input image as a target-object image according to said comparison result.

2. The method for detecting object image using hyperspectral imaging by band selection of claim 1, where in said step of said host performing one or more layer of convolution calculation on said plurality of second characteristic values according to a plurality of kernels, said host sets said plurality of kernels as mxnxp and normalizes a plurality of pixels of said input image to a plurality of pixel normalized values, and by multiplying said plurality of kernels by said plurality of pixel normalized values, said plurality of second characteristic values are extracted in a convolution layer, where m=n and m=1, 3, 5, 10, 19, or 38.

3. The method for detecting object image using hyperspectral imaging by band selection of claim 1, where in said step of giving one or more selected image according to said convolution result and said one or more obj ect-under-test image, said host integrates the region of said plurality of selected characteristic values and gives one or more distribution region on said input image.

4. The method for detecting object image using hyperspectral imaging by band selection of claim 1, where in said step of said host performing one or more layer of convolution calculation on said plurality of second characteristic values according to a plurality of kernels, said host performs convolution on each pixel of said input image according to a single shot multibox detector model for detecting said plurality of second characteristic values.

5. The method for detecting object image using hyperspectral imaging by band selection of claim 1, where in said step of said host performing a regression calculation according to said center displacement, said host performs the regression calculation according to a first location of said prediction box, a second location of said bounding box, and a sizing factor for aligning said object-under-test image.

6. The method for detecting object image using hyperspectral imaging by band selection of claim 1, where in said step of said host matching and comparing said aligned object-under-test image with one or more sample image, said host matches and compares said object-under-test image and said one or more sample image in a fully connected layer.

7. The method for detecting object image using hyperspectral imaging by band selection of claim 1, where in said step of said host classifying said input image as a target-object image according to said comparison result, when said host cannot judge said input image as a target-object image according to said one or more sample image, said host performs similarity comparison on said object-under-test image according to said one or more sample image.

8. The method for detecting object image using hyperspectral imaging by band selection of claim 7, where in said step of said host performing similarity comparison on said object-under-test image according to said one or more sample image, if said host judges that a similarity level of said object-under-test image is greater than a similarity threshold value, said host judges said input image as said target-object image.

9. The method for detecting object image using hyperspectral imaging by band selection of claim 1, wherein said hyperspectral image information corresponds to a plurality of white light images and a plurality of narrow-band images and includes a plurality of color matching functions, a calibration matrix, and a conversion matrix.

10. The method for detecting object image using hyperspectral imaging by band selection of claim 1, where in said step of said host matching and comparing said aligned object-under-test image with one or more sample image, said host reads said one or more sample image from a database for matching and comparing according to said aligned object-under-test image.

11. The method for detecting object image using hyperspectral imaging by band selection of claim 1, where after said step of said host performing band selection on said plurality of first hyperspectral characteristic values according to a cell and performing principal component analysis to simplify said hyperspectral image and generate a plurality of second characteristic values correspondingly, said host converts said plurality of first characteristic values to a simulation image according to said band selection and said hyperspectral image information.

12. The method for detecting object image using hyperspectral imaging by band selection of claim 1, wherein said cell is an esophageal cancer cell.

13. The method for detecting object image using hyperspectral imaging by band selection of claim 11, wherein said band is 415 nanometers and 540 nanometers.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0031] FIG. 1 shows a flowchart of the method according to an embodiment of the present application;

[0032] FIG. 2A to FIG. 2I show partial steps according to an embodiment of the present application;

[0033] FIG. 3 shows a schematic diagram of the kernel and the input image according to an embodiment of the present application;

[0034] FIG. 4 shows a schematic diagram of the located region according to an embodiment of the present application;

[0035] FIG. 5 shows a schematic diagram of constructing the prediction box according to an embodiment of the present application;

[0036] FIG. 6 shows a schematic diagram of displacement of the central point according to an embodiment of the present application;

[0037] FIG. 7 shows a schematic diagram of aligning the prediction box and the bounding box according to an embodiment of the present application;

[0038] FIG. 8 shows a schematic diagram of the image according to an embodiment of the present application;

[0039] FIG. 9A and FIG. 9B show schematic diagrams of the image according to an embodiment of the present application; and

[0040] FIG. 10A and FIG. 10B show schematic diagrams of the image according to an embodiment of the present application.

DETAILED DESCRIPTION

[0041] In order to make the structure and characteristics as well as the effectiveness of the present application to be further understood and recognized, the detailed description of the present application is provided as follows along with embodiments and accompanying figures.

[0042] To solve the problems of artificial errors or difficulty in image identification caused by complicated endoscope operations according to the prior art, the present application provides a method for detecting object image using hyperspectral imaging by band selection.

[0043] In the following, the properties and the accompanying system provided by the method for detecting object image using hyperspectral imaging by band selection according to the present application will be further illustrated.

[0044] First, please refer to FIG. 1, which shows a flowchart of the method according to an embodiment of the present application. As shown in the figure, the method for detecting object image using hyperspectral imaging by band selection according to the present application comprises steps of:

Step S05: Acquiring the hyperspectral image information formed by converting a reference image to a hyperspectral reference image;
Step S10: An image extraction unit extracting an input image to a host;
Step S15: Converting the input image according to the hyperspectral image information for giving a hyperspectral image;
Step S20: Analyzing the hyperspectral image for giving first hyperspectral characteristic values;
Step S25: Performing band selection on the first hyperspectral characteristic values according to a cell and performing principal component analysis to generate second characteristic values correspondingly;
Step S30: Performing convolution calculation on the second characteristic values according to kernels to give a convolution result, and giving a selected image according to the convolution result and an object-under-test image;
Step S35: The host generating a prediction box according to the edge of a selected image;
Step S40: The host extracting a bounding box of the input image for comparing a first central point of the prediction box with a second central point of the bounding box and giving a center displacement between the prediction box and the bounding box;
Step S45: The host performing a regression calculation according to the center displacement and giving a regression result;
Step S50: The host aligning the object-under-test image according to the regression result and the prediction box so that the selected image can move towards the second central point as the first central point moves towards the second central point;
Step S55: The host comparing the object-under-test image with a sample image for generating a comparison result; and
Step S60: The host classifying the input image as a target-object image or a non-target-object image according to the comparison result.

[0045] Please refer to FIG. 2A to FIG. 2I, which show detection system 1 adopted in the method for detecting object image using hyperspectral imaging by band selection according to the present application. The detection system 1 comprises a host 10 and an image extraction unit 20. According to the present embodiment, the host 10 is a computer including a processing unit 12, a memory 14, and a storage unit 16. Nonetheless, the present application is not limited to the embodiment. The host 10 can be a server, a notebook computer, a tablet computer, or any electronic device with calculation capability. A database 30 is built in, but not limited to, the storage unit 16. The database 30 can be built in an external storage unit of the host 10. The host 10 executes a convolution program P by the processing unit 12 and builds a convolutional neural network CNN correspondingly. In addition, according to the present embodiment, the image extraction unit 20 is an endoscope used for probing organs and tissue in human bodies, such as a cystoscope, a gastroscope, a colonscope, a bronchoscope, or a laparoscope.

[0046] In the step S05, as shown in FIG. 2A, the host 10 reads a corresponding reference image REF of the image extraction unit 20. The reference image REF includes one or more object reference image O1R and a background reference image BGR. The reference image REF can be a white light image and a narrow-band image stored in the database 30 or a reference color checker extracted from the 24-color checkers by the image extraction unit 20. The image extraction unit 20 according to the present embodiment acquires the white light reference image using the white light endoscope OLYMPUS EVIS LUCERA CV-260 SL, and the narrow-band reference image using the narrow-band endoscope OLYMPUS EVIS LUCERA CLV-260. In other words, the one or more object reference image O1R and the background reference image BGR are stored in the white light reference image and the narrow-band reference image of the reference image REF, respectively.

[0047] As shown in FIG. 2A, the host 10 acquires the hyperspectral image information HAS according to the reference image REF. Namely, the image extraction unit 20 calculates the input image using the visible hyperspectral algorithm (VIS-HSA) to give the conversion formula from a normal image color space (the color space of the extracted image) to the XYZ color space (CIE 1931 XYZ color space), for example, the formula for converting the sRGB color space to the XYZ color space. The hyperspectral image information according to the present application is the visible-light hyperspectrum corresponding to the visible-light hyperspectral technology and to the X-Rite Classic 24-color checkers. The 24-color checkers include the main colors (red, green, blue, grey) frequently seen in the nature. The hyperspectral image information HAS corresponds to the plurality of white light images and the plurality of narrow-band images as described above and includes a plurality of color matching functions (CMF), a calibration matrix C, and a conversion matrix M.

[0048] Firstly, the reference image REF and the spectrometer (Ocean Optics-QE65000) should be converted to the same XYZ color space. The conversion formula for the reference image REF is:

[00001]$\begin{array}{cc}\left[\begin{array}{c}X\\ Y\\ Z\end{array}\right]=\left[M_A\right]\left[T\right]\left[\begin{array}{c}f(R_{\mathrm{sRGB})}\\ f(G_{\mathrm{sRGB})}\\ f(B_{\mathrm{sRGB})}\end{array}\right]\times 100,0\le \begin{array}{c}{R}_{\mathrm{sRGB}}\\ G_{\mathrm{sRGB}}\\ B_{\mathrm{sRGB}}\end{array}\le 1& \left(1\right)\end{array}$$\mathrm{where}$$\begin{array}{cc}\left[T\right]=\left[\begin{array}{ccc}0.4104& 0.3576& 0.1805\\ 0.2126& 0.7152& 0.0722\\ 0.0193& 0.1192& 0.9505\end{array}\right]& \left(2\right)\end{array}$$\begin{array}{cc}f\left(n\right)=\{\begin{array}{cc}{\left(\frac{n+0.055}{1.055}\right)}^{2.4},& n>0.04045\\ \left(\frac{n}{12.92}\right),& \mathrm{otherwise}\end{array}& \left(3\right)\end{array}$$\begin{array}{cc}\left[M_A\right]=\left[\begin{array}{ccc}{X}_{\mathrm{SW}}/X_{\mathrm{CW}}& 0& 0\\ 0& Y_{\mathrm{SW}}/X_{\mathrm{CW}}& 0\\ 0& 0& Z_{\mathrm{SW}}/Z_{\mathrm{CW}}\end{array}\right]& \left(4\right)\end{array}$

[0049] f(n) is the gamma function capable of converting sRGB to linear RGB values. T is the conversion matrix. [M.sub.A] is a color adaptation matrix. According to (1), linear RGB values con be converted to XYZ values (XYZ endoscope) defined by the XYZ color space.

[0050] The formulae of converting the reflection spectrum data extracted by the spectrometer to the XYZ color space are as follows:

X=k∫.sub.380 nm.sup.780 nmS(λ)R(λ)x(λ)dλ  (5)

Y=k∫.sub.380 nm.sup.780 nmS(λ)R(λ)y(λ)dλ  (6)

Z=k∫.sub.380 nm.sup.780 nmS(λ)R(λ)z(λ)dλ  (7)

where k is given by (8)

k=100/∫.sub.380 nm.sup.780 nmS(λ)y(λ)dλ  (8)

[0051] x(λ), y(λ), and z(λ) are color matching functions (CMF); S(λ) is the light-source spectrum for the endoscope. Because the Y value in the XYZ color space is proportional to the brightness, the maximum brightness of the light-source spectrum (saturation brightness) can be deduced by using (6). Then defining the maximum Y value to be 100, the ratio k for brightness can be deduced. By using (5) to (7), the reflection spectrum data can be converted to the XYZ values (XYZspectrum) defined by the XYZ color space.

[0052] In addition, the calibration matrix C in (9) can be used to calibrate endoscope images:

[C]=[XYZ.sub.Spectrum]×pinν([V])  (9)

where the variable matrix [V] is given by analyzing the possible error factors of an endoscope. The error factors include nonlinear response of endoscope, dark currents of endoscope, inaccurate filtering of color filters, and color offsets (for example, white balance). Thereby, the XYZ values (XYZ.sub.Spectrum) can be calibrated.

[0053] Since the result of third-order calculation for the narrow-band image and the white light image shows they are similar, the nonlinear response correction adopts a third-order equation. To calibrate nonlinear response of the endoscope, the following equation (10) is adopted:

V.sub.Non-linear=[X.sup.3Y.sup.3Z.sup.3X.sup.2Y.sup.2Z.sup.2X Y Z 1].sup.T  (10)

[0054] The dark current of a general endoscope is fixed. It will not change significantly as the incoming light varies. Thereby, the influence of dark currents can be considered constant. By defining the calibration variable for dark currents as VDark, the influence of dark currents can be expressed as (11):

V.sub.Dark=[α]  (11)

[0055] The calibration variable for inaccurate filtering of color filter and color offset can be defined as V.sub.Color. x(λ), y(λ), and z(λ) are the color matching functions for converting the RGB color space to the XYZ color space. Thereby, according to the relation among x(λ), y(λ), and z(λ), the possible combinations of X, Y, Z can be listed in equation (12) for calibrating inaccurate filtering of color filter and color offset:

V.sub.Color=[XYZ XY XZ YZ X Y Z].sup.T  (12)

[0056] According to equations (10) to (12), the calibrated variable matric V can be deduced in (13):

V=[X.sup.3Y.sup.3Z.sup.3 X.sup.2Y X.sup.2Z Y.sup.2Z XY.sup.2 XZ.sup.2 YZ.sup.2 XYZ X.sup.2Y.sup.2Z.sup.2 XY XZ YZ XYZ α].sup.T  (13)

[0057] By using the variable matrix V and the calibration matrix C, the calibrated X, Y, Z values [XYZ.sub.Correct] are given in (14):

[XYZ.sub.Ccorrect]=[C]×[V]  (14)

[0058] The average error of the white light image in [XYZ.sub.Correct] and [XYZ.sub.Spectrum] is 1.40; the average error of the narrow-band image in [XYZ.sub.Correct] and [XYZ.sub.Spectrum] is 2.39.

[0059] Because the above calculations adopt the visible-light band with wavelength between 380 and 780 nanometers, the calibration result of the endoscope must be expressed in chromatic aberration. [XYZ.sub.Correct] and [XYZ.sub.Spectrum] are converted to the corresponding Lab color space of the CIE DE2000. The conversion functions are equations (15) to (17):

[00002]$\begin{array}{cc}{L}^{*}=116f\left(\frac{Y}{Y_n}\right)-16& \left(15\right)\end{array}$$\begin{array}{cc}{a}^{*}=500\left[f\left(\frac{X}{X_n}\right)-f\left(\frac{Y}{Y_n}\right)\right]& \left(16\right)\end{array}$$\begin{array}{cc}{b}^{*}=200\left[f\left(\frac{Y}{Y_n}\right)-f\left(\frac{Z}{Z_n}\right)\right]& \left(17\right)\end{array}$

where f(n) is:

[00003]$\begin{array}{cc}f\left(n\right)=\{\begin{array}{c}{n}^{\frac{1}{3}},n>0.008856\\ 7.787n+0.137931,\mathrm{otherwise}\end{array}& \left(18\right)\end{array}$

[0060] The average chromatic aberration of the white light image before calibration reaches 11.60. After calibration, the average chromatic aberration is 2.84. The average chromatic aberration of the narrow-band image before calibration reaches 29.14. After calibration, the average chromatic aberration is 2.58.

[0061] In the step S10, as shown in FIG. 2B, the host 10 acquires an input image IMG from the image extraction unit 20. Like the reference image REF, the input image IMG can include a white light image and a narrow-band image. The image extraction unit 20 according to the present embodiment use the white light endoscope OLYMPUS EVIS LUCERA CV-260 SL to give the white light image and use the narrow-band endoscope OLYMPUS EVIS LUCERA CLV-260 to give the narrow-band image. In the step S15, the host 10 judges if the input image IMG complies with the resolution requirement of a hyperspectral image according to a resolution threshold value, for example, 10 nanometers. Then the input image IMG is converted to a hyperspectral image HYI according to the hyperspectral image information HAS given in the step S05. Afterwards, the step S20 is executed.

[0062] In the step S20, the host 10 extracts a plurality of first hyperspectral characteristic values F1 correspondingly according to the hyperspectral image HYI. In the step S25, the host 10 performs band selection on the plurality of first hyperspectral characteristic values F1 acquired in the step S20 according to a band BND of a cell for further performing principal component analysis (PCA). To simplify the calculation result, the characteristic values with lower variations are filtered out. Then the hyperspectral image HYI is simplified and giving a plurality of second characteristic values F2.

[0063] The formula for PCA is:

y.sub.i=α.sub.j1(x.sub.1i−x.sub.1)+α.sub.j2(x.sub.2i−x.sub.2)+ . . . +α.sub.jn(x.sub.ni−ī.sub.n)  (19)

[0064] x.sub.1i to x.sub.ni represent the spectrum intensity of the first to the n-th band BND. x.sub.1 to x.sub.n represent the spectrum expectation values (the average spectrum intensity) of the first to the n-th band BND. α.sub.j1 to α.sub.jn represent the characteristic vector coefficients of the covariance matrix while performing covariance calculations on the spectrum. After reduction of dimension for the 401-dimension spectrum information by using PCA, the first three dimensions are required and hence reducing calculation complexity. According to the present embodiment, the spectrum intensity values for two bands, including 415 and 540 nanometers, are extracted.

[0065] The principle of selecting the band BND is as follows. When the red light is absorbed by the hemoglobins in blood vessels at different depth, the capillaries in shallow mucosa tissues appear brown while the blood vessels in submucosa tissues appear green, resulting in significant visual layering and thus facilitating identification of affections in mucosa tissues. Thereby, according to the present embodiment, the bands 415 nm and 540 nm for globins crucial to detection of esophageal cancer cells are selected from the hyperspectral image HYI. Nonetheless, the present application is not limited to the embodiment. To detect different cells, different bands can be selected. Besides, by using the calibration values [XYZ.sub.Correct] and the corresponding reflection spectrum data [R.sub.Spectrum] of the 24 color checkers as described above, the corresponding conversion matrix M is calculated. pinv[V] is the pseudo inverse matrix of the matrix M:

[M]=[Score]×pinν([V.sub.Color])  (20)

[S.sub.Spectrum].sub.380-780=[EV][M][V.sub.Color]  (21)

[0066] By performing PCA on the reflection spectrum [R.sub.Spectrum], a plurality of principal components (EV) will be given. According to the adopted principal component, the principal component score [Score] will be given. According to the present embodiment, 10 principal components with stronger explanatory power (with weighting percentage 8.0417%, 8.2212%, 2.6422%, 0.609%, 0.22382%, 0.10432%, 0.054658%, 0.0472%, 0.02638%, 0.012184%) are adopted to perform dimension reduction operation and giving the simulation spectrum [S.sub.Spectrum].sub.380-780. The error between the simulation spectrum [S.sub.Spectrum].sub.380-780 and the corresponding [XYZ.sub.Spectrum] of the input image IMG is corrected from 11.60 to 2.85 in the white light image and from 29.14 to 2.60 in the narrow-band image, and hence achieving almost indistinguishable color errors by bare eyes. Thereby, when color re-rendering is required, the present application provides better performance, and superior hyperspectral images in the visible light band can be simulated.

[0067] In the step S30, as shown in FIG. 2C and FIG. 3, the host 10 uses a plurality of kernels C to detect the corresponding plurality of second characteristic values F2 of the input image IMG given by the steps S20 to S25, especially the values between 0 and 1. The plurality of kernels C include the corresponding selected characteristic values F22 of a plurality of selected images O2 of the one or more object-under test image O2 and the corresponding peripheral characteristic values F24 of the adjacent peripheral image BGI of the one or more object-under-test image O1 for filtering out the background image BG not including the object-under-test image O1. The host 10 performs convolution on each pixel of the input image according to a single shot multidetector model, respectively, for detecting the plurality of characteristic values. The plurality of kernels C correspond to the corresponding selected characteristic values F22 of the plurality of selected images O2 of the one or more object-under test image O2 and the corresponding peripheral characteristic values F24 of the adjacent peripheral image BGI.

[0068] Please refer again to FIG. 2C and FIG. 3. The input image IMG has m×n pixels. Since p channels are used to extract features, the plurality of kernels C are m×m×p units and m=n, for example, 1×1×p, 3×3×p, 5×5×p, 10×10×p, 19×19×p, or 38×38×p. The plurality of kernels C detect the object-under-test image O1 in the input image IMG and the background image BG for filtering out the background image BG and thus reducing processes on the background image BG in subsequent steps. The processing unit 12 converts the corresponding plurality of second characteristic values F2 of the input image IMG to the corresponding selected characteristic value F22 and the peripheral characteristic value F24. The processing unit 12 multiplying the second characteristic values F2 of the input image IMG using the kernels C and gives different convolution results R. If the values are identical, the result is 1; otherwise, the result is −1. Then the unrelated background image BG will be filtered out. As shown in FIG. 4, in the corresponding second characteristic values F2 of the input image IMG, a portion or all of the corresponding one or more object-under-test O1 of the plurality of selected characteristic values F22 is given. Thereby, the located region A of a portion or all of the one or more object-under-test O1 can be given.

[0069] In the step S33, as shown in FIG. 2D and FIG. 5, the host 10 gives one or more selected image O2 according to the located region A of the one or more obj ect-under-test image O1. In other words, the convolution program P acquires the selected characteristic value F22 and the peripheral characteristic value F24 and build one or more prediction box D correspondingly. As shown in FIG. 6, the edges of the input image IMG form the initial bounding box B. The side length of the prediction box D is min_size=s.sub.k; the maximum side length is √{square root over (min_size×maxsize)}.Math.max_size=s.sub.k+1. In the following equation (22), the side length s.sub.k can be calculated:

[00004]$\begin{array}{cc}{s}_k=s_{\min }+\frac{s_{\max }-s_{\min }}{m-1}\left(k-1\right),k\in \left[1,m\right]& \left(22\right)\end{array}$

[0070] Meanwhile, the height and width can be calculated according to the side length s.sub.k:

h.sub.k=s.sub.k√{square root over (α.sub.r)}  (23)

w.sub.k=s.sub.k/√{square root over (α.sub.r)}  (24)

where h.sub.k represents the height of the anchor box of the k-th feature map; w.sub.k represents the width of the rectangular anchor box; α.sub.r represents the length-to-width ratio of the prediction box D; and α.sub.r is greater than 0.

[0071] In the step S40, as shown in FIG. 2F and FIG. 6, the host 10 extracts the corresponding bounding box B of the input image IMG and executes the convolution program P and extracts a first central point Dc of the prediction box D and a second central point Bc of the bounding box B. The corresponding center displacement DIS of the first central point Dc and the second central point Bc can be calculated. Next, in the step S45, as shown in FIG. 2E and FIG. 7, the processing unit 12 of the host 10 further performs regression calculation LOOP according to the center displacement DIS between the prediction box D and the bounding box B using the following calculations:

Location of Prediction box D: d=(d.sup.cx, d.sup.cy, d.sup.w, d.sup.h)  (25)

Location of bounding box B: b=(b.sup.cx, b.sup.cy, b.sup.w, b.sup.h)  (26)

Sizing factor: l=(l.sup.cx, l.sup.cy, l.sup.w, l.sup.h)  (27)

b.sup.cx=d.sup.wl.sup.cx+d.sup.cx  (28)

b.sup.cy=d.sup.hl.sup.cy+d.sup.cy  (29)

b.sup.w=d.sup.wexp(l.sup.w)  (30)

b.sup.h=d.sup.hexp(l.sup.h)  (31)

[0072] First, the center of the bounding box B is aligned with the center of the prediction box D, meaning moving the central point of the bounding box B to the central point of the prediction box D. In other words, as shown in FIG. 6, the first central point Dc and the second central point Bc overlap. As shown in equations (28) and (29), the size of the bounding box B is sized to close to the size of the prediction box D. As shown in equation (30) and (31), after the above displacement and sizing, the bounding box B can be infinitely close to the location of the prediction box D. In the step S50, the processing unit 12 of the host 10 executes the convolutional neural network CNN in the convolution program P and performs regression calculations continuously until the bounding box B is infinitely close to the location of the prediction box D. By overlapping the prediction box D and the bounding box Bm the location of the object-under-test image O1 can be defined accurately. In other words, while moving the first central point Dc to the second central point Bc, the selected image (namely, the image inside the prediction box D) will be moved to the second central point Bc correspondingly.

[0073] In addition, to define the location of the object-under-test image O1 more accurately, a loss equation is further adopted:

L.sub.loc(x, l, g)=Σ.sub.i∈Pos.sup.NΣ.sub.m∈{cx,cy,w,h}x.sub.ij.sup.k smooth.sub.L1(l.sub.i.sup.m−ĝ.sub.j.sup.m)  (32)

[0074] Thereby, the error between the location of the prediction box D and the location of the object-under-test image O1 can be verified.

[0075] In the step S35, as shown in FIG. 2G, after the processing unit 12 of the host 10 locates the object-under-test image O1, compare the object-under-test image O1 with the sample image SA in the database 30 and giving a comparison result R. Next, in the step S60, as shown in FIG. 2H, the processing unit 12 of the host 10 classifies the input image IMG as a target-object image TA or a non-target-object image NTA using the convolution program P according to the comparison result R. When the convolution program P executed by the processing unit 12 of the host 10 cannot match the object-under-test image O1 in the prediction box D to the one or more sample image SA, the host 10 classifies the input image IMG as the non-target-object image NTA. Otherwise, the convolution program P executed by the processing unit 12 of the host 10 classifies the input image IMG as the target-object image TA. Furthermore, when the convolution program P executed by the processing unit 12 of the host classifies the input image IMG as the non-target-object image NTA, the convolution program P will continue to compare the one or more sample image SA with the object-under-test image O1. When the convolution program P judges that the similarity of the comparison result of the object-under-test image O1 with the target-object image TA is greater than a similarity threshold value (for example, similarity is defined between 0 and 1; the similarity threshold value is 0.5), the convolution program P classifies the input image IMG as the target-object image TA. Otherwise, the convolution program P classifies the input image IMG as the non-target-object image NTA. According to intersection over union (IOU), when the intersection between the target-object image TA with higher similarity and the target-object image TA with lower similarity is greater than an intersection threshold value (for example, intersection is defined between 0 and 1; the intersection threshold value is 0.5), the convolution program P remove the target-object image TA with lower similarity from the target-object image TA.

[0076] FIG. 8 shows a schematic diagram of the image according to an embodiment of the present application. As shown in the figure, the input image IMG is an esophagoscope image. By using the method for detecting object image using hyperspectral imaging by band selection according to the present application, the prediction box D and the bounding box B are added to the input image IMG. The input image IMG is compared with the sample image, which is an esophagoscope image of a dysplasia regions, and giving the comparison result R. The accuracy can reach 93.0%.

[0077] According to the method for detecting object image using hyperspectral imaging by band selection according to the present application, band selection is performed on the hyperspectral image HYI. In the hyperspectral image HYI, the spectrum intensity of different bands of visible colors can be identified. The multi-dimensional hyperspectral image HYI can filtered to keep the required bands (415 nm and 540 nm are the two bands adopted according to the present embodiment). The required bands are processed by dimension reduction before subsequent calculations. Then the convolutional neural network is used to classify into target-object image or non-target-object image. 1780 results of finding lesions in esophagoscope images according to the present application is compared with the results using input images IMG with white light imaging (WLI) or with narrow-band imaging (NBI). After band selection, the accuracy of judging dysplasia regions in hyperspectral images HYI is raised to 98% (WLI: 96%; NBI: 85%); the accuracy of judging esophageal cancer (SCC) using band-selected hyperspectral images HYI is raised to 93% (WLI: 92%; NBI: 82%). Medical personnel can utilize the present application as supporting evidence for diagnoses. The present application does not adjust the colors of the input image IMG. Instead, the present application filters the light with colors.

[0078] According to another embodiment of the present application, the output of the band-selected hyperspectral image is a general image. After the step S25, the method further comprises a step:

Step S28: Converting the first hyperspectral characteristic values to a simulation image according to band selection and the hyperspectral image information.

[0079] The present application further provides an embodiment. As shown in FIG. 2I, in the step S28, the host 10 performs band selection on the plurality of first hyperspectral characteristic values F1 according to the band BND and converts the hyperspectral information HAS given in the step S05 to a simulation image SIM.

[0080] In the step S25, PCA and dimension reduction calculations are performed on the reflection spectrum data [R.sub.Spectrum] to give the simulation spectrum [S.sub.Spectrum].sub.380-780. In the step S28, according to the equations (5) to (7), the simulation spectrum [S.sub.Spectrum].sub.380-780 is converted to the XYZ values defined in the XYZ color space. Afterwards, (33) is used to convert the XYZ values to the simulation image SIM.

[00005]$\begin{array}{cc}\left[\begin{array}{c}R\\ G\\ B\end{array}\right]=\left[\begin{array}{ccc}3.240479& -1.53715& -0.498535\\ -0.969256& 1.875991& 0.041556\\ 0.055648& -0.204043& 1.057311\end{array}\right]\left[\begin{array}{c}X\\ Y\\ Z\end{array}\right]& \left(33\right)\end{array}$

[0081] In the embodiment, the data after dimension reduction can be converted to a general image (RGB image). According to an embodiment, the selection band is 415 nm and 540 nm, which will generate simulation images SIM similar to narrow-band images. The penetration of light with different wavelengths into tissues differs. When the red light is absorbed by the hemoglobins in blood vessels at different depth, the capillaries in shallow mucosa tissues appear brown while the blood vessels in submucosa tissues appear green. The simulation images SIM according to the present application are generated by the two selection bands from the white light images with 401 bands. Consequently, the lesion region (the target-object image TA) originally with similar colors with the background appears prominent and hence improving the contrast with respect to the background image BG.

[0082] FIGS. 9A, 9B, 10A, 10B show schematic diagrams of the image according to an embodiment of the present application. For example, identify if a white light image from an endoscope matches esophageal cancers. The host 10 extracts the input image IMG and gives the hyperspectral image HYI according to the hyperspectral image information HAS. The hyperspectral image HYI include 401 bands between 380 and 780 nm. The bands BND 415 nm and 540 nm are selected. By performing convolution calculations, the location of the object-under-test image can be boxed and compared with the sample image. Then the prediction box D, the bounding box B, and the comparison result R will be given. According to the hyperspectral image information, the simulation image SIM will be generated. The simulation image SIM is a simulated narrow-band image. FIG. 9A compares with the sample image with white light imaging in the dysplasia regions. FIG. 10A compares with the sample image with white light imaging in the esophageal cancer (SCC) regions. FIG. 9B and FIG. 10B are the simulated NBI images of the above white light images processed by the method according to the present application. It is apparent that the lesion region (the target-object image TA) in the simulated NBI image is more prominent. By increasing the contrast with respect to the background image BG, observation by medical personnel can be facilitated.

[0083] To sum up, the present application provides a method for detecting object image using hyperspectral imaging by band selection. The host acquires the hyperspectral image information and converts the input image to the hyperspectral image according to the hyperspectral image information. Next, the convolution program is executed continuously on the hyperspectral image according to the band selection and enables the host to build the convolutional neural network for performing convolution on the input image from the image extraction unit. By filtering the region to be detected, the prediction box is generated on the input image. Then, by using the regression calculation, the bounding box is used to locate the location of the object-under-test image. Finally, comparison with the sample image is performed. By using the comparison result, the target-object image and the non-target-object image can be classified. Thereby, the purpose of identifying object image using the hyperspectral technology can be achieved.