IMAGING DEVICE AND IMAGING METHOD

Abstract

An imaging device and an imaging method are provided that are capable of, for example, accurately recognizing the state of blood circulation of an examinee when the examinee undergoes surgery. An imaging device includes: an excitation light source configured to irradiate an examinee with excitation light to excite a fluorescent dye administered to the examinee; a capturing unit configured to obtain a fluorescent image by capturing fluorescence generated from the fluorescent dye; a pixel value measurement unit configured to measure, over time, a pixel value of each pixel in a predetermined region of the fluorescent image; a gradation conversion unit configured to perform gradation conversion of the pixel values measured by the pixel value measurement unit to a predetermined number of gradations; a gradation image generation unit configured to generate a gradation image based on the pixel values subjected to the gradation conversion by the gradation conversion unit; and an image display unit configured to display the gradation image.

Claims

1. An imaging device comprising: an excitation light source configured to irradiate an examinee with excitation light to excite a fluorescent dye administered to the examinee; a capturing unit configured to obtain a fluorescent image by capturing fluorescence generated from the fluorescent dye; a pixel value measurement unit configured to measure, over time, a pixel value of each pixel in a predetermined region of the fluorescent image; a gradation conversion unit configured to perform gradation conversion of the pixel values measured by the pixel value measurement unit to a predetermined number of gradations; a gradation image generation unit configured to generate a gradation image based on the pixel values subjected to the gradation conversion by the gradation conversion unit; and an image display unit configured to display the gradation image.

2. The imaging device according to claim 1, wherein the gradation image generation unit generates the gradation image with a different color for each gradation.

3. The imaging device according to claim 1, further comprising a contour line generation unit configured to detect a boundary between each gradation in the gradation image and generate a contour line image representing each gradation, wherein the image display unit displays at least any one of the gradation image or the contour line image.

4. The imaging device according to claim 3, wherein the contour line generation unit generates the contour line image with a different thickness for each gradation.

5. The imaging device according to claim 3, wherein the contour line generation unit generates the contour line image with a different color for each gradation.

6. The imaging device according to claim 1, wherein the predetermined region is an entire screen area of the fluorescent image.

7. The imaging device according to claim 1, wherein the predetermined region is a plurality of regions in the fluorescent image.

8. The imaging device according to claim 1, wherein the capturing unit captures a visible image in real time in an area corresponding to the fluorescent image, and the image display unit superimposes the gradation image on the visible image for display.

9. An imaging device comprising: an excitation light source configured to irradiate an examinee with excitation light to excite a fluorescent dye administered to the examinee; a capturing unit configured to obtain a fluorescent image by capturing fluorescence generated from the fluorescent dye; an image storage unit configured to store the fluorescent image over time; a pixel value measurement unit configured to measure, over time, a pixel value of each pixel in a predetermined region of the fluorescent image based on the fluorescent image stored in the image storage unit; a change curve creation unit configured to create a temporal change curve of the pixel value of each pixel in the predetermined region based on the pixel values measured by the pixel value measurement unit; an index calculation unit configured to analyze the change curve of each pixel in the predetermined region and calculate a predetermined index for each pixel in the predetermined region; a gradation conversion unit configured to perform gradation conversion of the predetermined index to a predetermined number of gradations; a gradation image generation unit configured to generate a gradation image based on the index subjected to the gradation conversion by the gradation conversion unit; and an image display unit configured to display the gradation image.

10. The imaging device according to claim 9, wherein the predetermined index is any one of gradient, a peak value, time to peak, an integral, or a curve shape of the change curve.

11. The imaging device according to claim 9, wherein the gradation image generation unit generates the gradation image with a different color for each gradation.

12. The imaging device according to claim 9, further comprising a contour line generation unit configured to detect a boundary between each gradation in the gradation image and generate a contour line image representing each gradation, wherein the image display unit is capable of switching between the gradation image and the contour line image for display.

13. The imaging device according to claim 12, wherein the contour line generation unit generates the contour line image with a different thickness for each gradation.

14. The imaging device according to claim 12, wherein the contour line generation unit generates the contour line image with a different color for each gradation.

15. The imaging device according to claim 9, wherein the predetermined region is an entire screen area of the fluorescent image.

16. The imaging device according to claim 9, wherein the predetermined region is a plurality of regions in the fluorescent image.

17. The imaging device according to claim 9, wherein the capturing unit captures a visible image in real time in an area corresponding to the fluorescent image, and the image display unit superimposes the gradation image on the visible image for display.

18. An imaging method comprising the steps of: irradiating an examinee with excitation light to excite a fluorescent dye administered to the examinee; obtaining a fluorescent image by capturing fluorescence generated from the fluorescent dye; measuring, over time, a pixel value of each pixel in a predetermined region of the fluorescent image; performing gradation conversion of the measured pixel values to a predetermined number of gradations; generating a gradation image based on the pixel values subjected to the gradation conversion; and displaying the gradation image.

19. The imaging method according to claim 18, wherein the generating a gradation image generates the gradation image with a different color for each gradation.

20. The imaging method according to claim 18, further comprising detecting a boundary between each gradation in the gradation image and generating a contour line image representing each gradation, wherein the displaying a gradation image displays at least any one of the gradation image or the contour line image.

21. The imaging method according to claim 20, wherein the generating a contour line image generates the contour line image with a different thickness for each gradation.

22. The imaging method according to claim 20, wherein the generating a contour line image generates the contour line image with a different color for each gradation.

23. The imaging method according to claim 18, wherein the predetermined region is an entire screen area of the fluorescent image.

24. The imaging method according to claim 18, wherein the predetermined region is a plurality of regions in the fluorescent image.

25. The imaging method according to claim 18, further comprising capturing a visible image in real time in an area corresponding to the fluorescent image, wherein the displaying a gradation image superimposes the gradation image on the visible image for display.

26. An imaging method comprising the steps of: irradiating an examinee with excitation light to excite a fluorescent dye administered to the examinee; obtaining a fluorescent image by capturing fluorescence generated from the fluorescent dye; storing the fluorescent image over time; measuring, over time, a pixel value of each pixel in a predetermined region of the fluorescent image based on the fluorescent image subjected to the storing; creating a temporal change curve of the pixel value of each pixel in the predetermined region based on the measured pixel values; analyzing the change curve of each pixel in the predetermined region and calculating a predetermined index for each pixel in the predetermined region; performing gradation conversion of the predetermined index to a predetermined number of gradations; generating a gradation image based on the index subjected to the gradation conversion; and displaying the gradation image.

27. The imaging method according to claim 26, wherein the predetermined index is any one of gradient, a peak value, time to peak, an integral, or a curve shape of the change curve.

28. The imaging method according to claim 26, wherein the generating a gradation image generates the gradation image with a different color for each gradation.

29. The imaging method according to claim 26, further comprising detecting a boundary between each gradation in the gradation image and generating a contour line image representing each gradation, wherein the displaying a gradation image is capable of switching between the gradation image and the contour line image for display.

30. The imaging method according to claim 29, wherein the generating a contour line image generates the contour line image with a different thickness for each gradation.

31. The imaging method according to claim 29, wherein the generating a contour line image generates the contour line image with a different color for each gradation.

32. The imaging method according to claim 26, wherein the predetermined region is an entire screen area of the fluorescent image.

33. The imaging method according to claim 26, wherein the predetermined region is a plurality of regions in the fluorescent image.

34. The imaging method according to claim 26, further comprising capturing in real time a visible image in an area corresponding to the fluorescent image, wherein the displaying a gradation image superimposes the gradation image on the visible image for display.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0011] FIG. 1 is a perspective view illustrating an imaging device in a first embodiment of the present invention.

[0012] FIG. 2 is a side view of the imaging device illustrated in FIG. 1.

[0013] FIG. 3 is a plan view of the imaging device illustrated in FIG. 1.

[0014] FIG. 4 is a perspective view of an illumination and capturing unit provided in the imaging device illustrated in FIG. 1.

[0015] FIG. 5 is a schematic view of a capturing unit in the illumination and capturing unit illustrated in FIG. 4.

[0016] FIG. 6 is a block diagram illustrating a major control system of the imaging device illustrated in FIG. 1.

[0017] FIG. 7 is a drawing (flow chart) sequentially illustrating the steps executed by the imaging device illustrated in FIG. 1.

[0018] FIG. 8 is an example of an image obtained by executing the steps illustrated in FIG. 7.

[0019] FIG. 9 is another example of an image obtained by executing the steps illustrated in FIG. 7.

[0020] FIG. 10 is still another example of an image obtained by executing the steps illustrated in FIG. 7.

[0021] FIG. 11 is still another example of an image obtained by executing the steps illustrated in FIG. 7.

[0022] FIG. 12 is a block diagram illustrating a major control system of an imaging device (second embodiment) of the present invention.

[0023] FIG. 13 is a drawing sequentially illustrating the steps executed by the imaging device illustrated in FIG. 12.

[0024] FIG. 14 is a graph illustrating temporal variation of a pixel value obtained from TIC analysis by the imaging device illustrated in FIG. 12.

[0025] FIG. 15 is an example of an image obtained by an imaging device (third embodiment) of the present invention.

[0026] FIG. 16 is a graph illustrating temporal variation of a pixel value obtained from TIC analysis by the imaging device illustrated in FIG. 15.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0027] The imaging device and the imaging method of the present invention are described below in detail based on preferred embodiments illustrated in the attached drawings.

First Embodiment

[0028] FIG. 1 is a perspective view illustrating an imaging device in the first embodiment of the present invention. FIG. 2 is a side view of the imaging device illustrated in FIG. 1. FIG. 3 is a plan view of the imaging device illustrated in FIG. 1. FIG. 4 is a perspective view of an illumination and capturing unit provided in the imaging device illustrated in FIG. 1. FIG. 5 is a schematic view of a capturing unit in the illumination and capturing unit illustrated in FIG. 4. FIG. 6 is a block diagram illustrating a major control system of the imaging device illustrated in FIG. 1. FIG. 7 is a drawing sequentially illustrating the steps executed by the imaging device illustrated in FIG. 1. FIG. 8 is an example of an image obtained by executing the steps illustrated in FIG. 7. FIG. 9 is another example of an image obtained by executing the steps illustrated in FIG. 7. FIG. 10 is still another example of an image obtained by executing the steps illustrated in FIG. 7. FIG. 11 is still another example of an image obtained by executing the steps illustrated in FIG. 7. In the following description, for the convenience of description, the upper side in FIGS. 1, 2, and 4 is referred to as top (above) and the lower side as bottom (below).

[0029] An imaging device 1 illustrated in FIG. 1 is a device to irradiate indocyanine green (ICG) as a fluorescent dye injected into the body of an examinee ST with excitation light and capture fluorescence emitted from the indocyanine green. Use of the imaging device 1 allows accurate recognition of, for example, the state of blood circulation of the examinee ST when the examinee ST undergoes surgery.

[0030] The imaging device 1 includes a cart 11 provided with four wheels 13, an arm mechanism 30 arranged around a front side (on the left in FIGS. 2 and 3) in a travel direction of the cart 11 on a top surface of the cart 11, an illumination and capturing unit 12 arranged in the arm mechanism 30 via a sub-arm 41, and an image display unit 15 as a monitor. To a rear side in the travel direction of the cart 11, a handle 14 used to move the cart 11 is attached. On the top surface of the cart 11, a recess 16 is formed to mount a remote controller for remote operation of the imaging device 1.

[0031] As illustrated in FIG. 7, the imaging device 1 is capable of executing an imaging method including the steps, in this order of steps, of visible light irradiating, visible image obtaining, excitation light irradiating, fluorescent image obtaining, pixel value measuring, gradation converting, gradation image generating, contour line generating, and gradation image displaying.

[0032] The arm mechanism 30 described above is arranged on the front side in the travel direction of the cart 11. The arm mechanism 30 is provided with a first arm member 31 coupled by a hinge 33 to a support portion 37 arranged on a column 36 set upright on the front side in the travel direction of the cart 11. The first arm member 31 is swingable relative to the cart 11 via the column 36 and the support portion 37 by the action of the hinge 33. The image display unit 15 described above is attached to the column 36.

[0033] The first arm member 31 has an upper end coupled to a second arm member 32 by a hinge 34. The second arm member 32 is swingable relative to the first arm member 31 by the action of the hinge 34. This allows the first arm member 31 and the second arm member 32 to be in a capturing position, as illustrated by imaginary lines of the reference sign C in FIG. 2, in which the first arm member 31 and the second arm member 32 are opened at a predetermined angle about the hinge 34 as a coupling portion between the first arm member 31 and the second arm member 32 and a standby position, as illustrated by solid lines of the reference sign A in FIGS. 1 through 3, in which the first arm member 31 and the second arm member 32 are arranged close to each other.

[0034] The second arm member 32 has a lower end coupled to a support portion 43 by a hinge 35. The support portion 43 is swingable relative to the second arm member 32 by the action of the hinge 35. The support portion 43 supports a rotating shaft 42. The sub-arm 41 supporting the illumination and capturing unit 12 then pivots about the rotating shaft 42 arranged at the distal end of the second arm member 32. This causes the illumination and capturing unit 12 to move, due to the pivoting of the sub-arm 41, between a front side position in the travel direction of the cart 11 with respect to the arm mechanism 30 for taking the capturing position as illustrated by the solid lines of the reference sign A in FIGS. 1 through 3 or the standby position as illustrated by the imaginary lines of the reference sign C in FIG. 2 and a rear side position in the travel direction of the cart 11 with respect to the arm mechanism 30 in a position to move the cart 11 as illustrated by imaginary lines of the reference sign B in FIGS. 2 and 3.

[0035] As illustrated in FIG. 4, the illumination and capturing unit 12 has a capturing unit 21 as a camera provided with a plurality of image sensors capable of capturing visible light and near infrared light, a visible light source 22 arranged on the outer periphery of the capturing unit 21, and an excitation light source 23 arranged on the outer periphery of the visible light source 22.

[0036] The visible light source 22 irradiates visible light such as, for example, white light. It is thus possible to perform the visible light irradiation step to irradiate the examinee ST with visible light.

[0037] The excitation light source 23 irradiates excitation light to excite the fluorescent dye. It is thus possible to perform the excitation light irradiation step to irradiate the examinee ST subjected to administration of a fluorescent dye with excitation light to excite the fluorescent dye. In the case of the fluorescent dye being indocyanine green, near infrared light with a wavelength of, for example, 810 nm is preferably used as the excitation light to excite the indocyanine green. From the indocyanine green irradiated with the near infrared light at 810 nm, near infrared light with a peak of 845 nm approximately is emitted as fluorescence.

[0038] It should be noted that, although the visible light source 22, the excitation light source 23, and the capturing unit 21 are integrated into the illumination and capturing unit 12 in the present embodiment, the visible light source 22 and the excitation light source 23 may be arranged separately from the capturing unit 21.

[0039] As illustrated in FIG. 5, the capturing unit 21 has a movable lens 54 to reciprocate for focusing, a wavelength selection filter 53, a visible light image sensor 51, and a fluorescence image sensor 52. The visible light image sensor 51 and the fluorescence image sensor 52 are configured with a CMOS or a CCD. The visible light and the fluorescence incident on the capturing unit 21 coaxially along its optical axis L pass through the movable lens 54 constituting a focusing mechanism and then reach the wavelength selection filter 53.

[0040] Out of the visible light and the fluorescence that are coaxial, the fluorescence passes through the wavelength selection filter 53 and is incident on the fluorescence image sensor 52. It is thus possible to perform the fluorescent image obtaining step to capture fluorescence generated from the fluorescent dye and obtain a fluorescent image IM2 of the examinee ST.

[0041] In addition, out of the visible light and the fluorescence that are coaxial, the visible light is reflected on the wavelength selection filter 53 and is incident on the visible light image sensor 51. It is thus possible to perform the visible image obtaining step to capture a visible image IM1 in real time in an area corresponding to the fluorescent image IM2 and obtain the visible image IM1 of the examinee ST. On the image display unit 15, a synthetic image IM3 is then displayed in which the visible image IM1 and the fluorescent image IM2 are synthesized.

[0042] In the illumination and capturing unit 12, by the action of the focusing mechanism including the movable lens 54, the visible light is focused on the visible light image sensor 51 and the fluorescence is focused on the fluorescence image sensor 52.

[0043] It should be noted that, although the visible image obtaining step and the fluorescent image obtaining step are executed in the order of the visible image obtaining step and then the fluorescent image obtaining step in the procedure illustrated in FIG. 7, the order of executing these steps is not limited to above and, for example, they may be performed in the order of the fluorescent image obtaining step and then the visible image obtaining step, that is, in an inverse order or the visible image obtaining step and the fluorescent image obtaining step may be performed in parallel, that is, in synchronization.

[0044] As illustrated in FIG. 6, the imaging device 1 includes a control unit 60 having a CPU to execute logical operation, a ROM to store an operation program necessary for device control, a RAM to temporarily store data or the like during control, and the like to control the entire device. The control unit 60 has an image processing unit 61 to execute various types of image processing to the visible image IM1 and the fluorescent image IM2. The image processing unit 61 is provided with a pixel value measurement unit 66 configured to measure, over time, a pixel value of each pixel in a predetermined region of the fluorescent image IM2, a gradation conversion unit 68 configured to perform gradation conversion of the pixel values measured by the pixel value measurement unit 66 to a predetermined number of gradations, a gradation image generation unit 69 configured to generate a gradation image IM4 based on the pixel values subjected to the gradation conversion by the gradation conversion unit 68, and a contour line generation unit 70 configured to detect a boundary between each gradation in the gradation image IM4 and generate a contour line image IM5 representing each gradation.

[0045] In this context, the predetermined region of the fluorescent image IM2 means a region generally called as a region of interest (ROI). In the imaging device 1, operation of an input unit 62 allows arbitrary setting and change of the position, the size (range), and the like of the ROI. The ROI may be the entire screen area of the fluorescent image IM2 or may be part of the fluorescent image IM2. In the former case, that is, the case where the ROI is the entire screen area of the fluorescent image IM2, it is possible to prevent, for example, oversight of the state of blood circulation depending on the case of the examinee ST. In addition, in the case where the ROI is the entire screen area of the fluorescent image IM2, it is preferred to perform a process such as, for example, reduction in a frame rate and reduction in screen size. Examples of the case of reduction in frame rate include a change from 60 fps to 15 fps and the like. Examples of the case of reduction in screen size include a change from 12801024 to 640512 and the like. In the latter case, that is, the case where the ROI is part of the fluorescent image IM2, it is possible to improve, for example, the image processing speed of the image processing unit 61.

[0046] The fluorescent dye administered to the examinee ST may be directly carried away without remaining in the ROI, or on the contrary, may mostly remain in part of the ROI. The pixel value of each pixel in the predetermined region of the fluorescent image IM2, that is, in the ROI is a value correlated to the concentration of the fluorescent dye in the ROI. The pixel value measurement unit 66 is capable of performing the pixel value measurement step to measure, over time, the pixel value of each pixel in the ROI.

[0047] The gradation conversion unit 68 is capable of performing the gradation conversion step to perform gradation conversion of the pixel values measured by the pixel value measurement unit 66 to a predetermined number of gradations. When the number of gradations is four, for example, the gradation conversion unit 68 sets four thresholds different in range (magnitude) to obtain the threshold in which range a pixel value measured by the pixel value measurement unit 66 belongs to. Based on this result, it is possible to determine which gradation out of the four gradations the pixel value belongs to. In the imaging device 1, the number of gradations and the ranges of thresholds may be arbitrarily set and changed by operating the input unit 62.

[0048] The gradation image generation unit 69 is capable of performing the gradation image generation step to generate the gradation image IM4 based on the pixel values subjected to the gradation conversion by the gradation conversion unit 68. In the case where the number of gradations in the gradation conversion unit 68 is four as described above, the gradation image generation unit 69 processes the image by, for example, classifying the pixel values belonging to each gradation into one respective group, to display the group as one continuous region. It is thus possible to generate the gradation image IM4 with the gradations divided into four, for example, as illustrated in FIGS. 8 and 9.

[0049] The contour line generation unit 70 is capable of performing the contour line generation step to generate the contour line image IM5 representing each gradation by detecting a boundary between each adjacent gradation in the gradation image IM4, that is, an edge as a contour of each gradation. The contour line generation unit 70 processes the image to display the detected boundary as a contour line surrounding each gradation. It is thus possible to generate the contour line image IM5 having four contour lines as illustrated in, for example, FIGS. 9 through 11.

[0050] The control unit 60 is electrically connected to the input unit 62 to input various types of information by an operator. The control unit 60 is also electrically connected to the image display unit 15 described above. The image display unit 15 is thus capable of performing the gradation image display step to display the synthetic image IM3 containing the gradation image IM4 and the like. The input unit 62 may be arranged in a remote controller for remote operation of the imaging device 1, or when the image display unit 15 is configured with a touch screen, may be arranged on the screen of the image display unit 15, or may be arranged in the cart 11.

[0051] The control unit 60 is also electrically connected to the illumination and capturing unit 12 provided with the capturing unit 21, the visible light source 22, and the excitation light source 23. The illumination and capturing unit 12 is thus capable of performing the visible light irradiation step, the visible image obtaining step, the excitation light irradiation step, and the fluorescent image obtaining step described above.

[0052] The case of recognizing the state of blood circulation of the examinee ST using the imaging device 1 for surgery on the examinee ST is then described with reference to the flow chart illustrated in FIG. 7 and the images illustrated in FIGS. 8 through 11. It should be noted that, prior to the surgery, the fluorescent dye is already administered to the examinee ST. In the imaging device 1, the position and size of the ROI are also set in advance.

[0053] First, the visible light source 22 is activated to establish a visible light irradiation state in which the examinee ST is irradiated with visible light (visible light irradiation step).

[0054] The capturing unit 21 is then activated while the visible light irradiation state is maintained to obtain the visible image IM1 (visible image obtaining step). The visible image IM1 as illustrated in FIGS. 8 through 11 is thus obtained. After the visible image IM1 is obtained, it is preferred to stop irradiation of visible light by the visible light source 22.

[0055] The excitation light source 23 is then activated to establish an excitation light irradiation state in which the examinee ST is irradiated with excitation light (excitation light irradiation step). It is thus possible to excite the fluorescent dye in the examinee ST and therefore to generate fluorescence from the fluorescent dye.

[0056] The capturing unit 21 is then activated while the excitation light irradiation state is maintained to obtain the fluorescent image IM2 (fluorescent image obtaining step). After the fluorescent image IM2 is obtained, it is preferred to stop irradiation of excitation light by the excitation light source 23.

[0057] Then, while the fluorescent image IM2 is obtained over time, the pixel value measurement unit 66 measures, over time, the pixel value of each pixel in the ROI in the fluorescent image IM2 (pixel value measurement step).

[0058] The gradation conversion unit 68 then performs gradation conversion of the pixel values measured by the pixel value measurement unit 66 to, for example, four gradations (gradation conversion step).

[0059] Then, based on the pixel values subjected to the gradation conversion by the gradation conversion unit 68, the gradation image generation unit 69 generates the gradation image IM4 (gradation image generation step). The fluorescent image IM2 containing the gradation image IM4 as illustrated in FIG. 8 is thus obtained.

[0060] In addition, the gradation image generation unit 69 is capable of generating the gradation image IM4 with a different color for each gradation. The color of each gradation is preferably selected based on, for example, HSV or HSL as a type of color space. Still in addition, the gradation image generation unit 69 is allowed to use, for example, complementary colors for colors of adjacent gradations. Such a gradation image IM4 causes the boundary between each gradation in the gradation image IM4 to become clear. It is also possible to intuitively recognize the state of blood circulation by appropriately setting the color of each gradation.

[0061] The contour line generation unit 70 then detects the boundary between each adjacent gradation in the gradation image IM4 to generate the contour line image IM5 representing each gradation (contour line generation step). The fluorescent image IM2 is thus obtained that contains the contour line image IM5 as illustrated in FIGS. 9 through 11. The contour line image IM5 is capable of emphasizing, that is, gaining more visibility for the boundary between adjacent gradations.

[0062] The contour line generation unit 70 is capable of generating the contour line image IM5 with a different color for each gradation, that is, different in color for each contour line. The color of each contour line is preferably selected based on, for example, HSV or HSL as a type of color space. In addition, the color of each contour line is preferably different from the color of each gradation. Such a contour line image IM5 is capable of more emphasis on the boundary between adjacent gradations.

[0063] The contour line generation unit 70 is also capable of generating the contour line image IM5 with a different thickness for each gradation, that is, different in thickness for each contour line. In this case, the fluorescent image IM2 is obtained that contains the contour line image IM5 as illustrated in FIG. 11. Such a contour line image IM5 allows the thickness of the contour line to reflect the brightness of each gradation. For example, the contour line may be relatively thick in a region of relatively bright gradation and the contour line may be relatively thin in a region of relatively dark gradation. This also allows more emphasis on the boundary between adjacent gradations.

[0064] The image display unit 15 then displays the synthetic image IM3 (gradation image display step). It is thus possible to display the synthetic image IM3 as illustrated in FIGS. 8 through 11 on the image display unit 15.

[0065] The synthetic image IM3 illustrated in FIG. 8 is an image where the gradation image IM4 is superimposed on the visible image IM1. This allows the doctor to readily visually recognize the boundary between each gradation in the gradation image IM4. Based on the boundary, the doctor can accurately recognize the boundary portion between good and bad states of blood circulation. This also allows the doctor, regardless of the level of the skills, to accurately determine the area to cut off a blood vessel, that is, a cut off line.

[0066] The synthetic image IM3 illustrated in FIG. 9 is an image where the gradation image IM4 and the contour line image IM5 are superimposed on the visible image IM1. This allows emphasizing the boundary between adjacent gradations in the gradation image IM4. Then, the doctor can more readily visually recognize the boundary between each gradation in the gradation image IM4.

[0067] The synthetic image IM3 illustrated in FIG. 10 is an image where the contour line image IM5 is superimposed on the visible image IM1. Such a synthetic image IM3 is suitable for the case of not using the gradation image IM4. This allows improvement in the image processing speed of the image processing unit 61.

[0068] The synthetic image IM3 illustrated in FIG. 11 is an image where the contour line image IM5 is superimposed on the visible image IM1 and is same as the synthetic image IM3 illustrated in FIG. 10 except for having a different thickness for each contour line. This allows emphasizing the boundary between adjacent gradations instead of the gradation image IM4, and thus the doctor can more readily visually recognize the boundary between each gradation.

[0069] As just described, use of the imaging device 1 for surgery on the examinee ST allows accurate recognition of the state of blood circulation of the examinee ST. It is thus possible to safely perform an operation on the examinee ST.

[0070] In addition, the imaging device 1 is capable of displaying at least any one of the gradation image IM4 or the contour line image IM5 or the gradation image IM4 and the contour line image IM5 by switching them on the image display unit 15. This allows the doctor to select the synthetic image IM3 containing the gradation image IM4, the synthetic image IM3 containing the contour line image IM5, or the synthetic image IM3 containing the gradation image IM4 and the contour line image IM5 in accordance with the preference to use for the operation.

Second Embodiment

[0071] FIG. 12 is a block diagram illustrating a major control system of an imaging device (second embodiment) of the present invention. FIG. 13 is a drawing sequentially illustrating the steps executed by the imaging device illustrated in FIG. 12. FIG. 14 is a graph illustrating temporal variation of a pixel value obtained from TIC analysis by the imaging device illustrated in FIG. 12.

[0072] The imaging device and the imaging method in the second embodiment of the present invention are described below with reference to these drawings while the description is mainly given to the difference from the embodiment described earlier to omit the description on similar points.

[0073] The present embodiment is same as the first embodiment except for the imaging device different in configuration.

[0074] As illustrated in FIG. 12, in the present embodiment, an imaging device 1 includes an image storage unit 63 configured to store, over time, an image captured by the capturing unit 21. The image storage unit 63 is electrically connected to a control unit 60. The image storage unit 63 is configured with a fluorescent image storage unit 64 to store the fluorescent image IM2 over time and a visible image storage unit 65 to store the visible image IM1 over time. It should be noted that the image storage unit 63 may have, instead of the fluorescent image storage unit 64 and the visible image storage unit 65, a synthesis image storage unit to store the synthetic image IM3 over time.

[0075] The control unit 60 further has, in addition to the image processing unit 61, a change curve creation unit 67 configured to create a temporal change curve of a pixel value of each pixel in the ROI based on the pixel values measured by the pixel value measurement unit 66 and an index calculation unit 71 configured to analyze the change curve of each pixel in the ROI and calculate a predetermined index for each pixel in the ROI.

[0076] As illustrated in FIG. 13, the imaging device 1 is capable of executing an imaging method including the steps, in this order of steps, of visible light irradiating, visible image obtaining, visible image storing, excitation light irradiating, fluorescent image obtaining, fluorescent image storing, pixel value measuring, change curve creating, index calculating, gradation converting, gradation image generating, and gradation image displaying.

[0077] The visible image storage step is a step of storing the visible image IM1 over time and is performed by the visible image storage unit 65.

[0078] The fluorescent image storage step is a step of storing the fluorescent image IM2 over time and is performed by the fluorescent image storage unit 64. The pixel value measurement unit 66 is then capable of measuring, over time, the pixel value of each pixel in the ROI of the fluorescent image IM2 as described above based on the fluorescent image IM2 stored in the image storage unit 63 (pixel value measurement step).

[0079] The change curve creation step is a step of creating a temporal change curve of the pixel value of each pixel in the ROI as illustrated in FIG. 14 based on the pixel values measured by the pixel value measurement unit 66 and is performed by the change curve creation unit 67. The change curve creation unit 67 displays, for example, the pixel values in the ROI as a graph.

[0080] The index calculation step is a step of analyzing the change curve of each pixel in the ROI and calculating a predetermined index for each pixel in the ROI and is performed by the index calculation unit 71. The predetermined index is an index to recognize the state of blood flow, and a preferred example of the index is any one of, but not particularly limited to, gradient of the change curve, a peak value, the time to peak, an integral, a curve shape, fluorescence starting time (the time of first detecting fluorescence in each pixel), or a final pixel value (stabilized pixel value after reaching the peak and then attenuated) of time intensity curve (TIC) analysis.

[0081] The index calculation unit 71 is capable of calculating any one of the above indices by TIC analysis. For example, the raising gradient in FIG. 14 is a parameter correlated to the blood flow, and the time to peak is a parameter (time) between administration of the fluorescent dye and delivery of the fluorescent dye to a target tissue. Then, it is finally possible to recognize the state of blood circulation, that is, how well the blood flows from any of the parameters.

[0082] The gradation conversion unit 68 is capable of performing gradation conversion of the predetermined index to a predetermined number of gradations (gradation conversion step). The gradation image generation unit 69 is capable of generating the gradation image IM4 based on the index subjected to the gradation conversion by the gradation conversion unit 68 (gradation image generation step). It is thus possible to, similar to the first embodiment, accurately recognize the state of blood circulation of the examinee ST when the examinee ST undergoes surgery.

Third Embodiment

[0083] FIG. 15 is an example of an image obtained by an imaging device (third embodiment) of the present invention. FIG. 16 is a graph illustrating temporal variation of a pixel value obtained from TIC analysis by the imaging device illustrated in FIG. 15.

[0084] The imaging device and the imaging method in the third embodiment of the present invention are described below with reference to these drawings, and the description is mainly given to the difference from the embodiments described earlier to omit the description on similar points.

[0085] The present embodiment is same as the second embodiment except for the difference in the number of ROI set in the fluorescent image IM2.

[0086] As illustrated in FIG. 15, in the present embodiment, two regions (region A and region B) are set as ROIs in the fluorescent image IM2. As illustrated in FIG. 16, in the fluorescent image IM2, fluorescence is generated in the region A, and after a delay, that is, after a time lag, fluorescence is generated in the region B. In this case, for example, measurement of the time lag in fluorescence generation between the region A and the region B allows recognition of the state of blood circulation.

[0087] It should be noted that the number of ROI set in the fluorescent image IM2 is not limited to two as in the present embodiment and may be, for example, three or more.

[0088] Although the imaging device and the imaging method of the present invention has been described above with reference to the illustrated embodiments, the present invention is not limited to above and each component constituting the imaging device and the imaging method may be substituted by an arbitrary configuration capable of exhibiting the same function. In addition, an arbitrary configuration may be added.

[0089] The imaging device of the present invention may be a combination of two or more arbitrary configurations (features) in the respective embodiments above. For example, the imaging device in the second embodiment may be configured to include a contour line generation unit same as that of the imaging device in the first embodiment.

[0090] Although the respective embodiments above are described for the case of emitting fluorescence in the near infrared region with a peak of roughly 810 nm from indocyanine green by using the indocyanine green as a material containing the fluorescent dye and irradiating the indocyanine green with near infrared light approximately from 600 nm to 850 nm as excitation light, light other than near infrared rays may be used.

[0091] Depending on the case of the examinee, 5-aminolevulinic acid (5-ALA), for example, may be used as the fluorescent dye instead of the indocyanine green.

Aspects

[0092] Those skilled in the art understand that the plurality of embodiments described above as exemplifications are specific examples of the following aspects.

First Aspect The imaging device according to an aspect includes:

[0093] an excitation light source configured to irradiate an examinee with excitation light to excite a fluorescent dye administered to the examinee;

[0094] a capturing unit configured to obtain a fluorescent image by capturing fluorescence generated from the fluorescent dye;

[0095] a pixel value measurement unit configured to measure, over time, a pixel value of each pixel in a predetermined region of the fluorescent image;

[0096] a gradation conversion unit configured to perform gradation conversion of the pixel values measured by the pixel value measurement unit to a predetermined number of gradations;

[0097] a gradation image generation unit configured to generate a gradation image based on the pixel values subjected to the gradation conversion by the gradation conversion unit; and

[0098] an image display unit configured to display the gradation image.

[0099] In accordance with the imaging device according to the first aspect, the boundary between each gradation in the gradation image becomes clear. It is also possible to intuitively recognize the state of blood circulation by appropriately setting the color of each gradation. It is thus possible to accurately recognize, for example, the state of blood circulation of an examinee when the examinee undergoes surgery.

Second Aspect In the imaging device according to the first aspect,

[0100] the gradation image generation unit generates the gradation image with a different color for each gradation.

[0101] In accordance with the imaging device according to the second aspect, the boundary between each gradation in the gradation image becomes clear.

Third Aspect In the imaging device according to the first or second aspect,

[0102] the imaging device further includes a contour line generation unit configured to detect a boundary between each gradation in the gradation image and generate a contour line image representing each gradation, wherein

[0103] the image display unit displays at least any one of the gradation image or the contour line image.

[0104] In accordance with the imaging device according to the third aspect, it is possible to readily visually recognize the state of blood circulation.

Fourth Aspect In the imaging device according to the third aspect,

[0105] the contour line generation unit generates the contour line image with a different thickness for each gradation.

[0106] In accordance with the imaging device according to the fourth aspect, it is possible to reflect the brightness of each gradation on the thickness of the contour line.

Fifth Aspect In the imaging device according to the third or fourth aspect,

[0107] the contour line generation unit generates the contour line image with a different color for each gradation.

[0108] In accordance with the imaging device according to the fifth aspect, the boundary between each gradation in the gradation image becomes clear.

Sixth Aspect In the imaging device according to any one of the first through fifth aspects,

[0109] the predetermined region is an entire screen area of the fluorescent image.

[0110] In accordance with the imaging device according to the sixth aspect, it is possible to prevent, for example, oversight of the state of blood circulation depending on the case of the examinee.

Seventh Aspect In the imaging device according to any one of the first through fifth aspects,

[0111] the predetermined region is a plurality of regions in the fluorescent image.

[0112] In accordance with the imaging device according to the seventh aspect, it is possible to improve, for example, the image processing speed.

Eighth Aspect In the imaging device according to any one of the first through seventh aspects,

[0113] the capturing unit captures a visible image in real time in an area corresponding to the fluorescent image, and

[0114] the image display unit superimposes the gradation image on the visible image for display.

[0115] In accordance with the imaging device according to the eighth aspect, it is possible to readily visually recognize the boundary between each gradation in the gradation image.

Ninth Aspect An imaging device according to an aspect includes:

[0116] an excitation light source configured to irradiate an examinee with excitation light to excite a fluorescent dye administered to the examinee;

[0117] a capturing unit configured to obtain a fluorescent image by capturing fluorescence generated from the fluorescent dye;

[0118] an image storage unit configured to store the fluorescent image over time;

[0119] a pixel value measurement unit configured to measure, over time, a pixel value of each pixel in a predetermined region of the fluorescent image based on the fluorescent image stored in the image storage unit;

[0120] a change curve creation unit configured to create a temporal change curve of the pixel value of each pixel in the predetermined region based on the pixel values measured by the pixel value measurement unit;

[0121] an index calculation unit configured to analyze the change curve of each pixel in the predetermined region and calculate a predetermined index for each pixel in the predetermined region;

[0122] a gradation conversion unit configured to perform gradation conversion of the predetermined index to a predetermined number of gradations;

[0123] a gradation image generation unit configured to generate a gradation image based on the index subjected to the gradation conversion by the gradation conversion unit; and

[0124] an image display unit configured to display the gradation image.

[0125] In accordance with the imaging device according to the ninth aspect, the boundary between each gradation in the gradation image becomes clear. It is also possible to intuitively recognize the state of blood circulation by appropriately setting the color of each gradation. It is thus possible to accurately recognize, for example, the state of blood circulation of an examinee when the examinee undergoes surgery.

Tenth Aspect In the imaging device according to the ninth aspect,

[0126] the predetermined index is any one of gradient, a peak value, time to peak, an integral, or a curve shape of the change curve.

[0127] In accordance with the imaging device according to the tenth aspect, it is possible to stably recognize the state of blood flow.

Eleventh Aspect In the imaging device according to the ninth or tenth aspect,

[0128] the gradation image generation unit generates the gradation image with a different color for each gradation.

[0129] In accordance with the imaging device according to the eleventh aspect, the boundary between each gradation in the gradation image becomes clear.

Twelfth Aspect In the imaging device according to any one of the ninth through eleventh aspects,

[0130] the imaging device further includes a contour line generation unit configured to detect a boundary between each gradation in the gradation image and generate a contour line image representing each gradation, wherein

[0131] the image display unit is capable of switching between the gradation image and the contour line image for display.

[0132] In accordance with the imaging device according to the twelfth aspect, it is possible to switch the image in accordance with the state of blood circulation and thus to readily visually recognize the state of blood circulation.

Thirteenth Aspect In the imaging device according to the twelfth aspect,

[0133] the contour line generation unit generates the contour line image with a different thickness for each gradation.

[0134] In accordance with the imaging device according to the thirteenth aspect, it is possible to reflect the brightness of each gradation on the thickness of the contour line.

Fourteenth Aspect In the imaging device according to the twelfth or thirteenth aspect,

[0135] the contour line generation unit generates the contour line image with a different color for each gradation.

[0136] In accordance with the imaging device according to the fourteenth aspect, the boundary between each gradation in the gradation image becomes clear.

Fifteenth Aspect In the imaging device according to any one of the ninth through fourteenth aspects,

[0137] the predetermined region is an entire screen area of the fluorescent image.

[0138] In accordance with the imaging device according to the fifteenth aspect, it is possible to prevent, for example, oversight of the state of blood circulation depending on the case of the examinee.

Sixteenth Aspect In the imaging device according to any one of the ninth through fourteenth aspects,

[0139] the predetermined region is a plurality of regions in the fluorescent image.

[0140] In accordance with the imaging device according to the sixteenth aspect, it is possible to improve, for example, the image processing speed.

Seventeenth Aspect In the imaging device according to any one of the ninth through sixteenth aspects,

[0141] the capturing unit captures a visible image in real time in an area corresponding to the fluorescent image, and

[0142] the image display unit superimposes the gradation image on the visible image for display.

[0143] In accordance with the imaging device according to the seventeenth aspect, it is possible to readily visually recognize the boundary between each gradation in the gradation image.

Eighteenth Aspect An imaging method according to an aspect includes the steps of:

[0144] irradiating an examinee with excitation light to excite a fluorescent dye administered to the examinee;

[0145] obtaining a fluorescent image by capturing fluorescence generated from the fluorescent dye;

[0146] measuring, over time, a pixel value of each pixel in a predetermined region of the fluorescent image;

[0147] performing gradation conversion of the measured pixel values to a predetermined number of gradations;

[0148] generating a gradation image based on the pixel values subjected to the gradation conversion; and

[0149] displaying the gradation image.

[0150] In accordance with the imaging method according to the eighteenth aspect, the boundary between each gradation in the gradation image becomes clear. It is also possible to intuitively recognize the state of blood circulation by appropriately setting the color of each gradation. It is thus possible to accurately recognize, for example, the state of blood circulation of an examinee when the examinee undergoes surgery.

Nineteenth Aspect In the imaging method according to the eighteenth aspect,

[0151] the generating a gradation image generates the gradation image with a different color for each gradation.

[0152] In accordance with the imaging method according to the nineteenth aspect, the boundary between each gradation in the gradation image becomes clear.

Twentieth Aspect In the imaging method according to the eighteenth or nineteenth aspect,

[0153] the imaging method further includes detecting a boundary between each gradation in the gradation image and generating a contour line image representing each gradation, wherein

[0154] the displaying a gradation image displays at least any one of the gradation image or the contour line image.

[0155] In accordance with the imaging method according to the twentieth aspect, it is possible to readily visually recognize the state of blood circulation.

Twenty-First Aspect In the imaging method according to the twelfth aspect,

[0156] the generating a contour line image generates the contour line image with a different thickness for each gradation.

[0157] In accordance with the imaging method according to the twenty-first aspect, it is possible to reflect the brightness of each gradation on the thickness of the contour line.

Twenty-Second Aspect In the imaging method according to the twelfth ore twenty-first aspect,

[0158] the generating a contour line image generates the contour line image with a different color for each gradation.

[0159] In accordance with the imaging method according to the twenty-second aspect, the boundary between each gradation in the gradation image becomes clear.

Twenty-Third Aspect In the imaging method according to any one of the eighteenth through twenty-second aspects,

[0160] the predetermined region is an entire screen area of the fluorescent image.

[0161] In accordance with the imaging method according to the twenty-third aspect, it is possible to prevent, for example, oversight of the state of blood circulation depending on the case of the examinee.

Twenty-Fourth Aspect In the imaging method according to any one of the eighteenth through twenty-second aspects,

[0162] the predetermined region is a plurality of regions in the fluorescent image.

[0163] In accordance with the imaging method according to the twenty-fourth aspect, it is possible to improve, for example, the image processing speed.

Twenty-Fifth Aspect In the imaging method according to any one of the eighteenth through twenty-fourth aspects,

[0164] the imaging method further includes capturing a visible image in real time in an area corresponding to the fluorescent image, wherein

[0165] the displaying a gradation image superimposes the gradation image on the visible image for display.

[0166] In accordance with the imaging method according to the twenty-fifth aspect, it is possible to readily visually recognize the boundary between each gradation in the gradation image.

Twenty-Sixth Aspect An imaging method according to an aspect includes the steps of:

[0167] irradiating an examinee with excitation light to excite a fluorescent dye administered to the examinee;

[0168] obtaining a fluorescent image by capturing fluorescence generated from the fluorescent dye;

[0169] storing the fluorescent image over time;

[0170] measuring, over time, a pixel value of each pixel in a predetermined region of the fluorescent image based on the fluorescent image subjected to the storing;

[0171] creating a temporal change curve of the pixel value of each pixel in the predetermined region based on the measured pixel values;

[0172] analyzing the change curve of each pixel in the predetermined region and calculating a predetermined index for each pixel in the predetermined region;

[0173] performing gradation conversion of the predetermined index to a predetermined number of gradations;

[0174] generating a gradation image based on the index subjected to the gradation conversion; and

[0175] displaying the gradation image.

[0176] In accordance with the imaging method according to the twenty-sixth aspect, the boundary between each gradation in the gradation image becomes clear. It is also possible to intuitively recognize the state of blood circulation by appropriately setting the color of each gradation. It is thus possible to accurately recognize, for example, the state of blood circulation of an examinee when the examinee undergoes surgery.

Twenty-Seventh Aspect In the imaging method according to the twelfth aspect,

[0177] the predetermined index is any one of gradient, a peak value, time to peak, an integral, or a curve shape of the change curve.

[0178] In accordance with the imaging method according to the twenty-seventh aspect, it is possible to stably recognize the state of blood flow.

Twenty-Eighth Aspect In the imaging method according to the twenty-sixth or twenty-seventh aspect,

[0179] the generating a gradation image generates the gradation image with a different color for each gradation.

[0180] In accordance with the imaging method according to the twenty-eighth aspect, the boundary between each gradation in the gradation image becomes clear.

Twenty-Ninth Aspect In the imaging method according to any one of the twenty-sixth through twenty-eighth aspects,

[0181] the imaging method further includes detecting a boundary between each gradation in the gradation image and generating a contour line image representing each gradation, wherein

[0182] the displaying a gradation image is capable of switching between the gradation image and the contour line image for display.

[0183] In accordance with the imaging method according to the twenty-ninth aspect, it is possible to switch the image in accordance with the state of blood circulation and thus to readily visually recognize the state of blood circulation.

Thirtieth Aspect In the imaging method according to the twenty-ninth aspect,

[0184] the generating a contour line image generates the contour line image with a different thickness for each gradation.

[0185] In accordance with the imaging method according to the thirtieth aspect, it is possible to reflect the brightness of each gradation on the thickness of the contour line.

Thirty-First Aspect In the imaging method according to the twenty-ninth or thirtieth aspect,

[0186] the generating a contour line image generates the contour line image with a different color for each gradation.

[0187] In accordance with the imaging method according to the thirty-first aspect, the boundary between each gradation in the gradation image becomes clear.

Thirty-Second Aspect In the imaging method according to any one of the twenty-sixth through the thirty-first aspects,

[0188] the predetermined region is an entire screen area of the fluorescent image.

[0189] In accordance with the imaging method according to the thirty-second aspect, it is possible to prevent, for example, oversight of the state of blood circulation depending on the case of the examinee.

Thirty-Third Aspect In the imaging method according to any one of the twenty-sixth through thirty-second aspects,

[0190] the predetermined region is a plurality of regions in the fluorescent image.

[0191] In accordance with the imaging method according to the thirty-third aspect, it is possible to improve, for example, the image processing speed.

Thirty-Fourth Aspect In the imaging method according to any one of the twenty-sixth through thirty-fourth aspects,

[0192] the imaging method further comprising capturing in real time a visible image in an area corresponding to the fluorescent image, wherein

[0193] the displaying a gradation image superimposes the gradation image on the visible image for display.

[0194] In accordance with the imaging method according to the thirty-fourth aspect, it is possible to readily visually recognize the boundary between each gradation in the gradation image.