Radiographic image processing apparatus, radiographic image processing method, and radiographic image processing program
11272898 · 2022-03-15
Assignee
Inventors
Cpc classification
H05G1/44
ELECTRICITY
A61B6/462
HUMAN NECESSITIES
A61B6/5217
HUMAN NECESSITIES
A61B6/463
HUMAN NECESSITIES
A61B6/12
HUMAN NECESSITIES
International classification
A61B6/00
HUMAN NECESSITIES
Abstract
A detection unit detects a region of a surgical tool in a radiographic image of a patient. In a case in which the surgical tool is detected, a display control unit displays the radiographic image on a display unit such that the detected region of the surgical tool is highlighted.
Claims
1. A radiographic image processing apparatus comprising at least one processor, wherein the processor is configured to: detect a region of a surgical tool in a radiographic image of a patient; in a case in which the surgical tool is detected, display the radiographic image in which the detected region of the surgical tool has been highlighted on a display; perform image processing for checking the surgical tool on the radiographic image to derive a processed radiographic image, display, on the display, the radiographic image before the image processing for checking the surgical tool, derive the processed radiographic image and the start a process of detecting the region of the surgical tool in response to a command to detect the region of the surgical tool, in a case in which the processed radiographic image is derived, display the processed radiographic image on the display, instead of the radiographic image before the image processing or together with the radiographic image before the image processing, and after the process of detecting the region of the surgical tool ends, display the radiographic image in which the region of the surgical tool has been highlighted on the display.
2. The radiographic image processing apparatus according to claim 1, wherein the radiographic image in which the detected region of the surgical tool has been highlighted and which is displayed on the display is the processed radiographic image.
3. The radiographic image processing apparatus according to claim 1, wherein the processor is configured to display the radiographic image in which the detected region of the surgical tool has been highlighted on another display that has a larger size and/or a higher resolution than the display.
4. The radiographic image processing apparatus according to claim 1, wherein the processor is configured to display the radiographic image before the image processing, the processed radiographic image, and the radiographic image in which the detected region of the surgical tool has been highlighted on another display that has a larger size and/or a higher resolution than the display.
5. The radiographic image processing apparatus according to claim 1, wherein the processor is configured to, in a case in which the surgical tool is not detected, notify the fact.
6. The radiographic image processing apparatus according to claim 1, wherein the radiographic image is acquired by a portable radiation detector or an imaging apparatus that is installed in an operating room for performing surgery on the patient.
7. The radiographic image processing apparatus according to claim 1, wherein the processor is configured to discriminate the region of the surgical tool in an input radiographic image by a discriminator trained so as to discriminate the region of the surgical tool in the input radiographic image.
8. The radiographic image processing apparatus according to claim 1, wherein the surgical tool includes at least one of gauze, a scalpel, scissors, a drain, a needle, a thread, or forceps.
9. The radiographic image processing apparatus according to claim 8, wherein at least a portion of the gauze includes a radiation absorbing thread.
10. A radiographic image processing method comprising: detecting a region of a surgical tool in a radiographic image of a patient; in a case in which the surgical tool is detected, displaying the radiographic image on a display such that the detected region of the surgical tool is highlighted; performing image processing for checking the surgical tool on the radiographic image to derive a processed radiographic image, displaying, on the display, the radiographic image before the image processing for checking the surgical tool, deriving the processed radiographic image and the start a process of detecting the region of the surgical tool in response to a command to detect the region of the surgical tool, in a case in which the processed radiographic image is derived, displaying the processed radiographic image on the display, instead of the radiographic image before the image processing or together with the radiographic image before the image processing, and after the process of detecting the region of the surgical tool ends, displaying the radiographic image in which the region of the surgical tool has been highlighted on the display.
11. A non-transitory computer-readable storage medium that stores a radiographic image processing program that causes a computer to perform: detecting a region of a surgical tool in a radiographic image of a patient; in a case in which the surgical tool is detected, displaying the radiographic image on a display such that the detected region of the surgical tool is highlighted; performing image processing for checking the surgical tool on the radiographic image to derive a processed radiographic image, displaying, on the display, the radiographic image before the image processing for checking the surgical tool, deriving the processed radiographic image and the start a process of detecting the region of the surgical tool in response to a command to detect the region of the surgical tool, in a case in which the processed radiographic image is derived, displaying the processed radiographic image on the display, instead of the radiographic image before the image processing or together with the radiographic image before the image processing, and after the process of detecting the region of the surgical tool ends, displaying the radiographic image in which the region of the surgical tool has been highlighted on the display.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
(1)
(2)
(3)
(4)
(5)
(6)
(7)
(8)
(9)
(10)
(11)
(12)
(13)
(14)
(15)
(16)
(17)
(18)
(19)
(20)
(21)
(22)
(23)
(24)
(25)
DETAILED DESCRIPTION
(26) Hereinafter, embodiments of the present disclosure will be described with reference to the drawings.
(27) The imaging apparatus 1 irradiates a radiation detector 5 with radiation which has been emitted from a radiation source 4, such as an X-ray source, and transmitted through a subject H to acquire a radiographic image G0 of the subject H that lies supine on an operating table 3. The radiographic image G0 is input to the console 2 which is the radiographic image processing apparatus.
(28) The radiation detector 5 can repeatedly perform the recording and reading of a radiographic image and may be a so-called direct-type radiation detector that directly receives the emitted radiation and generates charge or a so-called indirect-type radiation detector that converts radiation into visible light and then converts the visible light into a charge signal. As a method for reading a radiographic image signal, it is desirable to use a so-called thin film transistor (TFT) reading method which turns on and off a TFT switch to read a radiographic image signal or a so-called optical reading method which emits reading light to read a radiographic image signal. However, the present disclosure is not limited thereto and other methods may be used.
(29) The radiation detector 5 is a portable radiation detector and is attached to the operating table 3 by an attachment portion 3A that is provided in the operating table 3. The radiation detector 5 may be fixed to the operating table 3.
(30) A display unit 6 and an input unit 7 are connected to the console 2. The display unit 6 consists of display, for example, a cathode ray tube (CRT) or a liquid crystal display and assists the input of a radiographic image acquired by imaging and various kinds of data necessary for processes performed by the computer 2. The input unit 7 consists of, for example, a keyboard, a mouse, or a touch panel.
(31) A radiographic image processing program according to this embodiment is installed in the console 2. In this embodiment, the console 2 may be a workstation or a personal computer that is directly operated by an operator or a server computer that is connected to the console through a network. The radiographic image processing program is stored in a storage device of the server computer connected to the network or a network storage so as to be accessed from the outside and is downloaded and installed in the computer on demand. Alternatively, the radiographic image processing program is recorded on a recording medium such as a digital versatile disc (DVD) or a compact disc read only memory (CD-ROM), is distributed, and is installed in the computer from the recording medium.
(32) The radiography system according to this embodiment forms a portion of a radiographic image management system illustrated in
(33)
(34) The storage 13 consists of a storage device, such as a hard disk or a solid state drive (SSD), and stores various kinds of information including an imaging program for driving each unit of the imaging apparatus 1 and the radiographic image processing program. In addition, the radiographic image acquired by imaging is stored in the storage 13.
(35) The memory 12 temporarily stores, for example, the radiographic image processing program stored in the storage 13 in order to cause the CPU 11 to perform various processes. The radiographic image processing program defines the following processes as processes to be performed by the CPU 11: an image acquisition process that irradiates the radiation detector 5 with radiation, which has been emitted from the radiation source 4 and then transmitted through the subject H, to acquire the radiographic image G0; a detection process that detects a region of a surgical tool in the radiographic image; and a display control process that displays the radiographic image on the display unit 6 such that the detected region of the surgical tool is highlighted in a case in which the surgical tool is detected.
(36) The CPU 11 performs the above-mentioned processes according to the radiographic image processing program such that the console 2 functions as an image acquisition unit 21, a detection unit 22, and a display control unit 23.
(37) The image acquisition unit 21 drives the radiation source 4 to irradiate the subject H that has undergone surgery with radiation and detects the radiation transmitted through the subject H using the radiation detector 5 to acquire the radiographic image G0. In this case, the image acquisition unit 21 sets imaging conditions, such as the type of target and filter used in the radiation source 4, an imaging dose, a tube voltage, and a SID.
(38) The detection unit 22 detects the region of the surgical tool in the radiographic image G0. For the detection, a discriminator 30 that discriminates the region of the surgical tool included in the radiographic image G0 in a case in which the radiographic image G0 is input is applied to the detection unit 22. In a case in which the target radiographic image G0 is input to the detection unit 22, the detection unit 22 directs the discriminator 30 to discriminate the region of the surgical tool included in the radiographic image G0, thereby detecting the region of the surgical tool.
(39) Here, the discriminator 30 is constructed by training a machine learning model using a radiographic image including a surgical tool as training data. In this embodiment, it is assumed that gauze is used as the surgical tool.
(40)
(41) In a case in which the detection unit 22 detects the region of the surgical tool from the radiographic image G0, the display control unit 23 displays the radiographic image G0 on the display unit 6 such that the region of the surgical tool is highlighted.
(42) In addition, in a case in which the radiographic image GH0 in which the region of the surgical tool has been highlighted is displayed on the display unit 6, image processing for display, such as a gradation conversion process or a density conversion process, may be performed on the radiographic image G0 in order for the operator to easily observe the displayed radiographic image GH0. The display control unit 23 may perform the image processing for display or an image processing unit for performing the image processing for display may be separately provided. In a case in which the image processing for display is performed on the radiographic image G0, the detection unit 22 may detect the region of the surgical tool from the radiographic image G0 subjected to the image processing for display.
(43) In addition, in a case in which the detection unit 22 does not detect the region of the surgical tool from the radiographic image G0, the display control unit 23 notifies the fact.
(44) Next, a surgical tool region detection process according to the first embodiment will be described.
(45) Then, the display control unit 23 determines whether or not the region of the surgical tool has been detected from the radiographic image G0 (Step ST3). In a case in which the determination result in Step ST3 is “Yes”, the display control unit 23 displays the radiographic image G0 in which the region of the surgical tool has been highlighted on the display unit 6 (the display of the radiographic image in which the surgical tool has been highlighted; Step ST4). Then, the process ends. On the other hand, in a case in which the determination result in Step ST3 is “No”, the display control unit 23 notifies that the region of the surgical tool has not been detected (notification that the surgical tool has not been detected; Step ST5). Then, the process ends.
(46) As described above, in the first embodiment, in a case in which the region of the surgical tool is detected in the radiographic image G0 of the patient, the radiographic image G0 is displayed such that the detected region of the surgical tool is highlighted. Here, the radiographic image G0 can be acquired in a state in which the patient is in the operating room. Further, since the radiographic image G0 is a fluoroscopic image, it is easy to detect the surgical tool buried in the surgical field, as compared to an image acquired by a camera. Therefore, according to this embodiment, it is possible to reliably prevent a surgical tool from remaining in the body of the patient after surgery, without imposing a burden on the patient.
(47) Further, in the first embodiment, in a case in which the region of the surgical tool is not detected in the radiographic image G0, the fact is notified. Therefore, the operator can recognize that no surgical tools remain in the body of the patient.
(48) Next, a second embodiment of the present disclosure will be described.
(49) The image processing unit 24 performs the image processing for checking a surgical tool on the radiographic image G0. Examples of the image processing for checking a surgical tool include a process that emphasizes the sharpness of the radiographic image G0 and a gradation conversion process that converts the gradation of the radiographic image G0 in order to eliminate a blocked-up shadow in which a high-density component is saturated and a blown-out highlight in which a low-density component is saturated in the radiographic image G0.
(50) In addition, in the second embodiment, the detection unit 22 may detect the region of the surgical tool from the radiographic image G0 or may detect the region of the surgical tool from the processed radiographic image G1. Here, it is assumed that the region of the surgical tool is detected from the radiographic image G0.
(51) In addition, in the second embodiment, the radiographic image G0 and the processed radiographic image G1 are displayed as follows.
(52) Next, in a case in which the detection button 71 is selected and a command to detect the surgical tool is input from the input unit 7 (Step ST13), the image processing unit 24 performs image processing for checking a surgical tool on the radiographic image G0 (Step ST14). Further, the detection unit 22 detects the region of the surgical tool from the radiographic image G0 in response to the detection command (Step ST15). After the image processing for checking a surgical tool, the display control unit 23 displays the processed radiographic image G1 on the display unit 6 (Step ST16). The process in Step ST15 is performed in parallel to the processes in Steps ST14 and ST16. However, the process in Step ST15 may be performed after Step ST14 or Step ST16. In a case in which the region of the surgical tool is detected from the processed radiographic image G1, the process in Step ST15 may be performed in parallel to the process in Step ST16 after Step ST14 or may be performed after Step ST16.
(53)
(54) In a case in which the detection process ends, the display control unit 23 determines whether or not the region of the surgical tool has been detected from the radiographic image G0 (Step ST17). In a case in which the determination result in Step ST17 is “Yes”, the display control unit 23 displays the processed radiographic image G1 in which the region of the surgical tool has been highlighted on the display unit 6 (Step ST18). Then, the process ends. On the other hand, in a case in which the determination result in Step ST17 is “No”, the display control unit 23 notifies that the region of the surgical tool has not been detected (Step ST19). Then, the process ends.
(55)
(56) As described above, in the second embodiment, the image processing for checking a surgical tool is performed on the radiographic image G0 and the processed radiographic image G1 is displayed. Therefore, the region of the surgical tool can be easily checked by the processed radiographic image G1. Therefore, according to this embodiment, it is possible to reliably prevent a surgical tool from remaining in the body of the patient after surgery in combination with highlighting the region of the surgical tool.
(57) In the second embodiment, the image processing for checking a surgical tool is performed on the radiographic image G0 in response to the surgical tool detection command. However, the present disclosure is not limited to thereto. The image processing for checking a surgical tool may be performed on the radiographic image G0 without waiting for the surgical tool detection command or in response to a command for only image processing.
(58) In the second embodiment, in a case in which the detection command is input, the processed radiographic image G1 and the processed radiographic image GH1 in which the region of the surgical tool has been highlighted are displayed on the display unit 6, instead of the radiographic image G0. However, the processed radiographic image G1 and the processed radiographic image GH1 in which the region of the surgical tool has been highlighted may be displayed on the display unit 6 together with the radiographic image G0.
(59) Next, a third embodiment of the present disclosure will be described.
(60) The display unit 8 consists of, for example, a liquid crystal monitor and has a larger screen size than the display unit 6. For example, while the screen size of the display unit 6 is 14 to 15 inches, the screen size of the display unit 8 is 40 to 50 inches. The display unit 8 is attached to, for example, the wall of the operating room. The display unit 8 may have the same size as the display unit 6 or may have a higher resolution than the display unit 6. For example, in a case in which the resolution of the display unit 6 is 1 megapixel, the resolution of the display unit 8 may be 2 megapixels or 3 megapixels or more. The display unit 8 may have a larger size and higher resolution than the display unit 6.
(61) Next, a process performed in the third embodiment will be described.
(62) Next, in a case in which the detection button 71 is selected and a surgical tool detection command is input from the input unit 7 (Step ST23), the detection unit 22 detects the region of the surgical tool from the radiographic image G0 (Step ST24). In this case, a progressive bar indicating the progress of the surgical tool detection process may be displayed on the display screen 70.
(63) In a case in which the detection process ends, the display control unit 23 determines whether the region of the surgical tool has been detected from the radiographic image G0 (Step ST25). In a case in which the determination result in Step ST25 is “Yes”, the display control unit 23 displays the radiographic image G0 in which the region of the surgical tool has been highlighted on the two display units 6 and 8 (Step ST26). Then, the process ends. On the other hand, in a case in which the determination result in Step ST25 is “No”, the display control unit 23 notifies that the region of the surgical tool has not been detected (Step ST27). Then, the process ends. The notification may be displayed only on the display unit 6, may be displayed only on the display unit 8, or may be displayed on both the display unit 6 and the display unit 8.
(64) As described above, in the third embodiment, the radiographic image G0 is displayed on the display unit 8 having a larger screen and/or a higher resolution than the display unit 6. Therefore, the region of the surgical tool can be easily checked by the radiographic image displayed on the display unit 8. Therefore, according to this embodiment, it is possible to reliably prevent a surgical tool from remaining in the body of the patient after surgery in combination with highlighting the region of the surgical tool.
(65) Next, a fourth embodiment of the present disclosure will be described.
(66) Next, a process performed in the fourth embodiment will be described.
(67) Next, in a case in which the detection button 71 is selected and a surgical tool detection command is input from the input unit 7 (Step ST33), the image processing unit 24 performs image processing for checking a surgical tool on the radiographic image G0 (Step ST34). The detection unit 22 detects the region of the surgical tool from the radiographic image G0 in response to the detection command (Step ST35). Further, after the image processing for checking a surgical tool, the display control unit 23 displays the processed radiographic image G1 on the two display units 6 and 8 (Step ST36). Then, the display screen 75 illustrated in
(68) In a case in which the detection process ends, the display control unit 23 determines whether the region of the surgical tool has been detected from the radiographic image G0 (Step ST37). In a case in which the determination result in Step ST37 is “Yes”, the display control unit 23 displays the processed radiographic image G1 in which the region of the surgical tool has been highlighted on the two display units 6 and 8 (Step ST38). Then, the process ends. On the other hand, in a case in which the determination result in Step ST37 is “No”, the display control unit 23 notifies that the region of the surgical tool has not been detected (Step ST39). Then, the process ends. The notification may be displayed only on the display unit 6, may be displayed only on the display unit 8, or may be displayed on both the display unit 6 and the display unit 8.
(69) As described above, in the fourth embodiment, the image processing for checking a surgical tool is performed on the radiographic image G0 to derive the processed radiographic image G1 and the processed radiographic image G1 is also displayed on the display unit 8 having a large screen and/or a high resolution. Therefore, the region of the surgical tool can be easily checked by the processed radiographic image GH1 in which the region of the surgical tool has been highlighted and which is displayed on the display unit 8. Therefore, according to this embodiment, it is possible to reliably prevent a surgical tool from remaining in the body of the patient after surgery in combination with highlighting the region of the surgical tool.
(70) In the fourth embodiment, the image processing for checking a surgical tool is performed on the radiographic image G0 after the surgical tool detection command is input. However, the present disclosure is not limited thereto. The image processing for checking a surgical tool may be performed on the radiographic image G0 without waiting for the surgical tool detection command or in response to a command for only image processing.
(71) Further, in the fourth embodiment, in a case in which the detection command is input, the processed radiographic image G1 is displayed on the display units 6 and 8, instead of the radiographic image G0. However, the radiographic image G0 and the processed radiographic image G1 may be displayed together on the display units 6 and 8.
(72) In each of the above-described embodiments, the turn-on and turn-off of the highlight display of the radiographic image G0 and/or the processed radiographic image G1 may be switched in response to a command from the input unit 7.
(73) Further, in the above-described embodiments, the radiographic image G0 and/or the processed radiographic image G1 in which the region of the surgical tool has been detected may be transmitted to an external apparatus such as the PACS 101. In this case, information indicating that the region of the surgical tool has been detected may be given to the radiographic image G0 and/or the processed radiographic image G1. For example, in a protocol such as digital imaging and communication in medicine (DICOM) that defines the storage format of image data and communication between apparatuses, various kinds of information may be given as accessory information to the radiographic image G0 and/or the processed radiographic image G1. Therefore, the DICOM protocol may be used to give information indicating that the region of the surgical tool has been detected to the accessory information of the radiographic image G0 and/or the processed radiographic image G1. Further, a process of adding a frame or the like for highlighting the region of the surgical tool may be performed on the radiographic image G0 and/or the processed radiographic image G1 to generate the radiographic image GH0 and/or the processed radiographic image GH1 in which the region of the surgical tool has been highlighted and the generated radiographic image GH0 and/or processed radiographic image GH1 may be transmitted to an external apparatus.
(74) Further, in each of the above-described embodiments, the console 2 performs, for example, the surgical tool detection process and the image processing. However, the present disclosure is not limited thereto. For example, the radiographic image processing program according to each of the embodiments of the present disclosure may be installed in the PACS 101 illustrated in
(75) Further, in the second and fourth embodiments, the degree of image processing for the displayed processed radiographic image G1 may be changed. For example, the degree of emphasis of the sharpness and the degree of change of the gradation of the processed radiographic image G1 may be changed. In this case, it is possible to display the processed radiographic image G1 with the desired quality on the display unit 6 and/or the display unit 8.
(76) Further, in each of the above-described embodiments, the detection unit 22 detects the region of the surgical tool using the discriminator 30. However, the present disclosure is not limited thereto. A histogram of the radiographic image G0 may be calculated and it may be determined whether or not a signal value of the surgical tool is included in the histogram to detect the region of the surgical tool from the radiographic image G0 or the processed radiographic image G1.
(77) In each of the above-described embodiments, the radiographic image G0 is a still image. However, the present disclosure is not limited thereto. The radiographic image G0 may be a moving image. In this case, the process of detecting the region of the surgical tool is performed on each frame of the radiographic image G0 which is a moving image.
(78) In each of the above-described embodiments, the radiographic image G0 of the subject H is acquired by the radiation source 4 and the radiation detector 5. However, the present disclosure is not limited thereto. For example, in some cases, the operator performs surgery while observing the radiographic image G0 of the patient using a C-arm fluoroscopic apparatus. In this case, the radiographic image G0 may be acquired by the C-arm fluoroscopic apparatus.
(79) In the above-described embodiments, the gauze 40 as a surgical tool is a detection target. However, the present disclosure is not limited thereto. Any surgical tools, such as a scalpel, scissors, a drain, a needle, a thread, and forceps, which are used during surgery and should not remain in the body may be used as the detection targets. In this case, the discriminator 30 may be trained so as to discriminate the target surgical tool. The discriminator 30 is trained so as to detect a plurality of channels, which makes it possible to construct the discriminator 30 so as to discriminate not only one kind of surgical tool but also a plurality of kinds of surgical tools.
(80) In addition, in the above-described embodiments, the radiation is not particularly limited. For example, α-rays and γ-rays other than X-rays can be applied.
(81) In the above-described embodiments, for example, the following various processors can be used as a hardware structure of processing units performing various processes, such as the image acquisition unit 21, the detection unit 22, the display control unit 23, and the image processing unit 24. The various processors include, for example, a CPU which is a general-purpose processor executing software (program) to function as various processing units, a programmable logic device (PLD), such as a field programmable gate array (FPGA), which is a processor whose circuit configuration can be changed after manufacture, and a dedicated electric circuit, such as an application-specific integrated circuit (ASIC), which is a processor having a dedicated circuit configuration designed to perform a specific process.
(82) One processing unit may be configured by one of the various processors or a combination of two or more processors of the same type or different types (for example, a combination of a plurality of FPGAs or a combination of a CPU and an FPGA). Further, a plurality of processing units may be configured by one processor.
(83) A first example of the configuration in which a plurality of processing units are configured by one processor is an aspect in which one processor is configured by a combination of one or more CPUs and software and functions as a plurality of processing units. A representative example of this aspect is a client computer or a server computer. A second example of the configuration is an aspect in which a processor that implements the functions of the entire system including a plurality of processing units using one integrated circuit (IC) chip is used. A representative example of this aspect is a system-on-chip (SoC). As such, various processing units are configured by using one or more of the various processors as the hardware structure.
(84) Furthermore, specifically, an electric circuit (circuitry) obtained by combining circuit elements, such as semiconductor elements, can be used as the hardware structure of the various processors.