BONE FRACTURE DETECTION AND CLASSIFICATION

Abstract

The outline of a bone or at least a portion thereof may be determined based on deep neural net and optionally on an active shape model approach. An algorithm may detect a fracture of the bone. An algorithm may also classify the bone fracture and provide guidance on how to treat the fracture.

Claims

1. A device for assisting with bone fracture detection, the device comprising a processing unit configured to: receive image data of a medical image; identify a bone structure in the medical image; and determine a fracture line in the identified bone structure.

2. The device of claim 1, wherein the processing unit is further configured to detect at least a portion of an outline of the identified bone, to detect a point of the fracture line on the outline of the bone structure or both.

3. The device of claim 2, wherein the outline of the identified bone is detected based on edge detection.

4. The device of claim 1, wherein the processing unit is further configured to detect a displacement of bone parts relative to each other.

5. The device of claim 1, wherein the processing unit is further configured to classify the bone fracture and to generate a corresponding output.

6. The device of claim 1, further comprising implementing a neural net (DNN) in the processing unit for a corresponding output selected from a group including bone structure identification, fracture line determination, bone outline detection, fracture line point detection, bone part displacement detection, bone fracture classification, or a combination thereof.

7. The device of claim 1, wherein the processing unit is further configured to receive information from a data base that is selected from a group including a bone model, possible fracture lines and their likelihood, possible bone fragmentations and their likelihood, a classification of bone fractures or a combination thereof.

8. A method of assisting with bone fracture classification, the method comprising the steps of receiving image data of a medical image; identifying a bone structure in the medical image; and determining a fracture line at the identified bone structure.

9. The method of claim 8, further comprising detecting a point of the fracture line at an outline of the bone structure.

10. The method of claim 8, further comprising of detecting a displacement of bone parts relative to each other.

11. The method of claim 10, further comprising of classifying the bone fracture.

12. The method of claim 8, further comprising the steps of receiving a priori information and determining an expected location and path of a fracture line.

13. The method of claim 8, further comprising the steps of providing an output, the output providing information to a user related to potential fracture lines and/or fracture classifications, and receiving an input selecting at least one of the potential fracture lines and/or fracture classification.

14. (canceled)

15. The method of claim 9, further comprising classifying the bone fracture.

16. A computer program product comprising a processing unit operably connected to a memory having a sets of instructions which when executed on the processor unit carries out a method of receiving image data of a medical image, identifying a bone structure in the medical image, and determining a fracture line in the identified bone structure.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0053] The present invention will hereinafter be described in conjunction with the following drawing figures, wherein like numerals denote like elements, and:

[0054] FIG. 1 shows a flow chart of procedural steps in accordance with an embodiment.

[0055] FIG. 2 shows aspects of a device in accordance with an embodiment.

[0056] FIG. 3 shows an example of medical images suitable for training a neural net to recognize an aspect of a bone.

[0057] FIGS. 4 to 6 show examples of medical images including bone fractures.

[0058] Throughout the drawings, the same reference numerals and characters, unless otherwise stated, are used to denote like features, elements, components, or portions of the illustrated embodiments. Moreover, while the present disclosure will now be described in detail with reference to the figures, it is done so in connection with the illustrative embodiments and is not limited by the particular embodiments illustrated in the figures.

DETAILED DESCRIPTION OF THE EMBODIMENTS

[0059] The following detailed description is merely exemplary in nature and is not intended to limit the invention or the application and uses of the invention. Furthermore, there is no intention to be bound by any theory presented in the preceding background of the invention or the following detailed description.

[0060] The flow-chart in FIG. 1 illustrates the principle of the steps performed in accordance with an embodiment of the disclosed invention. It will be understood that the steps described are major steps, wherein these major steps might be differentiated or divided into several sub-steps. Furthermore, there might be also sub-steps between these major steps.

[0061] As a first aspect, “training” of a so-called “deep neural net” (DNN) is described with reference to steps S11, S12 and S13 in FIG. 1.

[0062] When training a DNN, for example, to be able to identify an anatomical structure like a bone in a medical image, an image with known ground truth is provided to the DNN. Indicated with the arrow ‘a’ in FIG. 1 is the possibility to provide a real image (generated by imaging a patient). The step S11 may be seen as a step of generating a simulated medical image. A simulated image may thus be provided at arrow ‘b’ to the DNN (alternatively or additionally to a provision of a real image along arrow ‘a’), which assesses the image in step S12. For example, the algorithm may assign to each pixel of the image a likelihood that a pixel is part of an anatomical feature or aspect. In step S13, the DNN (here DNN 1) may be able to provide information on whether a plurality of pixels constitute an anatomic structure. Based on processing a multiplicity of images and comparing the results with the known ground truth, the parameters of the DNN are adjusted, and the DNN “learns” to identify for example a femur so as to be able to do so even if an actual image shows a femur in a way that differs from all images used in the training phase. As already described above, the DNN may include aspects of active shape models, point distribution models, or the like.

[0063] As an example, FIG. 3 (panel A) shows an X-ray visualization of a hip joint with a part of hip bone and a proximal part of a thigh bone (a femur). When training a DNN to localize the contour of a femur, the ground truth (shown in panel B) is provided to the DNN along with the medical image. By using many such images in its training, a DNN is then able to generalize its localization ability to previously unseen images.

[0064] In step S21 in FIG. 1, a first medical image may be generated by an X-ray imaging device. The first medical image is then received in step S22 by a processing unit.

[0065] As used herein, the term “receiving an image” basically refers to the fact that at least one image is necessary to perform the subsequent steps. That is, the term “receiving an image” may encompass both receiving of an image directly when generated by an imaging device and loading of an image from a data base into a processing unit. It is just required that a received image is suitable for identifying an aspect or feature of a bone. A person skilled in the art will consequently understand that the image processing as described herein may be performed independently from an imaging device.

[0066] In step S23, the received image is processed utilizing DNN 1, wherein at least one anatomical feature is identified and a location thereof is determined. This may be done as described above.

[0067] In step S24, the identified bone is further processed so as to determine at least one fracture line. In the embodiment illustrated in FIG. 1, step S24 is performed utilizing a second deep neural net DNN 2 for determination of the fracture lines. The second deep neural net DNN 2 may be trained on the basis of images like those of FIGS. 4 to 6 with labelled ground truth.

[0068] FIG. 4 shows an X-ray image of a hip joint, generated more or less with an anterior to posterior view (AP). Indicated by arrow C is a fracture, which is recognizable as a darker area. FIG. 5 shows the same fracture from a different viewing angle (medial-lateral, ML). In this figure, the fracture is more easily recognizable as the dark area (indicated by arrow D), which extends from the outline of the bone inward. Because deep morphing classifies points on the object contour, it may be possible to first identify the fracture in FIG. 5, and then search for a fracture at the corresponding part of the bone in FIG. 4. This allows identifying a difficult-to-see fracture in FIG. 4 by using ‘a priori’ information gathered by processing another image.

[0069] FIG. 6 is a diagnostic X-ray image of a fracture where the proximal end of the femur is pushed into the shaft in axial direction and tilted in medial direction, i.e. towards the hip. The tilting may be recognized by the fact that the tip of the greater trochanter is now above the center x of the femur head when measured from a line perpendicular to the longitudinal axis of the femur shaft. Furthermore, a gap opens at arrow G in FIG. 6, visible as darker area, which is also an indication of the tilting. The compressed state of the bone can be recognized by the brighter area above arrow E. In a healthy bone, this brighter area would be positioned above arrow F, which is now darker than normal.

[0070] The results of the fracture detection in step S24 may be improved upon by information provided to the device as input In 1. In an exemplary scenario, it may be known how the accident of the patient took place, for example a sideward fall onto the hip joint. Based on that information, the device may determine fractures that may likely be expected in such a scenario, before assessing the medical image. Accordingly, the information may be denoted as a priori information. Thus, the device may focus on specific areas of the imaged bone when assessing the medical image in step S24.

[0071] Lines and areas as detected in step S24 may be identified or at least suggested as fractures in step S26. Assuming that the device shows only one line or area as result on a display device like a monitor, a user may check the result and may confirm or correct it via input In 2. In a case in which the device detects more than one line or area as potential fractures and shows these as result of step S24 on a monitor, a user may provide a selection by way of an input In 2. Finally, the device may show identified fractures as result of step S26.

[0072] Another scenario for the input In 1 can be explained with reference to FIGS. 4 and 5. These figures show the same fracture albeit from different viewing directions. When starting for example with the medical image of FIG. 5, the device may have identified the fracture indicated by arrow D in step S26. The information that the fracture is basicervical may then be provided (along the dashed arrow in FIG. 1) as input In 1 to the device when assessing the next image, for example the image of FIG. 4. Taking the input into account, the DNN 2 is more likely to detect the fracture as indicated by arrow C in FIG. 4. A skilled person will appreciate that a second medical image may improve the recognition of a fracture, as for example illustrated by the above scenario.

[0073] An algorithm trained on the basis of a multiplicity of images showing bones with the same type of fracture, may be able to determine a fracture with high accuracy.

[0074] The determined bone fracture may be classified in step S27. The device may have access to information of fracture classification as typically used in a clinical environment. In such a classification, typical fractures are sorted by bone, by fracture location, by length of fracture line, by extension direction of the fracture, by number of fractures, and so on. By comparing the determined or identified fracture with the fractures in the classification, the device may provide a classification of the determined fracture.

[0075] Additionally or alternatively, the device may provide suggestions for treatment in step S28. For example, the device may suggest a specific implant for fracture fixation. Otherwise, the device may also request further imaging and/or further measurement of the bone with the fracture.

[0076] It will be understood that the flow chart is only one embodiment, and that for example steps S21, S27 and S28 as well as In 1 and In 2 may not be executed so as to achieve a result as intended in accordance with the invention. As already mentioned, the aspects achieved by the deep neural nets DNN 1 and DNN 2 may also be achieved by only one DNN or by more than the mentioned two.

[0077] FIG. 2 shows an exemplary embodiment of a device. Substantially a processing unit 100 is part of the device, necessary for performing the above described process. The device may further comprise an input device 300, for example a computer mouse, a trackball, a keyboard, a touchpad or the like, a monitor 400, which may also be a touch screen, and a data base 600, which may also be a remote data base like an internet cloud.

[0078] An exemplary imaging device 200 includes an X-ray source 240 and an X-ray detector 260, wherein these two devices may be mounted on a C-arm 220. It will be understood that the device may also comprise a non-invasive imaging modality like a computer tomography device, a magnetic resonance device, or an ultrasound device as imaging device instead of or additional to the shown C-arm based X-ray device.

[0079] Finally, a region of interest 500 is shown. Within said region, for example a bone of a patient may be located which is to be treated.

[0080] While embodiments have been illustrated and described in detail in the drawings and afore-going description, such illustrations and descriptions are to be considered illustrative or exemplary and not restrictive, and the invention is not limited to the disclosed embodiments.

[0081] Other variations to the disclosed embodiments can be understood and effected by those skilled in the art in practising the claimed invention, from a study of the drawings, the disclosure and the appended claims. In the claims, the word “comprising” does not exclude other elements or steps, and the indefinite article “a” or “an” does not exclude a plurality. A single processor or other unit may fulfil the functions of several items recited in the claims.

[0082] The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures can not be used to advantage. The computer program may be stored/distributed on a suitable medium such as an optical storage medium or a solid-state medium supplied together with or as a part of another hardware, but may also be distributed in other forms, such as via the Internet or other wired or wireless telecommunication systems. Any reference signs in the claims should not be construed as limiting the scope.

[0083] While at least one exemplary embodiment has been presented in the foregoing detailed description, it should be appreciated that a vast number of variations exist. It should also be appreciated that the exemplary embodiment or exemplary embodiments are only examples, and are not intended to limit the scope, applicability, or configuration of the invention in any way. Rather, the foregoing detailed description will provide those skilled in the art with a convenient road map for implementing an exemplary embodiment, it being understood that various changes may be made in the function and arrangement of elements described in an exemplary embodiment without departing from the scope of the invention as set forth in the appended claims and their legal equivalents.