Orthopedic Surgical Planning System with Automated Bone Density Measurement

Abstract

Techniques are described herein for automated bone mineral density and fracture risk assessment from digital X-rays. A method includes: obtaining image data including information relating to cortical bone tissue of at least a part of a first bone; preprocessing the image data, by isolating a region of interest in the image data, to generate preprocessed image data; applying the preprocessed image data as inputs across a trained machine learning model to generate output indicative of bone mineral density of the first bone; determining a T-score value based on the bone mineral density of the first bone; and providing, on a user interface, an output based on the T-score value.

Claims

1. A method implemented by one or more processors, the method comprising: obtaining image data comprising information relating to cortical bone tissue of at least a part of a first bone; preprocessing the image data, by isolating a region of interest in the image data, to generate preprocessed image data; applying the preprocessed image data as inputs across a trained machine learning model to generate output indicative of bone mineral density of the first bone; determining a T-score value based on the bone mineral density of the first bone; and providing, on a user interface, an output based on the T-score value.

2. The method according to claim 1, further comprising: determining a Z-score value based on the bone mineral density of the first bone; and providing, on the user interface, an output based on the Z-score value.

3. The method according to claim 1, further comprising: determining a fracture risk based on the bone mineral density of the first bone; and providing, on the user interface, an output based on the fracture risk assessment.

4. The method according to claim 1, further comprising: determining a recommended surgical implant type based on the bone mineral density of the first bone or the T-score value; and providing, on the user interface, an output based on the recommended surgical implant type.

5. The method according to claim 4, wherein determining the recommended surgical implant type based on the bone mineral density of the first bone or the T-score value comprises: in response to the bone mineral density or the T-score satisfying a first threshold and satisfying a second threshold, determining that the recommended surgical implant type is a first surgical implant type; in response to the bone mineral density or the T-score satisfying the first threshold but not satisfying the second threshold, determining that the recommended surgical implant type is a second surgical implant type; and in response to the bone mineral density or the T-score not satisfying the first threshold and not satisfying the second threshold, determining that the recommended surgical implant type is a third surgical implant type.

6. The method according to claim 1, further comprising: determining a recommended treatment based on the bone mineral density of the first bone or the T-score value; and providing, on the user interface, an output based on the recommended treatment.

7. The method according to claim 6, wherein determining the recommended treatment based on the bone mineral density of the first bone or the T-score value comprises: in response to the bone mineral density or the T-score satisfying a first threshold and satisfying a second threshold, determining that the recommended treatment is a first treatment; in response to the bone mineral density or the T-score satisfying the first threshold but not satisfying the second threshold, determining that the recommended treatment is a second treatment; and in response to the bone mineral density or the T-score not satisfying the first threshold and not satisfying the second threshold, determining that the recommended treatment is a third treatment.

8. The method according to claim 1, wherein the trained machine learning model is a convolutional neural network or vision transformer.

9. The method according to claim 1, wherein: the first bone is a femur; and isolating the region of interest in the image data comprises applying auto-segmentation of various femoral geometries to the image data.

10. The method according to claim 9, wherein applying the preprocessed image data as the inputs across the trained machine learning model further generates output indicative of quantitative indices of geometric landmarks in the femur.

11. The method according to claim 10, further comprising: determining a Dorr type of the femur based on the quantitative indices of geometric landmarks in the femur; and providing, on the user interface, an output based on the Dorr type of the femur.

12. The method according to claim 1, wherein: the trained machine learning model is trained using a set of training images, each training image in the set of training images comprising information relating to cortical bone tissue of at least a part of a bone in the training image; and each training image in the set of training images is labeled with bone mineral density of the bone in the training image.

13. A computer program product comprising one or more computer-readable storage media having program instructions collectively stored on the one or more computer-readable storage media, the program instructions executable to: obtain image data comprising information relating to cortical bone tissue of at least a part of a first bone; preprocess the image data, by isolating a region of interest in the image data, to generate preprocessed image data; apply the preprocessed image data as inputs across a trained machine learning model to generate output indicative of bone mineral density of the first bone; determine a T-score value based on the bone mineral density of the first bone; and provide, on a user interface, an output based on the T-score value.

14. The computer program product according to claim 13, the program instructions further being executable to: determine a Z-score value based on the bone mineral density of the first bone; and provide, on the user interface, an output based on the Z-score value.

15. The computer program product according to claim 13, the program instructions further being executable to: determine a fracture risk based on the bone mineral density of the first bone; and provide, on the user interface, an output based on the fracture risk assessment.

16. The computer program product according to claim 13, the program instructions further being executable to: determine a recommended surgical implant type based on the bone mineral density of the first bone or the T-score value; and provide, on the user interface, an output based on the recommended surgical implant type.

17. A system comprising: a processor, a computer-readable memory, one or more computer-readable storage media, and program instructions collectively stored on the one or more computer-readable storage media, the program instructions executable to: obtain image data comprising information relating to cortical bone tissue of at least a part of a first bone; preprocess the image data, by isolating a region of interest in the image data, to generate preprocessed image data; apply the preprocessed image data as inputs across a trained machine learning model to generate output indicative of bone mineral density of the first bone; determine a T-score value based on the bone mineral density of the first bone; and provide, on a user interface, an output based on the T-score value.

18. The system according to claim 17, the program instructions further being executable to: determine a Z-score value based on the bone mineral density of the first bone; and provide, on the user interface, an output based on the Z-score value.

19. The system according to claim 17, the program instructions further being executable to: determine a fracture risk based on the bone mineral density of the first bone; and provide, on the user interface, an output based on the fracture risk assessment.

20. The system according to claim 17, the program instructions further being executable to: determine a recommended surgical implant type based on the bone mineral density of the first bone or the T-score value; and provide, on the user interface, an output based on the recommended surgical implant type.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0018] FIG. 1 depicts an example environment in which selected aspects of the present disclosure may be implemented, in accordance with various implementations.

[0019] FIG. 2 depicts a flowchart illustrating an example method for practicing selected aspects of the present disclosure.

[0020] FIG. 3 depicts an example architecture of a computing device.

[0021] FIG. 4 depicts an example of training a convolutional neural network algorithm.

[0022] FIG. 5 depicts an example of bone mineral density determination and treatment, in accordance with various implementations.

[0023] FIG. 6 depicts an example of using a T-score to select an implant and treatment, in accordance with various implementations.

DETAILED DESCRIPTION

[0024] FIG. 1 schematically depicts an example environment 100 in which selected aspects of the present disclosure may be implemented, in accordance with various implementations. Any computing devices depicted in FIG. 1 or elsewhere in the figures may include logic such as one or more microprocessors (e.g., central processing units or “CPUs”, graphical processing units or “GPUs”) that execute computer-readable instructions stored in memory, or other types of logic such as application-specific integrated circuits (“ASIC”), field-programmable gate arrays (“FPGA”), and so forth. Some of the systems depicted in FIG. 1, such as a cloud analysis server 110, may be implemented using one or more server computing devices that form what is sometimes referred to as a “cloud infrastructure,” although this is not required.

[0025] In implementations, the environment 100 may include a cloud analysis server 110 that includes an orthopedic surgical planning system 120 that implements an orthopedic surgical planning application that is accessible from various clients, including clients 150-1, . . . , 150-n that may be included in the environment 100, through either a thin client interface, such as a web browser (e.g., a web-based orthopedic surgical planning application), or a program interface. In implementations, the orthopedic surgical planning application that is implemented by the orthopedic surgical planning system 120 may be a software as a service (SaaS) orthopedic surgical planning application. The orthopedic surgical planning system 120 and the clients 150-1, . . . , 150-n may be in communication via a computer network 160, which may be any suitable network including any combination of a local area network (LAN), wide area network (WAN), or the Internet. The orthopedic surgical planning system 120 may be configured to perform selected aspects of the present disclosure in order to determine an accurate BMD score and subsequent fracture risk assessment from X-rays and/or to determine treatments to minimize fracture risk.

[0026] Each of the clients 150-1, . . . , 150-n may be, for example, a user computing device that is used by a user to access an orthopedic surgical planning application via an orthopedic surgical planning application user interface, such as a SaaS orthopedic surgical planning application, that is provided by the orthopedic surgical planning system 120, e.g., through a web browser. In an example, the clients 150-1, . . . , 150-n may be user computing devices associated with an individual or an entity or organization such as a hospital, doctor's office, clinic, etc. or any other organization that uses an orthopedic surgical planning application. For example, a surgeon may operate an orthopedic surgical planning application to determine a patient's BMD score, to perform a fracture risk assessment, and/or to determine an appropriate treatment and treat a patient.

[0027] In various implementations, the environment 100 may include picture archiving and communication system 130, which may store patient X-ray images that may be obtained and analyzed by the orthopedic surgical planning system 120 to determine a patient's BMD score. The picture archiving and communication system 130 may be implemented using one or more server computing devices that form what is sometimes referred to as a “cloud infrastructure,” although this is not required. The orthopedic surgical planning system 120 and the picture archiving and communication system 130 may be in communication via the computer network 160.

[0028] In various implementations, the environment 100 may include hospital information system (HIS)/radiology information system (RIS)/electronic patient records (EPR) 140, which may store patient records that may be obtained, utilized, and updated by the orthopedic surgical planning system 120. The HIS/RIS/EPR 140 may be implemented using one or more server computing devices that form what is sometimes referred to as a “cloud infrastructure,” although this is not required. The orthopedic surgical planning system 120 and the HIS/RIS/EPR 140 may be in communication via the computer network 160.

[0029] FIG. 2 is a flowchart illustrating an example method 200 of automated bone mineral density and fracture risk assessment, in accordance with implementations disclosed herein. For convenience, the operations of the flowchart are described with reference to a system that performs the operations. This system may include various components of various computer systems, such as one or more components of the cloud analysis server 110. Moreover, while operations of method 200 are shown in a particular order, this is not meant to be limiting. One or more operations may be reordered, omitted, or added.

[0030] At block 205, the system may obtain image data including information relating to cortical bone tissue of at least a part of a first bone. In implementations, at block 205, the orthopedic surgical planning system 120 of the cloud analysis server 110 may obtain image data including information relating to cortical bone tissue of at least a part of a first bone, e.g., from the picture archiving an communication system 130. In an example, responsive to a request from one of the clients 150-1, . . . , 150-n to perform a BMD analysis and/or fracture risk assessment for a particular patient having an electronic patient record in the HIS/RIS/EPR 140, the orthopedic surgical planning system 120 may obtain portions of the electronic patient record for the patient from the HIS/RIS/EPR 140 and may obtain X-ray image data for the patient from the picture archiving and communication system 130.

[0031] Still referring to FIG. 2, at block 210, the system may preprocess the image data, by isolating a region of interest in the image data, to generate preprocessed image data. In implementations, at block 210, the orthopedic surgical planning system 120 of the cloud analysis server 110 may preprocess the image data obtained at block 205, to generate preprocessed image data. In some implementations, the orthopedic surgical planning system 120 may pre-process the image data by co-registering it to a common space, isolating regions of interest (e.g., a single femur), and optionally performing image segmentation.

[0032] Still referring to FIG. 2, at block 215, the system may apply the preprocessed image data as inputs across a trained machine learning model to generate output indicative of bone mineral density of the first bone. In implementations, at block 215, the orthopedic surgical planning system 120 of the cloud analysis server 110 may apply the preprocessed image data generated at block 210 as inputs across a trained machine learning model to generate output indicative of bone mineral density of the first bone. The trained machine learning model may be a convolutional neural network. In other implementations, at block 215, the orthopedic surgical planning system 120 of the cloud analysis server 110 may use vision transformers to generate output indicative of bone mineral density of the first bone.

[0033] Still referring to FIG. 2, at block 220, the system may determine a T-score value based on the bone mineral density of the first bone. In implementations, at block 220, the orthopedic surgical planning system 120 of the cloud analysis server 110 may determine a T-score value based on the bone mineral density of the first bone, determined based on the output of the trained machine learning model at block 215. In addition to using the bone mineral density of the first bone, the orthopedic surgical planning system 120 of the cloud analysis server 110 may use demographic information (e.g., ethnicity, race, etc.) to determine the T-score value.

[0034] Still referring to FIG. 2, at block 225, the system may provide, on a user interface, an output based on the T-score value. In implementations, at block 225, the orthopedic surgical planning system 120 of the cloud analysis server 110 may provide, on a user interface of the client 150-1, . . . , 150-n that requested the BMD analysis and/or fracture risk assessment for the particular patient, an output based on the T-score value determined at block 220.

[0035] Still referring to FIG. 2, at block 230, the system may determine a Z-score value based on the bone mineral density of the first bone. In implementations, at block 230, the orthopedic surgical planning system 120 of the cloud analysis server 110 may determine a Z-score value based on the bone mineral density of the first bone determined based on the output of the trained machine learning model at block 215.

[0036] Still referring to FIG. 2, at block 235, the system may provide, on a user interface, an output based on the Z-score value. In implementations, at block 235, the orthopedic surgical planning system 120 of the cloud analysis server 110 may provide, on a user interface of the client 150-1, . . . , 150-n that requested the BMD analysis and/or fracture risk assessment for the particular patient, an output based on the Z-score value determined at block 230.

[0037] Still referring to FIG. 2, at block 240, the system may determine a fracture risk based on the bone mineral density of the first bone. In implementations, at block 240, the orthopedic surgical planning system 120 of the cloud analysis server 110 may determine a fracture risk based on the bone mineral density of the first bone determined based on the output of the trained machine learning model at block 215.

[0038] Still referring to FIG. 2, at block 245, the system may provide, on a user interface, an output based on the fracture risk assessment. In implementations, at block 245, the orthopedic surgical planning system 120 of the cloud analysis server 110 may provide, on a user interface of the client 150-1, . . . , 150-n that requested the BMD analysis and/or fracture risk assessment for the particular patient, an output based on the fracture risk assessment determined at block 240.

[0039] Still referring to FIG. 2, at block 250, the system may determine a recommended surgical implant type based on the bone mineral density of the first bone or the T-score value. In implementations, at block 250, the orthopedic surgical planning system 120 of the cloud analysis server 110 may determine a recommended surgical implant type based on the bone mineral density of the first bone determined based on the output of the trained machine learning model at block 215 or the T-score value determined at block 220.

[0040] Still referring to FIG. 2, at block 255, the system may provide, on a user interface, an output based on the recommended surgical implant type. In implementations, at block 255, the orthopedic surgical planning system 120 of the cloud analysis server 110 may provide, on a user interface of the client 150-1, . . . , 150-n that requested the BMD analysis and/or fracture risk assessment for the particular patient, an output based on the recommended surgical implant type determined at block 250.

[0041] Still referring to FIG. 2, at block 260, the system may determine a recommended treatment based on the bone mineral density of the first bone or the T-score value. In implementations, at block 260, the orthopedic surgical planning system 120 of the cloud analysis server 110 may determine a recommended treatment based on the bone mineral density of the first bone determined based on the output of the trained machine learning model at block 215 or the T-score value determined at block 220.

[0042] Still referring to block 260, in an example, a first threshold may be −3, and a second threshold may be −1. In response to determining that a femur or other bone has a T-score that satisfies (e.g., is greater than) both the first threshold and the second threshold (e.g., has a T-score that is greater than −1), the orthopedic surgical planning system 120 may recommend no change to surgical protocol, and standard implant selection. In response to determining that a femur or other bone has a T-score that satisfies the first threshold but does not satisfy the second threshold (e.g., has a T-score that is in the range of −1 to −3), the orthopedic surgical planning system 120 may recommend considering implant selection and prophylactic treatment for osteopenia. In response to determining that a femur or other bone has a T-score that does not satisfy the first threshold and does not satisfy the second threshold (e.g., has a T-score that is less than −3), the orthopedic surgical planning system 120 may recommend initiating pharmacologic/orthobiologic intervention and cemented stem and prophylactic intraoperative cerclage wiring. In other implementations, a different number of thresholds may be used (e.g., one threshold, three thresholds, etc.). In other implementations, bone mineral density, Z-score, etc. may be used instead of or in addition to a T-score.

[0043] Still referring to FIG. 2, at block 265, the system may provide, on a user interface, an output based on the recommended treatment. In implementations, at block 265, the orthopedic surgical planning system 120 of the cloud analysis server 110 may provide, on a user interface of the client 150-1, . . . , 150-n that requested the BMD analysis and/or fracture risk assessment for the particular patient, an output based on the recommended treatment determined at block 260.

[0044] FIG. 3 is a block diagram of an example computing device 310 that may optionally be utilized to perform one or more aspects of techniques described herein. Computing device 310 typically includes at least one processor 314 that communicates with a number of peripheral devices via bus subsystem 312. These peripheral devices may include a storage subsystem 324, including, for example, a memory subsystem 325 and a file storage subsystem 326, user interface output devices 320, user interface input devices 322, and a network interface subsystem 316. The input and output devices allow user interaction with computing device 310. Network interface subsystem 316 provides an interface to outside networks and is coupled to corresponding interface devices in other computing devices.

[0045] User interface input devices 322 may include a keyboard, pointing devices such as a mouse, trackball, touchpad, or graphics tablet, a scanner, a touchscreen incorporated into the display, audio input devices such as voice recognition systems, microphones, and/or other types of input devices. In general, use of the term “input device” is intended to include all possible types of devices and ways to input information into computing device 310 or onto a communication network.

[0046] User interface output devices 320 may include a display subsystem, a printer, a fax machine, or non-visual displays such as audio output devices. The display subsystem may include a cathode ray tube (CRT), a flat-panel device such as a liquid crystal display (LCD), a projection device, or some other mechanism for creating a visible image. The display subsystem may also provide non-visual display such as via audio output devices. In general, use of the term “output device” is intended to include all possible types of devices and ways to output information from computing device 310 to the user or to another machine or computing device.

[0047] Storage subsystem 324 stores programming and data constructs that provide the functionality of some or all of the modules described herein. For example, the storage subsystem 324 may include the logic to perform selected aspects of the methods of FIG. 2, as well as to implement various components depicted in FIG. 1.

[0048] These software modules are generally executed by processor 314 alone or in combination with other processors. The memory subsystem 325 included in the storage subsystem 324 can include a number of memories including a main random access memory (RAM) 330 for storage of instructions and data during program execution and a read only memory (ROM) 332 in which fixed instructions are stored. A file storage subsystem 326 can provide persistent storage for program and data files, and may include a hard disk drive, a floppy disk drive along with associated removable media, a CD-ROM drive, an optical drive, or removable media cartridges. The modules implementing the functionality of certain implementations may be stored by file storage subsystem 326 in the storage subsystem 324, or in other machines accessible by the processor(s) 314.

[0049] Bus subsystem 312 provides a mechanism for letting the various components and subsystems of computing device 310 communicate with each other as intended. Although bus subsystem 312 is shown schematically as a single bus, alternative implementations of the bus subsystem may use multiple busses.

[0050] Computing device 310 can be of varying types including a workstation, server, computing cluster, blade server, server farm, or any other data processing system or computing device. Due to the ever-changing nature of computers and networks, the description of computing device 310 depicted in FIG. 3 is intended only as a specific example for purposes of illustrating some implementations. Many other configurations of computing device 310 are possible having more or fewer components than the computing device depicted in FIG. 3.

[0051] FIG. 4 depicts an example of training a CNN. In some implementations, as illustrated in FIG. 4, the CNN may be trained using retrospective geometrical indices measurement of preoperative hip arthroplasty or other hip x-rays of patients who also had a concomitant dual-energy x-ray absorptiometry (DXA) osteoporosis examination within, e.g., one year of the original hip x-ray. Patient-specific DXA results paired with the patient-specific hip x-ray may be used as the ground truth for the CNN algorithm. In an example, approximately 600,000 hip x-rays may be used to train the CNN, with femur types classified according to DXA T-scores. Patients with previous fragility and THA periprosthetic fractures may be identified. In an example, femurs having T-scores in the range of −1 to −2.5 may be labeled as having osteopenia, and femurs having T-scores less than or equal to −2.5 may be labeled as having osteoporosis.

[0052] FIG. 5 depicts an example of bone mineral density determination and treatment, in accordance with various implementations. Hip arthritis may be diagnosed, with hip replacement indicated. The orthopedic surgical planning system 120 of FIG. 1 may provide an immediate pre-operative BMD report, and hip replacement may be performed using an implant type, size, and/or position recommended by the orthopedic surgical planning system 120. Patient readmissions may be reduced, and patient satisfaction may be increased.

[0053] FIG. 6 depicts an example of using a T-score to select an implant and treatment, in accordance with various implementations. In an example, in response to determining that a femur or other bone has a T-score greater than −1, the orthopedic surgical planning system 120 may recommend no change to surgical protocol, and standard implant selection. In response to determining that a femur or other bone has a T-score in the range of −1 to −3, the orthopedic surgical planning system 120 may recommend considering implant selection and prophylactic treatment for osteopenia. In response to determining that a femur or other bone has a T-score less than −3, the orthopedic surgical planning system 120 may recommend initiating pharmacologic/orthobiologic intervention and cemented stem and prophylactic intraoperative cerclage wiring.

[0054] While several implementations have been described and illustrated herein, a variety of other means and/or structures for performing the function and/or obtaining the results and/or one or more of the advantages described herein may be utilized, and each of such variations and/or modifications is deemed to be within the scope of the implementations described herein. More generally, all parameters, dimensions, materials, and configurations described herein are meant to be exemplary and that the actual parameters, dimensions, materials, and/or configurations will depend upon the specific application or applications for which the teachings is/are used. Those skilled in the art will recognize, or be able to ascertain using no more than routine experimentation, many equivalents to the specific implementations described herein. It is, therefore, to be understood that the foregoing implementations are presented by way of example only and that, within the scope of the appended claims and equivalents thereto, implementations may be practiced otherwise than as specifically described and claimed. Implementations of the present disclosure are directed to each individual feature, system, article, material, kit, and/or method described herein. In addition, any combination of two or more such features, systems, articles, materials, kits, and/or methods, if such features, systems, articles, materials, kits, and/or methods are not mutually inconsistent, is included within the scope of the present disclosure.