METHOD FOR OBTAINING A CT-LIKE REPRESENTATION AND VIRTUAL X-RAY IMAGES IN ARBITRARY VIEWS FROM A TWO-DIMENSIONAL X-RAY IMAGE

Abstract

A method for obtaining a three-dimensional (3-D) representation of a body part of a subject from at least one two-dimensional (2-D) X-ray image of the body part comprising: providing at least one 2-D X-ray image of a body part of a subject; providing a continuous parametric 3-D model of the body part corresponding to the imaged body part of the subject; adjusting at least one feature of the continuous parametric 3-D model to match the corresponding feature of the at least one 2-D X-ray image of the body part of the subject, thereby generating a 3-D representation of the imaged body part of the subject from the continuous parametric 3-D model; and generating at least one virtual 2-D X-ray image from the 3-D representation of the imaged body part of the subject generated.

Claims

1. A method for obtaining a three-dimensional (3-D) representation of a body part of a subject from at least one two-dimensional (2-D) X-ray image of the body part comprising: a. obtaining at least one 2-D X-ray image of a body part of a subject, b. obtaining a continuous parametric 3-D model of the body part corresponding to the imaged body part of the subject, c. adjusting at least one feature of the continuous parametric 3-D model to match the corresponding feature of the at least one 2-D X-ray image of the body part of the subject, wherein the at least one feature is selected from the group comprising an anatomical landmark, contour, shape, image intensity, structure and silhouette of the at least one 2-D X-ray image of the body part of the subject, thereby generating a 3-D representation of the imaged body part of the subject from the continuous parametric 3-D model, d. generating at least one virtual 2-D X-ray image from the 3-D representation of the imaged body part of the subject generated in step c., e. determining the difference between the at least one virtual 2-D X-ray image and the at least one 2-D X-ray image of the body part of the subject with respect to the at least one feature, f. further adjusting the least one feature of the continuous parametric 3-D model to decrease the difference to the corresponding feature of the at least one 2-D X-ray image of the body part of the subject, thereby generating an improved 3-D representation of the imaged body part of the subject from the continuous parametric 3-D model, g. repeating steps d. to f. until the difference between the at least one virtual 2-D X-ray image generated from the improved 3-D representation and the at least one 2-D X-ray image of the body part of the subject is not further decreased, thereby obtaining a final (or optimized) 3-D representation of the body part of the subject.

2. The method according to claim 1, wherein at least one X-ray marker of defined dimension is present in the 2-D X-ray image and wherein the at least one X-ray marker is used to determine at least one feature selected from the group comprising an anatomical landmark, the distance between the imaged body part and the X-ray source and/or the detector, the size and/or orientation of the imaged body part.

3. The method according to claim 1, further comprising: h. generating a virtual computed tomography (CT) data set from the final 3-D representation of step g.

4. The method according to claim 3, wherein the continuous parametric 3-D model comprises data on variations in shape and image intensity learned from CT-data sets the continuous parametric 3-D model is derived from, wherein the at least one adjusted feature comprises image intensity, and wherein step h. comprises sampling the intensity data of the final 3-D representation to a grid, thereby modelling the intensity data in a volume inside and/or outside of the final 3-D representation.

5. The method according to claim 3, further comprising: i. generating at least one virtual 2-D X-ray image from the virtual CT data set generated in step h.

6. The method according to claim 5, wherein step i. comprises: defining a virtual X-ray setup comprising at least a source and a detector, virtually arranging the virtual CT data set between the source and the detector of the virtual X-ray setup, simulating the X-ray process within the virtual X-ray setup to create at least one virtual 2-D X-ray from the virtual CT data set.

7. The method according to claim 1, wherein the imaged body part is a severed body part, and wherein the continuous parametric 3-D model and/or the (final) 3-D representation are a healthy or intact representation of the severed body part.

8. The method according to claim 1, wherein a continuous fluoroscopic (real-time moving) 2-D X-ray image of a moving body part of a subject is provided, and wherein the (final) 3-D representation is a moving 3-D representation of the imaged body part in motion.

9. The method according to claim 1, wherein the body part of the subject is selected from the group of one or more organ(s), limb(s), bone(s), joint(s) and/or a portion and/or a combination thereof.

10. A device or system for carrying out the steps of the method according to claim 3, comprising a graphical user interface system configured to display the (final) 3-D representation and/or the at least one virtual CT data set, generated according to claim 3 of the body part of the subject.

11. A computer-program for obtaining a three-dimensional (3-D) representation of a body part of a subject from at least one two-dimensional (2-D) X-ray image of the body part of the subject, comprising instructions which, when the program is executed by a computer, cause the computer to execute the steps of the method according to claim 1 and to display the (final) 3-D representation and/or the at least one virtual CT data set of the body part of the subject on a graphical user interface system.

12. A computer-readable storage medium having stored thereon the computer program of claim 11.

13. A method of constructing a joint prosthesis or an osteosynthesis plate or prosthesis comprising using a 3-D representation, relating to the shape and appearance of at a body part, and obtained according to the method according to claim 1.

14. A method of planning and/or evaluating the outcome of a surgical procedure on an imaged body part comprising using a 3-D representation obtained according to the method of claim 1.

15. The method according to claim 1, wherein the body part of the subject comprises at least one implant, screw, plate, joint prosthesis, osteosynthesis plate and/or any combination thereof.

16. A method of treating a subject comprising evaluating a three-dimensional (3-D) representation of a body part of the subject obtained by the method according to claim 1 and designing a treatment based on the three-dimensional (3-D) representation of a body part of the subject.

Description

FIGURES

[0113] The invention is further described by the following figures. These are not intended to limit the scope of the invention but represent preferred embodiments of aspects of the invention provided for greater illustration of the invention described herein.

DETAILED DESCRIPTION OF THE FIGURES

[0114] FIG. 1: Intensity-based registration of a 3-D model (a-SSIM) to an 2-D X-ray. Training data is used to create an a-SSIM. The candidates for a potential match are selected from the statistical model. To evaluate the fit, virtual X-ray images are projected from the candidates and then compared to the original radiographs by means of 2-D similarity measures. This process iterates until the anatomical representation that best fits the intensity information in the clinical (real X-ray) radiograph is found. This process results in an optimized representation of an a-SSIM. It can be used as a 3D model and for the computation of 3D clinical parameters.

[0115] FIG. 2: Particular embodiments of the method according to the invention. The illustration depicts the process of generating a virtual/pseudo CT data set and virtual X-rays. The optimized representation of the a-SSIM not only contains information about the shape but also about the bone density and the articulation. This information is used for the generation of a virtual/pseudo CT data set and/or a virtual X-ray.

[0116] FIG. 3: Example of automated reconstruction of the pelvis from a single radiograph. Here 32 landmarks are detected in the AP radiograph (a), corresponding 3-D model points (blue) are morphed to back-projected landmarks for initialization (b), and virtual and reference 2-D radiographs are compared to match the anatomical shape (c). Anatomical parameters of the hip. Four points at the iliac spines and pubic tubercles define the anterior pelvic plane (APP), another set of landmarks around the acetabular rim characterize the orientation of the acetabulum (in blue). The sacral slope is obtained by connecting two implicit landmarks and projecting them into sagittal plane. Clinical tilt is the oriented angle between the line connecting sacral midpoint and the midpoint of the hip axis. The APP tilt relates the orientation of the APP to the X-ray plane (d).

[0117] FIG. 4: Reconstructed surfaces via the automated method and reference radiographs. (a) clinical radiograph overlapped with reconstruction (hemipelvis); DRR and reconstruction of the same patient in (b) single-view and (c) dual-view setup.

[0118] FIG. 5: Shape and intensity model of the right femur. The mean model (a) containing shape and intensity information is deformed using the first four modes of variation (b-e). Depicted here is the variation in shape, with opaque surfaces representing the deformation in negative direction and transparent surfaces in positive direction of the respective mode. (f) Compactness and generalization ability of the femur SSIMs with intensity functions of degree d=2. The generalization ability of the model is given in terms of average surface distances and standard deviations (black bars) when fitting the combined shape/intensity to unseen datasets using the n most significant modes of variation.

[0119] FIG. 6: Overview of the reconstruction pipeline for deriving the plate and femur. Two radiographs and the implant geometry are given as an input (a). The implant geometry is registered to the X-rays in order to derive the X-ray calibration and implant pose (b). A SSIM of the femur is then fitted to both radiographs simultaneously, resulting in the 3-D shape of the patient-specific femur in relative pose to the implant as depicted in the reference images. Comparison between ground-truth and 3-D reconstruction from 2-D radiographs (Ehlke et al., 2015). Ground-truth shape of the femur and implant from CT (left) of case 1-L and the 3D-reconstructed femoral shape and implant pose based on reference DRRs (right). The SSIM extrapolates over the fracture gap depicted in the X-rays (d).

[0120] FIG. 7: Particular embodiments of the method according to the invention. The illustration depicts a general overview of three outputs that can be generated with the method according to the invention, based on at least one 2-D X-ray image. The obtained output, such as (1) a 3-D anatomical model representation of the imaged body part, (2) virtual CT data sets and (3) virtual 2-D X-ray images generated either from 3-D models or from virtual CT datasets, can be analyzed, displayed and further processed by common/standard computer programs for diagnostics, treatment planning, as well as prosthesis design.

EXAMPLES

[0121] The invention is further described by the following examples. These are not intended to limit the scope of the invention but represent preferred embodiments of aspects of the invention provided for greater illustration of the invention described herein.

Example 1

[0122] The performance of the initialization and reconstruction stages was evaluated quantitatively by comparing the reconstructed shapes of the hip-joint to ground-truth surfaces from 24 clinical CTs. For 10 CTs, AP radiographs of the same patients were available that showed the hip-joint in various pathological stages of femoral acetabular impingement (FAI). The proximal parts of the ilia and sacrum were not visible in eight of the images. Four patients wore gonadal shields that overlapped with the pelvis in the radiographs (FIG. 4a). The other 14 CTs were preselected from the training base of the A-SSIM such that they show the full pelvis, proximal femur and upper parts of the femoral shaft. Digitally reconstructed radiographs (DRRs) were generated from all CTs, mimicking 2-D radiographs in clinical single-view (AP) and dual-view (AP and sagittal) setups with a simulated source-detector distance of one meter (FIGS. 4b and c).

[0123] The goal was to assess the performance of the initialization method based on auto-mated landmark detection, the surface reconstruction errors after intensity-based reconstruction, as well as the extraction of anatomical parameters using vertex correspondences in single-view and dual-view setups. For this purpose, the DRRs and clinical radiographs were subdivided into three groups: [0124] FAI-XRAY contains the 10 clinical AP radiographs of the FAI cases. [0125] FAI-DRR contains AP and sagittal DRRs from the corresponding CTs of the FAI cases. [0126] MODEL-DRR contains AP and sagittal DRRs that were generated from the training base (CTs) of the A-SSIM.

[0127] Landmark Detection and Initialization

[0128] The performance of the proximal femur FALA system was previously evaluated in Lindner et al. (2013). The pelvis FALA system was assessed using three-fold cross-validation experiments based on manually annotated ground-truth. It showed an average point-to-curve error of less than 1.8 mm for 99% of all 150 images. Of note is that this performance was achieved using only 100 training images in a dataset with considerable variation due to gender, positioning and anatomical variation. Average re-projection (2-D) error per landmark after initialization of the statistical model was measured as 2.4 mm (FAI-XRAY, 10 datasets), 2.2 mm (FAI-DRR, 10 datasets) and 2.3 mm (MODEL-DRR, 14 datasets), with the first five modes of shape variation optimized in the initialization stage. A qualitative assessment of the results showed no obvious misfits between A-SSIM and X-rays, such as alignment of the wrong anatomy (pelvis fitted to the femur) or inverted orientations of the model (posterior-anterior pose in an AP setup, toe-to-head pose in head-to-toe setup). Quantitative values for the three experimental series are given in the first rows of Table 1, com-paring the surface distances and orientation of the pelvis to the ground-truth from CT.

[0129] Shape Reconstruction

[0130] The reconstructed anatomical shapes were assessed by measuring the Euclidean surface distance to ground-truth shapes from CT. For this purpose, 2-D/3-D reconstructions were performed from the given clinical radiographs and DRRs using the 20 most significant modes of variation of the A-SSIM, which together explained 67% of the shape and intensity variation in the training population. The experiments were then repeated using the full shape and intensity parameter space of the model.

[0131] The average point-to-surface distance between surfaces of the A-SSIM and ground-truth was recorded after landmark-based initialization as well as after intensity-based registration (full reconstruction). Before evaluating surface distances, the ilia and femurs of the model were aligned rigidly to the ground-truth in order eliminate the effects of global transformation and articulation on the measure. The reconstruction error in anatomical pose was assessed separately by measuring the differences in angle of the APP normal between ground-truth and reconstruction result.

[0132] Table 1, first column, lists the results from the 10 clinical radiographs (FAI-XRAY) in terms of mean distance and deviation over all experiments. Since the patient-specific pose in the original X-ray setup was unknown, the APP could not be compared to ground-truth directly and is hence marked as “n/a” in the table. It was, however, possible to estimate the true, patient-specific orientation by rigidly aligning the ground-truth shapes to the clinical X-ray images. Refer to Table 3 for a comparison of the reconstructed pelvic tilt to the estimated values from ground-truth.

[0133] The results from the DRR-based experiments (FAI-DRR and MODEL-DRR) are given in Table 1, columns two and three. The MODEL-DRR experiments were per-formed both with the full model (full) and in a leave-out fashion, where the training set depicted in the respective DRRs was removed from the model before reconstruction. The run-time per reconstruction was less than five minutes for single-view and less than ten minutes for dual-view setups.

[0134] The optional refinement stage was further applied on the FAI experiment series in order to study the impact of less restrictive geometric priors in 2-D/3-D reconstruction. In single-view scenarios, the reconstruction accuracy with respect to the ilia was reduced to 2.0 mm (FAI-XRAY) and 1.9 mm (FAI-DRR) when using refinement. The reconstruction accuracy was, however, slightly improved in dual-view experiments (1.5 mm).

TABLE-US-00001 TABLE 1 Reconstructed surfaces from clinical radiographs (FAI-XRAY) and DRRs (FAI-DRR, MODEL-DRR) compared to ground-truth. The error is given in terms of mean Euclidean surface distance, averaged over pairs of ilia and femurs (±standard deviation). The error in reconstructed anatomical pose is expressed as the mean angle between the normals of the APP from the reconstruction and from the ground-truth. Full: Ground-truth instance was contained in the A-SSIM; Leave-out: Instance was removed from the A-SSIM prior to reconstruction. FAI-XRAY FAI-DRR MODEL-DRR (n = 10) (n = 11) (n = 14) Landmark-based initialization Ilia (mm) 2.49 (±0.57) 2.47 (±0.59) 2.55 (±0.40) Femurs (mm) 2.06 (±0.59) 1.79 (±0.29) 1.59 (±0.54) APP-angle (°) n/a 4.94 (±2.63) 5.71 (±3.53) Single-view reconstruction (20 modes) leave-out full Ilia (mm) 1.86 (±0.39) 1.77 (±0.38) 1.97 (±0.28) 0.97 (±0.38) Femurs (mm) 1.35 (±0.44) 1.33 (±0.38) 1.31 (±0.30) 0.59 (±0.38) APP-angle (°) n/a 3.19 (±2.19) 3.71 (±1.96) 1.97 (±1.60) Single-view reconstruction (45 modes) leave-out full Ilia (mm) 1.91 (±0.35) 1.73 (±0.29) 2.07 (±0.39) 0.63 (±0.20) Femurs (mm) 1.28 (±0.38) 1.30 (±0.39) 1.24 (±0.33) 0.33 (±0.08) APP-angle (°) n/a 2.35 (±1.24) 4.01 (±2.50) 1.48 (±0.70) Dual-view reconstruction (45 modes) leave-out full Ilia (mm) n/a 1.59 (±0.35) 1.77 (±0.34) 0.53 (±0.19) Femurs (mm) n/a 1.19 (±0.34) 1.21 (±0.28) 0.35 (±0.11) APP-angle (°) n/a 2.34 (±1.66) 2.30 (±1.35) 0.56 (±0.62)

TABLE-US-00002 TABLE 2 Reconstructed intensities from the MODEL-DRR experiments compared to ground-truth. The results are given in terms of mean absolute error in Hounsfield units between intensities from ground-truth model instances and the intensities sampled from the reconstruction outcome, averaged over pairs of ilia and femurs (±standard deviation). Full: Ground-truth instance was contained in the A-SSIM; Leave-out: Instance was removed from the A-SSIM prior to reconstruction. MODEL-DRR Leave-out MODEL-DRR Full Single-view reconstruction (20 modes) Ilia 140.12 (±26.69) 67.77 (±25.83) Femurs 119.23 (±16.63) 55.17 (±21.19) Single-view reconstruction (45 modes) Ilia 146.97 (±18.82) 40.52 (±16.48) Femurs 116.34 (±12.00) 30.68 (±12.12) Dual-view reconstruction (45 modes) Ilia 138.34 (±14.84) 46.10 (±24.89) Femurs 113.51 (±14.21) 37.94 (±22.91)

TABLE-US-00003 TABLE 3 Anatomical parameters from clinical radiographs and DRRs compared to ground-truth (±standard deviation) in terms of absolute differences in angle (degrees). APP tilt represents the inclination angle of the anterior pelvic plane. Clinical tilt refers to the angle between midpoint of the sacral plateau and hip axis. The sacral slope was computed w.r.t. the X-ray plane in AP view. Acetabular anteversion and inclination parameters are given relative to the APP. FAI-XRAY FAI-DRR MODEL-DRR (n = 10) (n = 11) (n = 14) Single-view reconstruction (20 modes) leave-out full APP tilt 3.5 (±3.4) 3.0 (±2.4) 3.7 (±3.0) 2.1 (±1.7) Clinical tilt 2.9 (±3.0) 3.7 (±3.4) 4.1 (±3.1) 2.1 (±1.6) Sacral slope 7.2 (±3.8) 7.6 (±3.7) 10.1 (±7.2)  1.8 (±1.6) Acet. version 3.9 (±2.9) 3.3 (±2.4) 4.1 (±3.1) 2.0 (±1.2) Acet. inclination 3.4 (±3.3) 3.6 (±2.8) 3.6 (±2.0) 1.7 (±1.8) Single-view reconstruction (45 modes) leave-out full APP tilt 3.3 (±2.9) 2.3 (±0.8) 4.5 (±3.5) 1.6 (±0.8) Clinical tilt 3.8 (±4.5) 2.7 (±2.1) 3.7 (±2.5) 1.4 (±1.0) Sacral slope 7.9 (±4.5) 5.4 (±3.8) 8.7 (±5.4) 1.3 (±1.1) Acet. version 3.2 (±2.3) 3.2 (±2.3) 4.5 (±2.8) 1.5 (±1.0) Acet. inclination 4.2 (±3.1) 3.9 (±2.8) 3.6 (±2.0) 1.0 (±0.5) Dual-view reconstruction (45 modes) leave-out full APP tilt n/a 1.5 (±1.9) 3.0 (±2.0) 0.6 (±0.6) Clinical tilt n/a 2.9 (±2.2) 2.6 (±1.7) 1.3 (±1.2) Sacral slope n/a 5.9 (±5.2) 8.9 (±5.0) 2.0 (±1.7) Acet. version n/a 2.7 (±1.2) 3.3 (±2.8) 0.8 (±0.7) Acet. inclination n/a 3.4 (±2.8) 3.0 (±2.3) 0.8 (±0.9)

[0135] Intensity Reconstruction

[0136] The intensities reconstructed in the MODEL-DRR experiments were compared to the ground-truth intensities from the corresponding training sets of the A-SSIM. To evaluate intensity information exclusively, shape and transformations of the reconstruction outcome were normalized following the approach of sampling Bernstein coefficients from CT. The Bernstein coefficients from the ground-truth instances, which originated from the CTs that the DRRs were generated from, were mapped to the mean tetrahedral grid. The 2-D/3D reconstructed coefficients from the DRR-based experiments were transferred onto the same tetrahedral grid, making use of the vertex correspondence between model instances. Both polynomial distributions (reconstructed and ground-truth) were then sampled as Hounsfield units onto CT-like voxel grids of resolution 0.50.50.5 mm and evaluated against each other, ignoring surrounding voxels that were located outside the mean tetrahedral grid. The results are given in Table 2 for single-view and dual-view experiments.

[0137] Reconstruction of Anatomical Parameters

[0138] The anatomical parameters of the imaged body part were derived from the reconstructed shapes and then evaluated with respect to the patient-specific pose in the given X-ray setups. To establish the ground-truth pose from clinical radiographs (FAIXRAY), the CTs of the patients were fitted rigidly to the given X-ray images by means of 2-D/3-D registration. The outcome was then confirmed qualitatively by an expert observer. The ground-truth pose in the DRR-based experiments (FAI-DRR, MODELDRR) was given at the time of DRR generation by the orientation of the CT datasets relative to the camera setup (extrinsic camera parameters). Table 3 compares the anatomical parameters from the shape reconstruction experiments to ground-truth from CT. The APP, clinical tilt, sacral slope and acetabular anteversion/inclination were computed from anatomical 3-D landmarks on the reconstructed shapes and in the CTs. The landmarks on the reconstructed shapes were extracted automatically using vertex identifiers and the hip-joint center of the A-SSIM. Corresponding ground-truth landmarks were defined manually on the CTs and then transformed in the X-ray setups using the known pose of the patients.

Example 2

[0139] SSIMs of the Femur

[0140] Following the method according to the invention, SSIMs of the whole femur were generated based on 18 preoperative CT scans of mixed male and female patients that were treated for osteoarthritis of the knee. The images showed either the right or left side of the lower limb together with the hip and knee joints at a resolution below 0.50.50.9 mm per voxel in all datasets. The CTs were mirrored along the transverse axis to represent opposite sides, resulting in 18 training sets for the left and the right femur. Each set was segmented using the automated approach described in Bindernagel et al. (2011) and segmentations later refined manually by an expert observer.

[0141] An existing SSM of the femur, originally created from 95 MRIs in previous experiments, was utilized to establish vertex correspondence with respect to outer points. The SSM was fitted to 3D label field representations of the segmented CTs, mimicking the iterative approach proposed in Bindernagel et al. (2011); Kainmueller et al. (2009). Training shapes were then extracted by projecting the vertices of the matched SSMs along their normal direction to the segmented contours. The vertices and triangulation of the original SSM were thereby transferred to the segmentations from CT. In order to establish correspondence on inner points, a template mesh of the mean shape, consisting of 140 k tetrahedra, was morphed to the training surfaces via mean value coordinates deformation (Floater, 2003). The intensities were incorporated into the model by sampling Bernstein intensity functions of degree d=2 from the CTs onto the meshes. A PCA was then computed on the combined shape and intensity information as well as for shape and intensity independently. The combined shape and intensity model of the right femur is depicted in FIG. 5. The most significant modes of variation express the scale and orientation of the femoral neck and trochanter, cortical thickness, as well as the length and thickness of the femoral shaft. High-frequency variations of the anatomical shape due to pathologies in the joint regions are captured by the least significant modes of the model. The compactness and generalization ability of the femur SSIM were evaluated for Bernstein intensity functions of second order. The compactness was analyzed based on the eigenvalues associated with a PCA on the tetrahedral meshes in the model, as well as for the combined PCA on shape and intensity information. A leave-one-out cross validation was performed on the SSIM to measure the generalization ability in terms of average surface distance to unseen training data across the most significant modes of variation. The results of the analysis on compactness and generalization ability are depicted in FIG. 5f.

[0142] Evaluation Based on DRRs

[0143] The evaluation utilizes CT datasets that have originally been acquired for an assessment of interfragmentary lag screw fixation in locking plate constructs (Mardian et al., 2015). The CTs depict four pairs of cadaveric distal femoral bones and two individual femurs, with a fracture gap at the distal shaft. Surrounding tissue is not present in the datasets. A standard 9-hole 4.5/5.0 LISS DF2 plate fixates the fracture using seven locking screws distally and three screws proximally. Based on the CT scans, digitally reconstructed radiographs (DRRs) were generated in AP and mediolateral view in order to mimic clinical X-ray images taken at a source detector distance of one meter. The global X-ray setup was then derived as proposed using a CAD model of the implant. Afterwards, the 3-D shape of the femur was reconstructed in the global X-ray setup. The cadaveric bones were not contained in the training base of the statistical prior. A surface model of the intact bone was registered to the CT data, which then acted as ground-truth for evaluation. To assess the distance between implant and bone at consistent point locations, an ideal plate geometry was registered rigidly to each reconstructed plate as well as to the plates depicted in the CTs. The surface distance between the reconstructed bone surface from the statistical model and the reconstructed plate as well as the surface distance between the intact bone and the ground-truth plate were then compared to each other. In addition, the surface error between intact femur and reconstructed femur was assessed after rigid registration of the reconstructed femoral shape to the ground-truth.

TABLE-US-00004 TABLE 4 Error in plate-to-bone distance in mm between reconstructed plate and femur against ground-truth per case. Listed are the root-mean-square error (RMSE), mean error as well as the maximum error over all plate nodes including proximal, distal and mid-section of the osteosynthesis. L and R denote left and right femur respectively, with results from both sides listed when available. Cases 5-L and 6-R were treated as outliers, since the implant plate penetrated the femur. 1-L/R 2-L/R 3-L/R 4-L 5-L/R 6-R RMSE [mm] 1.5/1.2 1.3/1.5 1.7/1.2 1.36 1.9/1.7 2.6 Mean [mm] 0.1/0.8 0.3/0.8 0.6/0.2 0.2 1.5/1.1 1.5 Max [mm] 4.7/2.4 4.0/4.3 5.1/4.1 4.9 3.7/5.2 6.0

[0144] FIG. 6 exemplary shows the osteosynthesis from CT compared to the reconstructed femur and implant for case 1-L. The mean error in plate-to-bone surface distance over the whole plate is given for each case individually in Table 4. Root-mean-square-error (RMSE) was at most 2.6 mm for all tested cases, with an absolute mean deviation smaller than 1.5 mm. The surface of the femur penetrated the implant in reconstructions of cases 5-L and 6-R, thus representing anatomically implausible osteosynthesis constructs. Excluding these outliers yields an overall RMSE of at most 1.7 mm and a maximum mean error of 1.1 mm. Further restriction of nodes to the midsection of the plate results in RMSE less than 1.4 mm, with the span ranging from −2.4 mm to +2.9 mm. Compared to the intact bone surface, the femoral surface is reconstructed with a mean error of at most 1.5 mm, excluding cases 5-L and 6-R. The two outliers exhibit a mean error of 1.8 mm and 2.0 mm respectively.

REFERENCES

[0145] M. Ehlke, H. Ramm, H. Lamecker, H. Hege and S. Zachow, “Fast Generation of Virtual X-ray Images for Reconstruction of 3D Anatomy,” in IEEE Transactions on Visualization and Computer Graphics, vol. 19, no. 12, pp. 2673-2682, December 2013, doi: 10.1109/TVCG.2013.159. [0146] S. Z. Aydin, Kasapoglu G. E., E. Kurum, S. Akar, H. E. Mungan, F. Alibaz-Oner, R. G. Lambert, P. Atagunduz, H. Marzo Ortega, D. McGonagle, and W. P. Maksymowych. Limited reliability of radiographic assessment of spinal progression in ankylosing spondylitis. Rheumatology, 56(12):2162-2169, 2017. [0147] M. Ehlke, M. Heyland, S. Mardian, G. N. Duda, and S. Zachow. 3D assessment of osteosynthesis based on 2D radiographs. In Jahrestagung der Deutschen Gesellschaft fürComputer- und Roboterassistierte Chirurgie (CURAC), 2015.ZIB-Report15-4. [0148] C. Lindner, S. Thiagarajah, J. M. Wilkinson, The arcOGEN Consortium, G. A. Wallis, and T. F. Cootes. Fully automatic segmentation of the proximal femur using random forest regression voting. IEEE Transactions on Medical Imaging, 32(8):1462-1472, 2013. [0149] J. C. Clohisy, J. C. Carlisle, R. Trousdale, Y. J. Kim, P. E. Beaule, P. Morgan, K. Steger-May, P. L. Schoenecker, and M. Millis. Radiographic evaluation of the hip has limited reliability. Clinical Orthopaedics and Related Research, 467(3):666-675, 2009. [0150] G. Zheng. Statistically deformable 2D/3D registration for accurate determination of post-operative cup orientation from single standard X-ray radiograph. In Medical Image Computing and Computer-Assisted Intervention (MICCAI), pages 820-827, 2009. [0151] S. Schumann, Y. Sato, Y. Nakanishi, F. Yokota, M. Takao, N. Sugano, and G. Zheng. Cup implant planning based on 2-D/3-D radiographic pelvis reconstruction—First clinical results. IEEE Transactions on Biomedical Engineering, 62(11):2665-2673, 2015. [0152] N. Baka, B. L. Kaptein, M. de Bruijne, T. van Walsum, J. E. Giphart, W. J. Niessen, and B. P. Lelieveldt. 2D-3D shape reconstruction of the distal femur from stereo X-ray imaging using statistical shape models. Medical Image Analysis, 15(6):840-850, 2011. [0153] N. Baka, B. L. Kaptein, J. E. Giphart, M. Staring, M. de Bruijne, B. P. Lelieveldt, and E. Valstar. Evaluation of automated statistical shape model based knee kinematics from biplane fluoroscopy. Journal of Biomechanics, 47(1):122-129, 2014. [0154] J. Dworzak, H. Lamecker, J. von Berg, T. Klinder, C. Lorenz, D. Kainmüller, H. Seim, H.-C. Hege, and S. Zachow. 3D reconstruction of the human rib cage from 2D projection images using a statistical shape model. International Journal of Computer Assisted Radiology and Surgery, 5(2):111-124, 2010. [0155] H. Lamecker, T. H. Wenckebach, and H.-C. Hege. Atlas-based 3D-shape reconstruction from X-ray images. In International Conference on Pattern Recognition (ICPR), volume 1, pages 371-374, 2006. [0156] G. Zheng. Statistically deformable 2D/3D registration for accurate determination of post-operative cup orientation from single standard X-ray radiograph. In Medical Image Computing and Computer-Assisted Intervention (MICCAI), pages 820-827, 2009. [0157] T. Whitmarsh, L. Humbert, M. De Craene, L. M. Del Rio Barquero, and A. F. Frangi. econstructing the 3D shape and bone mineral density distribution of the proximal femur from dual-energy X-ray absorptiometry. IEEE Transactions on Medical Imaging, 30(12):2101-2114, 2011. [0158] V. Karade and B. Ravi. 3D femur model reconstruction from biplane X-ray images: A novel method based on Laplacian surface deformation. International Journal of Computer Assisted Radiology and Surgery, 10(4):473-485, 2015. [0159] J. Yao. A statistical bone density atlas and deformable medical image registration. PhD thesis, The Johns Hopkins University, 2002. [0160] J. Georgii and R. Westermann. A generic and scalable pipeline for GPU tetrahedral grid rendering. IEEE Transactions on Visualization and Computer Graphics, 12(5): 1345-1352, 2006. [0161] M. Weiler, M. Kraus, M. Merz, and T. Ertl. Hardware-based view-independent cell projection. IEEE Transactions on Visualization and Computer Graphics, 9(2):163-175, 2003. [0162] O. Sadowsky, J. D. Cohen, and R. H. Taylor. Rendering tetrahedral meshes with higherorder attenuation functions for digital radiograph reconstruction. In IEEE Visualization, pages 303-310, 2005. [0163] O. Sadowsky, J. D. Cohen, and R. H. Taylor. Projected tetrahedra revisited: A barycentric formulation applied to digital radiograph reconstruction using higher-order attenuation functions. IEEE Transactions on Visualization and Computer Graphics, 12(4): 461-473, 2006. [0164] M. Bindernagel, D. Kainmüller, H. Seim, H. Lamecker, S. Zachow, and H.-C. Hege. An articulated statistical shape model of the human knee. In Bildverarbeitung für die Medizin (BVM)), pages 59-63, 2011. [0165] D. Kainmueller, H. Lamecker, S. Zachow, and H.-C. Hege. An articulated statistical shape model for accurate hip joint segmentation. In Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pages 6345-6351, 2009. [0166] M. S. Floater. Mean Value Coordinates. Computer Aided Geometric Design, 20(1):19-27, 2003. [0167] S. Mardian, W. Schmölz, K.-D. Schaser, G. N. Duda, and M. Heyland. Interfragmentary lag screw fixation in locking plate constructs increases stiffness in simple fracture patterns. Clinical Biomechanics, 30(8):814-819, 2015.