Image segmentation and prediction of segmentation

Abstract

Systems and methods are provided for generating and using statistical data which is indicative of a difference in shape of a type of anatomical structure between images acquired by a first imaging modality and images acquired by a second imaging modality. This statistical data may then be used to modify a first segmentation of the anatomical structure which is obtained from an image acquired by the first imaging modality so as to predict the shape of the anatomical structure in the second imaging modality, or in general, to generate a second segmentation of the anatomical structure as it may appear in the second imaging modality based on the statistical data and the first segmentation.

Claims

1. A system configured for image segmentation, comprising: an image data interface configured to access an image of an anatomical structure of a patient, wherein the image is acquired by a first imaging modality; a memory comprising instruction data representing a set of instructions; a processor configured to communicate with the image data interface and the memory and to execute the set of instructions, wherein the set of instructions, when executed by the processor, cause the processor to: segment the image to obtain a first segmentation of the anatomical structure of the patient; access statistical data indicative of a structural difference in shape of the type of anatomical structure between a) images acquired by the first imaging modality and b) images acquired by a second imaging modality, wherein the statistical data comprises a weighted sum applied to a mean mesh generated from paired images respectively generated by the first imaging modality and the second imaging modality; and based on the first segmentation and the statistical data, generate a second segmentation of the anatomical structure which represents an estimate of the structural shape of the anatomical structure of the patient in an image acquired by the second imaging modality.

2. The system according to claim 1, wherein the set of instructions, when executed by the processor, cause the processor to compute a measurement from the second segmentation of the anatomical structure.

3. The system according to claim 2, wherein the measurement comprises at least one of: a volume measurement, a distance measurement, an area measurement, a curvature measurement, a measurement of a circumference, and a measurement of a diameter.

4. The system according to claim 1, wherein the image is a pre-interventional image, and wherein the set of instructions, when executed by the processor, cause the processor to overlay the second segmentation of the anatomical structure over an interventional image which is acquired by the second imaging modality.

5. A method for image segmentation, comprising: accessing an image of an anatomical structure of a patient, wherein the image is acquired by a first imaging modality; segmenting the image to obtain a first segmentation of the anatomical structure of the patient; accessing statistical data indicative of a structural difference in shape of the type of anatomical structure between a) images acquired by the first imaging modality and b) images acquired by a second imaging modality, wherein the statistical data comprises a weighted sum applied to a mean mesh generated from paired images respectively generated by the first imaging modality and the second imaging modality; and based on the first segmentation and the statistical data, generating a second segmentation of the anatomical structure which represents an estimate of the structural shape of the anatomical structure of the patient in an image acquired by the second imaging modality.

6. A non-transitory computer readable medium comprising instructions that, when executed by one or more processors, cause the one or more processors to: access an image of an anatomical structure of a patient, wherein the image is acquired by a first imaging modality; segment the image to obtain a first segmentation of the anatomical structure of the patient; access statistical data indicative of a structural difference in shape of the type of anatomical structure between a) images acquired by the first imaging modality and b) images acquired by a second imaging modality, wherein the statistical data comprises a weighted sum applied to a mean mesh generated from paired images respectively generated by the first imaging modality and the second imaging modality; and generate, based on the first segmentation and the statistical data, a second segmentation of the anatomical structure which represents an estimate of the structural shape of the anatomical structure of the patient in an image acquired by the second imaging modality.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

(1) These and other aspects of the invention will be apparent from and elucidated further with reference to the embodiments described by way of example in the following description and with reference to the accompanying drawings, in which

(2) FIG. 1 shows a system for generating statistical data;

(3) FIG. 2 shows a system for segmentation based on the statistical data;

(4) FIG. 3A shows a histogram of the scaling factor between mesh models obtained from cardiac segmentation of Ultrasound images and mesh models obtained from cardiac segmentation of Magnetic Resonance images for the end diastolic cardiac phase;

(5) FIG. 3B shows a mesh model for cardiac segmentation, with the gray levels indicating the mean Euclidian point-to-surface distance between the Ultrasound mesh models and the Magnetic Resonance mesh models for the end diastolic cardiac phase;

(6) FIG. 4A corresponds to FIG. 3A but for the end systolic cardiac phase;

(7) FIG. 4B corresponds to FIG. 3B but for the end systolic cardiac phase;

(8) FIG. 5 shows a method for generating statistical data;

(9) FIG. 6 shows a method for segmentation based on the statistical data; and

(10) FIG. 7 shows a computer readable medium comprising non-transitory data.

(11) It should be noted that the figures are purely diagrammatic and not drawn to scale. In the figures, elements which correspond to elements already described may have the same reference numerals.

LIST OF REFERENCE NUMBERS

(12) The following list of reference numbers is provided for facilitating the interpretation of the drawings and shall not be construed as limiting the claims. 010 image repository 012 data communication 020 first set of images 025 second set of images 030 statistical data 040 image repository 042 data communication 050 image 060 display 062 display data 080 user input device 082 user input data 100 system for generating statistical data 120 image data interface 122 internal data communication 140 processor 142 internal data communication 160 memory 200 system for segmentation 220 image data interface 222 internal data communication 240 processor 242, 244 internal data communication 260 memory 280 user interface subsystem 282 display processor 284 user input interface 300, 302 histogram of scale factors 310, 312 bins representing scale factor 320, 322 occurrence 350, 352 point-to-surface distance between US and MR meshes 400 method for generating statistical data 410 accessing first set and second set of images 420 segmenting first set of images 430 segmenting second set of images 440 generating statistical data 500 method for image segmentation 510 accessing image of patient 520 accessing statistical data 530 segmenting the image 540 generating second segmentation 600 computer readable medium 610 non-transitory data

DETAILED DESCRIPTION OF EMBODIMENTS

(13) Systems and methods are described for generating and using statistical data which is indicative of a difference in shape of a type of anatomical structure between images acquired by a first imaging modality and images acquired by a second imaging modality. This statistical data may then be used to modify a segmentation of the anatomical structure which is obtained from an image acquired by the first imaging modality so as to predict the shape of the anatomical structure in the second imaging modality. Accordingly, the statistical data may also be termed ‘shape difference data’, but is in this description referred to as statistical data since it is generated based on a statistical analysis of a set of segmentations.

(14) FIG. 1 shows a system 100 for generating the statistical data. The system 100 is shown to comprise an image data interface 120 configured to access, via data communication 012, a first set 020 and a second set 025 of images of a type of anatomical structure, with each of the sets having been acquired by a different imaging modality. In the example of FIG. 1, the image data interface 120 is shown to be connected to an external image repository 010 which comprises the image data of the sets of images 020, 025. For example, the image repository 010 may be constituted by, or be part of, a Picture Archiving and Communication System (PACS) of a Hospital Information System (HIS) to which the system 100 may be connected or comprised in. Accordingly, the system 100 may obtain access to the sets of images 020, 025 via the HIS. Alternatively, the sets of images 020, 025 may be accessed from an internal data storage of the system 100. In general, the image data interface 120 may take various forms, such as a network interface to a local or wide area network, e.g., the Internet, a storage interface to an internal or external data storage, etc.

(15) The system 100 is further shown to comprise a processor 140 configured to internally communicate with the image data interface 120 via data communication 122, and a memory 160 accessible by the processor 140 via data communication 142.

(16) The processor 140 may be configured to, during operation of the system 100, segment individual images of the first set of images 020 to obtain a first set of segmentations of the type of anatomical structure, segment individual images of the second set of images 025 to obtain a second set of segmentations of the type of anatomical structure, and based on the first set of segmentations and the second set of segmentations, generate statistical data 030 which is indicative of a difference in shape of the type of anatomical structure between a) the images acquired by the first imaging modality and b) the images acquired by the second imaging modality. FIG. 1 shows the statistical data 030 being output by the processor 140. For example, the statistical data 030 may be stored in the memory 160 or in another internal or external storage medium. Additionally or alternatively, the system 100 may comprise an output interface for outputting the statistical data 030, e.g., to another system.

(17) FIG. 2 shows a system 200 for segmentation based on the statistical data. The system 200 is shown to comprise an image data interface 220 configured to access, via data communication 042, an image 050 which is acquired by a first imaging modality. The image data interface 220 may, but does not need to be, a same type of interface as the image data interface of the system of FIG. 1. In the example of FIG. 2, the image data interface 120 is shown to be connected to an external image repository 040 which comprises the image data of the image 050. The image repository 040 may, but does not need to be, a same type of repository as the image repository described with reference to the system of FIG. 1.

(18) The system 100 is further shown to comprise a processor 240 configured to internally communicate with the image data interface 220 via data communication 222, a memory 260 accessible by the processor 240 via data communication 242, and a user interface subsystem 280 with a display processor 282 and a user input interface 284 which is configured to internally communicate with the processor 240 via data communication 244.

(19) The processor 240 may be configured to, during operation of the system 200, segment the image 050 to obtain a first segmentation of the anatomical structure of the patient, access statistical data indicative of a difference in shape of the type of anatomical structure between a) images acquired by the first imaging modality and b) images acquired by a second imaging modality, and based on the first segmentation and the statistical data, generate a second segmentation of the anatomical structure which represents an estimate of the shape of the anatomical structure of the patient in an image acquired by the second imaging modality. Although the statistical data itself is not shown in FIG. 2, it may be accessed by the system 200 from a storage medium, e.g., the memory 260 or another storage medium, including but not limited to network-accessible storage media. For that purpose, the system 200 may comprise an input interface (not shown) such as a network interface.

(20) The user interface subsystem 280 may be configured to, during operation of the system 200, enable a user to interact with the system 200 via a graphical user interface. For that purpose, the display processor 282 may be configured to generate display data 062 for a display 060 so as to display the graphical user interface to a user. The graphical user interface may be represented by a set of interface instructions stored as data in a memory accessible to the display processor 282, being for example the memory 260 or another memory of the system 200. The user input interface 284 may be configured to receive user input data 082 from a user device 080 operable by the user. The user input device 080 may take various forms, including but not limited to a computer mouse, touch screen, keyboard, microphone, etc. FIG. 2 shows the user input device to be a computer mouse 080. In general, the user input interface 284 may be of a type which corresponds to the type of user input device 080, i.e., it may be a thereto corresponding type of user device interface 284.

(21) In general, each of the systems of FIGS. 1 and 2 may be embodied as—or in—a device or apparatus, such as a workstation or imaging apparatus. The device or apparatus may comprise one or more (micro)processors which execute appropriate software. The processor of each system may each be embodied by one or more of these (micro)processors, or by a same (micro)processor. Software implementing functionality of each system, may have been downloaded and/or stored in a corresponding memory or memories, e.g., in volatile memory such as RAM or in non-volatile memory such as Flash. Alternatively, the processor of each system may be implemented in the device or apparatus in the form of programmable logic, e.g., as a Field-Programmable Gate Array (FPGA). The image data interface and user input interface may be implemented by respective interfaces of the device or apparatus. In general, each unit of each system may be implemented in the form of a circuit. It is noted that each system may also be implemented in a distributed manner, e.g., involving different devices or apparatuses. For example, the distribution may be in accordance with a client-server model, e.g., using a server and a thin-client workstation.

(22) FIGS. 3A-4B indicate the relevance of predicting the shape of an anatomical structure in a second imaging modality from a segmentation of the anatomical structure which is obtained from an image acquired by a first imaging modality. Namely, in clinical practice, measurements are frequently performed on image data from different modalities. An example for such a measurement is the ejection fraction of the left ventricle of the heart or the aortic valve opening area. Depending on the measured value being above or below a certain threshold, a disease may be diagnosed and treatment may be performed.

(23) However, it has been found that there are systematic differences in the results depending on which imaging modality the measurements are performed. This followed from a comparison of a set of Magnetic Resonance (MR) segmentation results and corresponding Ultrasound (US) segmentation results, which involved comparing a US mesh to a corresponding MR mesh for a set of patients. Before a MR mesh and an US mesh were pairwise compared, the US mesh was registered to the MR mesh by applying a point-based transformation comprising rotation, translation, and, depending on the type of comparison, also a scaling. All registrations were performed with respect to the left ventricle, since the right ventricle and the atria were not fully covered by either of the two imaging modalities.

(24) The scaling factors of a rigid point-based transformation including scaling were compared and are shown in FIGS. 3A and 4A for the end diastolic (ED) cardiac phase and the end systolic (ES) cardiac phase, respectively, with each figure showing a histogram 300, 302 of the scaling factor 310, 312 used in the transformation and the vertical axis indicating the occurrence 320, 322 of a particular scaling factor. It can be seen that there are systematic differences between the mesh models obtained from Ultrasound images and those obtained from Magnetic Resonance images, with the latter tending to be larger than the former (which is represented by a positive scaling factor). Namely, the MR meshes tended to be larger than the US meshes for the end diastolic cardiac phase, yielding scaling factors between approximately 10% and 30% and having a mean value of 8.74%. The scaling factor for the end systolic phase varied between 20% and 25% and had a mean value of 5.21%.

(25) A second registration was performed to investigate the Euclidean distance between the MR mesh and the US mesh. For the second registration, a rigid point-based registration without scaling was applied (rotation and translation). For each triangle in the MR mesh, the closest triangle in the US mesh was determined by calculating the Euclidean distance between the triangles' centers. The Euclidean distance for each triangle was then averaged over all segmentation results and depicted in FIGS. 3B and 4B, where the gray levels indicate the mean Euclidian point-to-surface distance between the Ultrasound mesh models and the Magnetic Resonance mesh models for the end diastolic cardiac phase (FIG. 3B) and the end systolic cardiac phase (FIG. 4B). Although the exact magnitudes of mean Euclidian point-to-surface distance, e.g., how many millimeters, is not of particular relevance here, it can be seen that these differences are locally distributed rather than globally uniform, indicating that there exists a difference in shape rather than only a difference in size.

(26) It is possible to compensate for such shape differences, or in particular to predict the shape of an anatomical structure in a second imaging modality from a segmentation of the anatomical structure which is obtained from an image acquired by a first imaging modality, as described in the following. Here, an exemplary embodiment is given which is considered as illustrative and as not limiting the invention, of which modifications may be made without departing from the scope of the invention as set forth in the claims.

(27) It may be assumed that a set of S corresponding segmentation results of two imaging modalities are available that show the anatomical structure of interest in a corresponding state, with the anatomical structure of interest being in the following an organ. The corresponding state may, for example, be a same heart phase in the case of cardiac images, or in general there being no interventions between the acquisitions of the images.

(28) Model-based segmentation may be used to segment the organ and to generate a set of corresponding points on the organ surface. An example of such model-based segmentation is described in “Automatic Model-based Segmentation of the Heart in CT Images” by Ecabert et al., IEEE Transactions on Medical Imaging 2008, 27(9), 1189-1201. An example of a set of M corresponding points on the organ surface is described in “Automated 3-D PDM construction from segmented images using deformable models” by Kaus et al., IEEE Transactions on Medical Imaging 2003, 22(8), 1005-1013. In a specific example, shape-constrained deformable models may be used, e.g., as described in “Shape-constrained deformable models and applications in medical imaging” by Weese et al., Lecture Notes in Computational Vision and Biomechanics, 14:151-184, 2014, which may take advantage of the a-priori knowledge about the shape of the object similar to active shape models, but which may also be flexible similar to active contour models.

(29) In general, the models used in both imaging modalities may be the same, e.g., have a same geometry and level of detail, but may also differ in their geometry and/or level of detail. An example of the latter case is that for MR, a shape-constrained deformable cine model may be used which may have 7284 vertices and 14767 triangles, whereas for Ultrasound, a shape-constrained deformable model may be used having 2411 vertices and 5126 triangles and therefore having a coarser structure compared to the MR model.

(30) In general, the segmentation may result in the shape of the organ of interest being represented as a point distribution model (PDM). The PDM or mesh of the first image modality may comprise M vertices while the mesh of the second image modality may comprise N vertices. Each vertex may represent a three-dimensional vector describing a position in space and may be referred to as x.sub.v.sup.i, with i being an index indicating the patient data set (i∈{1, . . . , S}) and v giving the vertex number of the PDM (v∈{1, . . . , M}) or (v∈{1, . . . , N}). The meshes of the first and second imaging modality may then be defined as:

(31) x.sup.i=(x.sub.1.sup.ix.sub.2.sup.i . . . x.sub.M.sup.i).sup.T i=1, . . . , S; referring to the first imaging modality

(32) y.sup.i=(y.sub.1.sup.iy.sub.2.sup.i . . . y.sub.N.sup.i).sup.T i=1, . . . , S; referring to the second imaging modality

(33) As a first processing step, a patient j∈i=1, . . . , S with a typical organ shape may be selected and a rigid point-based registration may be performed:
x.sup.j=Ry.sup.j+T

(34) to register the mesh of the second imaging modality y.sup.j to the first imaging modality x.sup.j resulting in a registered mesh y.sub.reg.sup.j. R is the rotation matrix and T the translation vector. A transformation that involved scaling may not be needed since the size difference may be modeled by the shape model of differences, as further described in the following.

(35) In the next step, the remaining meshes x.sub.i(i≠j) may be aligned to the selected patient x.sup.j and the remaining meshes y.sub.i(i≠j) may be aligned to the registered reference mesh y.sub.reg.sup.j. Such alignment may involve a rigid point-based registration (rotation and translation) resulting in x.sub.reg.sup.i=(x.sub.1,reg.sup.ix.sub.2,reg.sup.i . . . x.sub.M,reg.sup.i).sup.T and y.sub.reg.sup.i=(y.sub.1,reg.sup.iy.sub.2,reg.sup.i . . . y.sub.N,reg.sup.i).sup.T.

(36) Next, the mean meshes of both modalities may be computed as followed:

(37) $\stackrel{\_}{x}=\frac{1}{S}\underset{i=1}{\overset{S}{.Math.}}\left(\begin{array}{c}{x}_{1,\mathrm{reg}}^i\\ \mathrm{.Math.}\\ x_{M,\mathrm{reg}}^i\end{array}\right)\mathrm{and}\stackrel{\_}{y}=\frac{1}{S}\underset{i=1}{\overset{S}{.Math.}}\left(\begin{array}{c}{y}_{1,\mathrm{reg}}^i\\ \mathrm{.Math.}\\ y_{N,\mathrm{reg}}^i\end{array}\right)$

(38) and an eigenvalue analysis of matrix AA.sup.t may be performed with:

(39) $A=\left(\begin{array}{ccc}{x}_{1,\mathrm{reg}}^i-{\stackrel{\_}{x}}_1& & x_{1,\mathrm{reg}}^S-{\stackrel{\_}{x}}_1\\ \mathrm{.Math.}& \mathrm{.Math.}& \mathrm{.Math.}\\ x_{M,\mathrm{reg}}^i-{\stackrel{\_}{x}}_M& & x_{M,\mathrm{reg}}^S-{\stackrel{\_}{x}}_M\\ y_{1,\mathrm{reg}}^i-{\stackrel{\_}{y}}_1& & y_{1,\mathrm{reg}}^S-{\stackrel{\_}{y}}_1\\ \mathrm{.Math.}& \mathrm{.Math.}& \mathrm{.Math.}\\ y_{N,\mathrm{reg}}^i-{\stackrel{\_}{y}}_M& & y_{N,\mathrm{reg}}^S-{\stackrel{\_}{y}}_M\end{array}\right)$

(40) The shape of the organ-of-interest may now be approximated according to:

(41) $\left(\begin{array}{c}x\\ y\end{array}\right)\approx \left(\begin{array}{c}\stackrel{\_}{x}\\ \stackrel{\_}{y}\end{array}\right)+\underset{k=1}{\overset{p}{.Math.}}{w}_k\left(\begin{array}{c}{\mu }_k\\ v_k\end{array}\right)$

(42) where k=1, . . . , p refer to the p greatest eigenvectors and μ.sub.k and v.sub.k refer to the corresponding normalized eigenvectors of the first and second image modality, respectively. The number p of eigenvalues was determined by p=S−1 and w.sub.k are weights.

(43) The shape difference between the two imaging modalities may then calculated as:

(44) $y-x=\left(\stackrel{\_}{y}+\underset{k=1}{\overset{p}{.Math.}}{w}_k{v}_k\right)-\left(\stackrel{\_}{x}+\underset{k=1}{\overset{p}{.Math.}}{w}_k{\mu }_k\right)=\stackrel{\_}{y}-\stackrel{\_}{x}+\underset{k=1}{\overset{p}{.Math.}}{w}_k\left(v_k-{\mu }_k\right)$

(45) The above steps may be performed by the system of FIG. 1. Having done so, statistical data may be generated which enables the system of FIG. 2 and similar systems to generate a segmentation of the anatomical structure as it may appear in an image acquired by a second imaging modality based on the statistical data and a first segmentation which is obtained from an image acquired by the first imaging modality. For example, the statistical data may comprise the Eigenvectors of the matrix AA.sup.t that belong to the p greatest eigenvectors and the mean meshes x and y. Various alternatives of generating such statistical data are within reach of the skilled person based on this description.

(46) The system of FIG. 2 or similar system may then perform the following steps. Given a non-training image acquired by the first imaging modality, the organ may be segmented using, e.g., the previously described model-based segmentation. This may result in a segmented organ of interest for the first image modality having M vertices:
{tilde over (x)}=({tilde over (x)}.sub.1{tilde over (x)}.sub.2 . . . {tilde over (x)}.sub.M).sup.T

(47) In a first step, the given mesh may be registered to the mean mesh of the first imaging modality {tilde over (x)} leading to a registered mesh {tilde over (x)}.sub.reg. The shape of the organ may be approximated by using a weighted sum of the eigenvectors and the mean mesh:

(48) $\tilde{x}\approx \stackrel{\_}{x}+\underset{k=1}{\overset{p}{.Math.}}{\tilde{w}}_k{\mu }_k$

(49) The weights {tilde over (w)}.sub.k which may provide the best approximation of the new mesh may be calculated based on:

(50) $0\begin{array}{c}!\\ =\end{array}\tilde{x}-\left(\tilde{x}+\underset{k=1}{\overset{p}{.Math.}}{\tilde{w}}_k{\mu }_k\right)$

(51) which minimizes the difference between the approximated mesh and the original mesh. Here, the “!” over “=” symbol is used to denote that the equation should be zero. With X={tilde over (x)}−x and M=(μ.sub.1 μ.sub.2 . . . μ.sub.p), this may be rewritten as:
X=M{tilde over (w)}

(52) The weighting factors {tilde over (w)}=({tilde over (w)}.sub.1 {tilde over (w)}.sub.2 . . . {tilde over (w)}.sub.p).sup.T of this overdetermined system may be determined by applying a QR decomposition, e.g., as described in the handbook “Matrix computations” 3rd ed., The Johns Hopkins University Press, 1983.

(53) By reformulating the earlier described shape difference y−x, the shape of the organ as it would have been observed in the second imaging modality may now be approximated by:

(54) $0\begin{array}{c}!\\ =\end{array}\tilde{x}-\left(\stackrel{\_}{x}+\underset{k=1}{\overset{p}{.Math.}}{\tilde{w}}_k{\mu }_k\right)$

(55) It is noted that if the numbers of vertices of the mesh of the first imaging modality and the mesh of the second image modality are not the same (M≠N), the numbers may be adapted before this equation can be applied, e.g., by using a mapping that maps the vertices of the left ventricle of the US mesh to their corresponding vertices in the MR mesh.

(56) It will be appreciated that various alternative ways of generating and using the statistical data are conceived and are within reach of the skilled person based on this description. In particular, various other statistical analysis techniques may be instead of principle component analysis (PCA), including but not limited to PCA with Orthomax, sparse PCA, Independent component analysis, Maximum autocorrelation factor (MAF) analysis, Kernel PCA, etc. Moreover, alternatively to linear eigenvalue decomposition, also non-linear decompositions may be used. For example, any suitable non-linear eigenvalue decomposition may be used as introduced in section 3.2 of “Statistical shape models for 3D medical image segmentation: A review” by Heimann et al, Medical Image Analysis 13 (2009), pp. 543-563 which is hereby incorporated by reference with respect to the generation of a mean shape.

(57) Exemplary use cases include, but are not limited, to the following.

(58) Volume measurements such as the heart chamber volume or the volume of brain structures may be computed from the volume enclosed by the corresponding mesh structure after adapting the mesh model to an image. Similarly, diameter measurements, etc., may be derived from the mesh structure. The described approach allows to approximately compute the measurement as it would have been observed in the second imaging modality or to provide information about the variation of the measurement between different imaging modalities. This information may be of help in, e.g., follow-up studies to assess disease progression or treatment outcome when different imaging modalities have been used.

(59) For example, clinical guidelines and recommendations for clinical measurements, such as the threshold of the fraction ejection of the left ventricle, are usually defined for a specific imaging modality, but are used in clinical practice independently of the imaging modality as it may be laborious and expensive to use several imaging modalities for diagnoses or clinical treatment planning. This may result in inaccurate or erroneous measurements since the measured quantity may vary, e.g., in size or shape, across different imaging modalities. The described approach allows to approximately compute the measurement as it would have been observed in the second imaging modality.

(60) For interventional guidance, pre-operatively acquired models of one imaging modality are often overlaid onto interventional images of a second imaging modality. The described approach allows to compensate shape differences between both modalities and to generate an intra-operative overlay over the interventional images with improved accuracy.

(61) FIG. 5 shows a computer-implemented method 400 for generating statistical data. It is noted that the method 400 may, but does not need to, correspond to an operation of the system 100 as described with reference to FIG. 1 and others.

(62) The method 400 comprises, in an operation titled “ACCESSING FIRST SET AND SECOND SET OF IMAGES”, accessing 410 a first set and a second set of images of a type of anatomical structure, wherein the first set of images is acquired by a first imaging modality and the second set of images is acquired by a second imaging modality. The method 400 further comprises, in an operation titled “SEGMENTING FIRST SET OF IMAGES”, segmenting 420 individual images of the first set of images to obtain a first set of segmentations of the type of anatomical structure. The method 400 further comprises, in an operation titled “SEGMENTING SECOND SET OF IMAGES”, segmenting 430 individual images of the second set of images to obtain a second set of segmentations of the type of anatomical structure. The method 400 further comprises, in an operation titled “GENERATING STATISTICAL DATA”, based on the first set of segmentations and the second set of segmentations, generating 440 statistical data which is indicative of a difference in shape of the type of anatomical structure between a) the images acquired by the first imaging modality and b) the images acquired by the second imaging modality. For example, generating 440 may comprise generating statistical data which is indicative of a difference in physical shape of the underlying (e.g. actual or real) anatomical structure imaged by the first and second modalities. As described above, the difference in shape represents differences in the shape of the physical anatomy (e.g. the anatomical structure being imaged) which may be due to, for example, the first image modality imaging different parts of the anatomical structure more or less clearly than the second image modality.

(63) FIG. 6 shows a computer-implemented method 500 for segmentation based on the statistical data. It is noted that the method 500 may, but does not need to, correspond to an operation of the system 200 as described with reference to FIG. 2 and others.

(64) The method 500 comprises, in an operation titled “ACCESSING IMAGE OF PATIENT”, accessing 510 an image of an anatomical structure of a patient, wherein the image is acquired by a first imaging modality. The method 500 further comprises, in an operation titled “ACCESSING STATISTICAL DATA”, accessing 520 statistical data indicative of a difference in shape of the type of anatomical structure between a) images acquired by the first imaging modality and b) images acquired by a second imaging modality. Operation 520 may comprise accessing statistical data generated using the method 400 as described above. The method 500 further comprises, in an operation titled “SEGMENTING THE IMAGE”, segmenting 530 the image to obtain a first segmentation of the anatomical structure of the patient. The method 500 further comprises, in an operation titled “GENERATING SECOND SEGMENTATION”, based on the first segmentation and the statistical data, generating 540 a second segmentation of the anatomical structure which represents an estimate of the shape of the anatomical structure of the patient in an image acquired by the second imaging modality.

(65) It will be appreciated that the operations of FIGS. 5 and 6 may be performed in any suitable order, e.g., consecutively, simultaneously, or a combination thereof, subject to, where applicable, a particular order being necessitated, e.g., by input/output relations.

(66) Each method may be implemented on a computer as a computer implemented method, as dedicated hardware, or as a combination of both. As also illustrated in FIG. 7, instructions for the computer, e.g., executable code, may be stored on a computer readable medium 600, e.g., in the form of a series 610 of machine readable physical marks and/or as a series of elements having different electrical, e.g., magnetic, or optical properties or values. The executable code may be stored in a transitory or non-transitory manner. Examples of computer readable mediums include memory devices, optical storage devices, integrated circuits, servers, online software, etc. FIG. 7 shows an optical disc 600.

(67) Examples, embodiments or optional features, whether indicated as non-limiting or not, are not to be understood as limiting the invention as claimed.

(68) It will be appreciated that the invention also applies to computer programs, particularly computer programs on or in a carrier, adapted to put the invention into practice. The program may be in the form of a source code, an object code, a code intermediate source and an object code such as in a partially compiled form, or in any other form suitable for use in the implementation of the method according to the invention. It will also be appreciated that such a program may have many different architectural designs. For example, a program code implementing the functionality of the method or system according to the invention may be sub-divided into one or more sub-routines. Many different ways of distributing the functionality among these sub-routines will be apparent to the skilled person. The sub-routines may be stored together in one executable file to form a self-contained program. Such an executable file may comprise computer-executable instructions, for example, processor instructions and/or interpreter instructions (e.g. Java interpreter instructions). Alternatively, one or more or all of the sub-routines may be stored in at least one external library file and linked with a main program either statically or dynamically, e.g. at run-time. The main program contains at least one call to at least one of the sub-routines. The sub-routines may also comprise function calls to each other. An embodiment relating to a computer program product comprises computer-executable instructions corresponding to each processing stage of at least one of the methods set forth herein. These instructions may be sub-divided into sub-routines and/or stored in one or more files that may be linked statically or dynamically. Another embodiment relating to a computer program product comprises computer-executable instructions corresponding to each means of at least one of the systems and/or products set forth herein. These instructions may be sub-divided into sub-routines and/or stored in one or more files that may be linked statically or dynamically.

(69) The carrier of a computer program may be any entity or device capable of carrying the program. For example, the carrier may include a data storage, such as a ROM, for example, a CD ROM or a semiconductor ROM, or a magnetic recording medium, for example, a hard disk. Furthermore, the carrier may be a transmissible carrier such as an electric or optical signal, which may be conveyed via electric or optical cable or by radio or other means. When the program is embodied in such a signal, the carrier may be constituted by such a cable or other device or means. Alternatively, the carrier may be an integrated circuit in which the program is embedded, the integrated circuit being adapted to perform, or used in the performance of, the relevant method.

(70) It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design many alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. Use of the verb “comprise” and its conjugations does not exclude the presence of elements or stages other than those stated in a claim. The article “a” or “an” preceding an element does not exclude the presence of a plurality of such elements. The invention may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the device claim enumerating several means, several of these means may be embodied by one and the same item of hardware. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.