SLICE ALIGNMENT FOR SHORT AXIS CARDIAC MR CINE SLICE STACKS

Abstract

Slice alignment approaches are described for short axis cardiac magnetic resonance cine slice stacks, which do not require additional scans, such as long axis scans or full 3D scans, and which are able to deal with cardiac structures having complex shapes. Both approaches do not need contours to follow a quadratic curvature function, and are well suitable for the purpose of obtaining a segmentation of a cardiac structure using a deformable surface model. Namely, such a deformable surface model is unable, but also not desired, to fully adapt to the ‘zig-zag’-shaped pattern in the boundary of the cardiac structure due to the slice misalignment. Having removed or reduced the misalignment between image slices, such a deformable surface model may better adapt to the cardiac structure in the image data and 10 thereby provide a better segmentation of the cardiac structure.

Claims

1. A system for slice alignment of short axis cardiac magnetic resonance cine slice stacks, the system comprising: an input interface for accessing image data of an input set of image slices acquired using a short axis cardiac magnetic resonance cine protocol; a processor subsystem configured to: access trained model data defining a machine trained model, wherein the machine trained model is trained on training data comprising image data of a training set of image slices acquired using a short axis cardiac magnetic resonance cine protocol, wherein one or more adjacent image slices are purposefully mutually misaligned by being mutually shifted with respect to each other using known shift values, wherein the training data further comprises the shift values and wherein the machine trained model is trained to predict the shift vales based on the image data of sets of adjacent image slices; apply the machine trained model to sets of adjacent image slices of the input set of image slices, thereby obtaining at least one shift value for at least one of the image slices of the sets of adjacent image slices; and shift said image slice based on the shift value.

2. The system according to claim 1, wherein the processor subsystem is configured to: apply the machine trained model to the sets of adjacent image slices of the input set of image slices to obtain a series of shift values; remove an offset or linear trend from the series of shift values; shift respective image slices of the sets of adjacent image slices based on respective shift values of the series of shift values.

3. The system according to claim 1, wherein the machine trained model is configured and trained to use as further input positional information which is indicative of a position of respective sets of adjacent image slices relative to a cardiac structure which is shown in the input set of image slices.

4. The system according to claim 1, wherein the machine trained model is configured and trained to use as further input angular information which is indicative of an orientation of a cardiac structure which is shown in the input set of image slices, relative to a coordinate system associated with the input set of image slices.

5. The system according to claim 3, wherein the processor subsystem configured to obtain at least one of the positional information and the angular information by segmenting the cardiac structure in the input set of image slices, for example by applying a deformable surface model to the image data.

6. The system according to claim 5, wherein the processor subsystem is configured to mask a part of the image data of the input set of image slices which does not belong to the cardiac structure before applying the machine trained model to the sets of adjacent image slices of the input set of image slices.

7. The system according to claim 1, wherein the input set of image slices is a first set of image slices, wherein the input interface is configured for accessing image data of a second set of image slices acquired during a different cardiac phase than the first set of image slices, and wherein the machine trained model is configured and trained to use spatially corresponding samples of the first set of image slices and the second set of image slices as joint input.

8. A computer-readable medium comprising: transitory or non-transitory data defining a machine trained model, wherein the machine trained model is configured and trained to be applied to a set of adjacent image slices of a set of image slices acquired using a short axis cardiac magnetic resonance cine protocol, wherein the machine trained model is trained to output a shift value if the set of adjacent image slices is mutually misaligned for reducing mutual misalignment.

9. A system for slice alignment of short axis cardiac magnetic resonance cine slice stacks, the system comprising: an input interface for accessing image data representing a set of image slices acquired using a short axis cardiac magnetic resonance cine protocol; a processor subsystem configured to: access surface model data defining a deformable surface model for segmenting a cardiac structure in short axis cardiac MR cine slice stacks, wherein deformability of the surface model is constrained by shape regularization; adapt the surface model to the cardiac structure by detecting boundary points of the cardiac structure in the image data and deforming the surface model towards the boundary points to obtain an adapted surface model which is adapted in shape to the cardiac structure in the image data; and shift at least one image slice relative to other image slices so that the boundary points in the image slice obtain an improved match with a cross-sectional representation of the surface model in the respective image slice.

10. The system according to claim 9, wherein the processor subsystem is configured to deform the surface model towards the boundary points of the cardiac structure based on a cost function penalizing a distance of the surface model to the boundary points, and to shift the at least one image slice relative to the other image slices so that the match is improved according to the cost function.

11. The system according to claim 9, wherein the processor subsystem is configured for iterative slice alignment by repeating said adapting of the surface model and said shifting of the at least one image slice at least twice.

12. The system according to claim 9, wherein the processor subsystem is configured to: after adapting the surface model, obtain a series of shift values for respective image slices of the set of image slices to obtain the improved match with the sectional representation of the surface model in the respective image slices; remove an offset or linear trend from the series of shift values; shift the respective image slices based on the respective shift values of the series of shift values.

13. A computer-implemented method for slice alignment of short axis cardiac magnetic resonance cine slice stacks, comprising: accessing image data of an input set of image slices acquired using a short axis cardiac magnetic resonance cine protocol; accessing trained model data defining a machine trained model, wherein the machine trained model is trained on training data comprising image data of a training set of image slices acquired using a short axis cardiac magnetic resonance cine protocol, wherein one or more adjacent image slices are purposefully mutually misaligned, by being mutually shifted with respect to each other using known shift values, wherein the training data further comprises the shift values and wherein the machine trained model is trained to predict the shift vales based on the image data of sets of adjacent image slices; applying the machine trained model to sets of adjacent image slices of the input set of image slices, thereby obtaining at least one shift value for at least one of the image slices of the sets of adjacent image slices; and shifting said image slice based on the shift value.

14. A computer-implemented method for slice alignment of short axis cardiac magnetic resonance cine slice stacks, comprising: accessing image data representing a set of image slices acquired using a short axis cardiac magnetic resonance cine protocol; accessing surface model data defining a deformable surface model for segmenting a cardiac structure in short axis cardiac MR cine slice stacks, wherein deformability of the surface model is constrained by shape regularization; adapting the surface model to the cardiac structure by detecting boundary points of the cardiac structure in the image data and deforming the surface model towards the boundary points to obtain an adapted surface model which is adapted in shape to the cardiac structure in the image data; and shifting least one image slice relative to other image slices so that the boundary points in the image slice obtain an improved match with a cross-sectional representation of the surface model in the respective image slice.

15. A computer-readable medium comprising transitory or non-transitory data representing a computer program, the computer program comprising instructions stored on a non-transitory computer readable medium for causing a processor system to perform the method according to claim 13.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0055] These and other aspects of the invention will be apparent from and elucidated further with reference to the embodiments described by way of example in the following description and with reference to the accompanying drawings, in which

[0056] FIG. 1 shows a system for slice alignment of short axis cardiac magnetic resonance cine slice stacks, which in one embodiment may use a machine trained model and, in another embodiment, a deformable surface model;

[0057] FIG. 2 shows a system for training a machine trainable model for slice alignment of short axis cardiac magnetic resonance cine slice stacks;

[0058] FIG. 3 shows a cross-section of a misaligned slice stack, showing boundary points of a cardiac structure and an applied deformable surface model;

[0059] FIG. 4 shows the slice stack of FIG. 3 after shifting several slices to bring the boundary points into alignment with the applied deformable surface model;

[0060] FIG. 5 shows a set of adjacent slices and sampling grids indicating which intensity values are used as input for the machine trained model;

[0061] FIG. 6 shows adjacent slices of two slice stacks acquired during different cardiac phases, in which sampling grids are defined in both slice stacks;

[0062] FIG. 7 shows an image slice of a cardiac structure and a cross-section of the applied deformable surface model which is shown to be misaligned with the boundary of the cardiac structure due to a misalignment between image slices;

[0063] FIG. 8 shows the slice stack of FIG. 7 after shifting the image slice;

[0064] FIG. 9 shows a computer-implemented method for slice alignment using a machine trained model;

[0065] FIG. 10 shows a computer-implemented method for slice alignment using a deformable surface model; and

[0066] FIG. 11 shows a computer-readable medium comprising data.

[0067] It should be noted that the figures are purely diagrammatic and not drawn to scale. In the figures, elements which correspond to elements already described may have the same reference numerals.

LIST OF REFERENCE NUMBERS

[0068] The following list of reference numbers is provided for facilitating the interpretation of the drawings and shall not be construed as limiting the claims. [0069] 020, 022 data storage [0070] 030, 032 short axis cardiac MR cine slice stack [0071] 040 shift values [0072] 050 trained model data [0073] 060 surface model data [0074] 100 system for slice alignment [0075] 120 input interface [0076] 122, 124 data communication [0077] 140 processor subsystem [0078] 160 communication interface [0079] 200 system for training model for slice alignment [0080] 220 input interface [0081] 222, 224 data communication [0082] 240 processor subsystem [0083] 260 communication interface [0084] 300 misaligned short axis cardiac MR cine slice stack [0085] 302 aligned short axis cardiac MR cine slice stack [0086] 310 part of cardiac structure [0087] 320 wall of cardiac structure [0088] 360, 362 surface of deformable surface model [0089] S1-S3 shift applied to image slice [0090] 400 set of adjacent image slices from a given cardiac phase [0091] 402 set of adjacent image slices from a further cardiac phase [0092] 410-412 image slices [0093] 420-424 sampling grid defining input to machine trained model [0094] 500 image slice before slice alignment [0095] 502 image slice after slice alignment [0096] 560 surface of deformable surface model [0097] 600 method for slice alignment using machine trained model [0098] 610 accessing image data of slice stack [0099] 620 accessing data of machine trained model [0100] 630 applying machine trained model to adjacent slices [0101] 640 shifting image slice(s) [0102] 700 method for slice alignment using surface model [0103] 710 accessing image data of slice stack [0104] 720 accessing data of deformable surface model [0105] 730 adapting the surface model to cardiac structure [0106] 740 shifting image slice(s) [0107] 750 repeated iterations [0108] 800 computer-readable medium [0109] 810 non-transitory data

DETAILED DESCRIPTION OF EMBODIMENTS

[0110] FIG. 1 shows a system 100 for slice alignment of short axis cardiac magnetic resonance cine slice stacks, which in one embodiment may use a machine trained model and, in another embodiment, a deformable surface model to perform the slice alignment. The system 100 is shown to comprise an input interface 120 for accessing image data 030 of an input set of image slices acquired using a short axis cardiac magnetic resonance cine protocol. For example, as also shown in FIG. 1, the input interface 120 may provide data access 122 to an external data storage 020 which may comprise said image data 030. Alternatively, the input interface 120 may provide data access to an internal data storage which is part of the system 100. Alternatively, the image data 030 may be accessed via a network. In general, the input interface 120 may take various forms, such as a network interface to a local or wide area network, e.g., the Internet, a storage interface to an internal or external data storage, etc. The data storage 020 may take any known and suitable form.

[0111] The data storage is further shown to comprise trained model data 050 and surface model data 060 which are further explained in the following. Depending on the embodiment, the data storage may comprise one or both types of data 050, 060. In some embodiments, the image data 030, the trained model data 050 and the surface model data 060 may each be accessed from a different data storage.

[0112] The system 100 is further shown to comprise a processor subsystem 140 which may internally communicate with the input interface 120 via data communication 124, and as an optional component, a communication interface 160. As also shown in FIG. 1, the communication interface 160 may be an external communication interface such as a network interface to a local or wide area network, e.g., the Internet. In some embodiments, the input interface 120 may be the same interface as the communication interface 160. In other embodiments, the communication interface 160 may be a network interface via which data is received and the input interface 120 may be a data storage interface via which data is stored.

[0113] In one embodiment, the processor subsystem 140 may be configured to access the trained model data 050 which defines a machine trained model which is trained on training data comprising image data of a training set of image slices acquired using a short axis cardiac magnetic resonance cine protocol. Such a machine trained model may for example be obtained from the system described with reference to FIG. 2. The processor subsystem 140 may be configured to apply the machine trained model to sets of adjacent image slices of the input set of image slices, thereby obtaining at least one shift value for at least one of the image slices of the sets of adjacent image slices, and to shift said image slice based on the shift value. Various details and aspects of this embodiment, including optional aspects thereof, will be further elucidated with reference to, inter alia, FIGS. 5 and 6.

[0114] In another embodiment, the processor subsystem 140 may be configured to access the surface model data 060 defining a deformable surface model for segmenting a cardiac structure in short axis cardiac MR cine slice stacks. The deformability of the surface model may be constrained by shape regularization. The processor subsystem 140 may be configured to adapt the surface model to the cardiac structure by detecting boundary points of the cardiac structure in the image data and deforming the surface model towards the boundary points to obtain an adapted surface model which is adapted in shape to the cardiac structure in the image data, and to shift at least one image slice relative to other image slices so that the boundary points in the image slice obtain an improved match with a cross-sectional representation of the surface model in the respective image slice. Various details and aspects of this embodiment, including optional aspects thereof, will be further elucidated in this specification with reference to FIGS. 3 and 4.

[0115] In general, the system 100 may be embodied as, or in, a single device or apparatus, such as a workstation, e.g., laptop or desktop-based, or a server. The device or apparatus may comprise one or more microprocessors which execute appropriate software. For example, the processor subsystem may be embodied by a single Central Processing Unit (CPU), but also by a combination or system of such CPUs and/or other types of processing units. The software may have been downloaded and/or stored in a corresponding memory, e.g., a volatile memory such as RAM or a non-volatile memory such as Flash. Alternatively, the functional units of the system, e.g., the input interface and the processor subsystem, may be implemented in the device or apparatus in the form of programmable logic, e.g., as a Field-Programmable Gate Array (FPGA). In general, each functional unit of the system may be implemented in the form of a circuit. It is noted that the system 100 may also be implemented in a distributed manner, e.g., involving different devices or apparatuses, such as distributed servers, e.g., in the form of cloud computing.

[0116] FIG. 2 shows a system 200 for training a machine trainable model for slice alignment of short axis cardiac magnetic resonance cine slice stacks. The system 200 is shown to comprise an input interface 220 and a processor subsystem 240 which communicate via data communication 224. The system 200 is further shown to comprise a communication interface 260. The input interface 220 is shown to provide access to a data storage 022 via data communication 222. The thus-far described components of the system 200 may correspond in general type to the respective components of the system 100 of FIG. 1. In particular, the embodiment options described in the previous paragraph may also apply to the system 200.

[0117] However, unlike the system 100 of FIG. 1, the processor subsystem 240 of the system 200 is configured to train the machine trainable model, namely using training data which comprises image data 032 of a training set of image slices acquired using a short axis cardiac magnetic resonance cine protocol and in which one or more adjacent image slices are mutually misaligned. In addition, the training data used for the training may comprise shift values 040 for reducing said mutual misalignment by shifting one or more of the image slices. The processor subsystem 240 may be configured to train the machine trainable model using the training data to obtain a machine trained model, and in particular to obtain trained model data 050 representing the machine trained model. The machine trainable model may be of any suitable type, such as a neural network, for example a deep neural network (DNN) comprising one or more convolutional layers (Convolutional DNN). Such a neural network may be trained using any suitable machine learning technique.

[0118] FIG. 3 shows a cross-section of a misaligned slice stack 300 which may be obtained by acquisition using a short axis cardiac magnetic resonance cine protocol. It can be seen that in such slice stacks, the boundary of a (part of a) cardiac structure 310 may be misaligned in a ‘zig-zag’ pattern between the individual slices, which is visible in FIG. 3 by the wall 320 of the cardiac structure 310, being in this example the myocardial wall, following this ‘zig-zag’ pattern. Such misalignment may hinder the adaptation of a deformable surface model to the image data, and thereby hinder the segmentation of the cardiac structure 310. Namely, such a deformable surface model may, when adapted to the image data, follow the general boundary of the cardiac structure 310 but may be prevented from exactly following the irregular ‘zig-zag’ misalignment pattern due to shape regularization. FIG. 3 indeed illustrates that surfaces 360, 362 of the deformable surface model may follow the general boundary of the cardiac structure 310 but may in individual slices often be misaligned with the wall 320 due to the aforementioned ‘zig-zag’ misalignment pattern. At the same time, the ‘zig-zag’ pattern may obfuscate finer anatomical details of the cardiac structure 310, and thereby prevent or hinder the further adaptation of the deformable surface model to such finer anatomical details.

[0119] The system 100 described with reference to FIG. 1 may in one embodiment perform slice alignment by adapting a deformable surface model to the boundary of the cardiac structure, e.g., as illustrated in FIG. 3, and then shift individual slices of the slice stack so that the boundary points contained in a respective slice better match the cross-section of the thus-far adapted deformable surface model in the slice, e.g., as illustrated in FIG. 4 by the arrows labelled S.sub.1-S.sub.3 denoting the direction in which a respective slice is shifted. It is noted that while FIGS. 3 and 4 do not explicitly show such boundary points, these may be detected in a manner as known per se and may indicate to the system 100 where the boundary of the cardiac structure in each slice lies, being here the wall 320. Effectively, in this step of the slice alignment, individual slices may be adapted to the deformable surface model (by appropriate shifting) instead of the other way around. The overall slice alignment process may be performed iteratively, in that the deformable surface model may be adapted to the slice stack, individual slices may then be adapted to the deformable surface model (by appropriate shifting), after which the deformable surface model may be again adapted to the slice stack, etc. This process may be repeated, e.g., a fixed-number of times or until a convergence criterion is reached.

[0120] With continued reference to FIGS. 3 and 4, it is noted that the applying of a deformable surface model to the image data for segmentation purposes is also known as model-based image segmentation. Model-based image segmentation typically uses a triangulated surface mesh, and may apply the surface mesh to the image data by iteratively detecting the boundary points in the image, for example per triangle and by searching for boundaries along the triangle normal, and subsequently adapting the surface mesh to the detected boundaries. A shape model may regularize the mesh deformations. The surface model-based slice alignment may be based on the observation that a surface model may roughly interpolate through the original slice stack. Due to the shape model regularization, an exact fit to the ‘zig-zag’ boundary of the cardiac structure may not be achieved. The approach shown in FIGS. 3 and 4 may add a new step to this iterative adaptation. Namely, after the surface mesh has achieved an intermediate fit to the slice stack, the adaptation strategy may be changed: rather than attracting the triangles of the surface mesh to the detected boundary points, an individual slice may be shifted with its boundary points towards the surface mesh. To improve the overall results, mesh adaptation and slice shifts may be alternated in a configurable scheme. For example, after a start condition is satisfied, every second iteration may be used to shift the slices, while every other iteration may be used for further mesh adaptation. This may improve the slice alignment but also the segmentation of the cardiac structure, as better aligned slices may lead to improved boundary detection (due to interpolation between slices during boundary detection) and thereby to better mesh adaptation.

[0121] In a specific embodiment, the boundary detection for a triangle may be limited to searching for boundary points in the same slice, e.g., to avoid shifting slices in a plane that is far ‘above or below’ the corresponding triangle. The difference vector pointing from the triangle to the boundary may also be required to be reasonably parallel to the slice, e.g., as quantified by a corresponding metric, rather than pointing mostly along the stacking direction of the slices. This may improve the numerical stability of the slice alignment. Also, the boundary detection may be configured to reject doubtful boundaries, so that outliers may be filtered out.

[0122] As also referenced earlier, FIG. 4 shows the slice stack of FIG. 3 after shifting several slices into alignment with the applied deformable surface model.

[0123] It is noted that even though the adapted surface model may generally follow the cardiac structure in the non-aligned slice stack and the estimated slice shifts are expected to roughly sum up to zero and thereby not introduce any tilting of the slice stack, actual slice shift estimates may be imperfect. Estimating and applying slice shifts over many iterations may therefore lead to a slow translational drift and/or tilting (skewing) of the slice stack, and the surface model may follow this drift when iteratively adapting the surface model to the slice stack. To compensate for such drift and/or tilting, an additional slice shift normalization step may be introduced which may comprise the following (described with reference to the x-component of the shift; the y-component may be normalized correspondingly, with x and y referring to the in-slice coordinate system): Let {dx[z] } be the estimated x-displacements for all slices indexed by z. Linearly transformed ‘normalized’ displacements may be defined as {T(dx[z], z)} with T (dx[z], z)=dx[z]+a.Math.z+b where the linear transformation parameters a and b may be estimated such that Σ.sub.Z Σ.sub.Z T(dx[z], z) is minimized, e.g., such that T (dx[z], z) deviates as little as possible from a straight line along the slice normal (in z direction) centred at x=0. This also implies Σ.sub.Z T(dx[z], z)=0, thereby eliminating translational drift. Such normalization may thereby remove an offset or a linear trend from a sequence of shift values which represents the shift values of a sequence of image slices.

[0124] With further reference to the slice alignment using a machine trained model, such as a Convolutional DNN, this approach considers slice shift estimation as a regression task that is solved using machine learning techniques. Here, pairs or n-tuples of successive slices may be input into a machine trained model that infers a relative shift that will bring the slices into an anatomically aligned position based on the image intensity values of the slices, and in some embodiments, based on auxiliary information. To train such a model, a slice stack aligned by the approach described with reference to FIGS. 3 and 4 may be used as reference. For example, such a slice stack may be purposefully misaligned using known shift values, and the purposefully misaligned slice stack may together with the known shift values be used as training data for the machine trainable model to learn to predict the slice shifts.

[0125] FIG. 5 shows a set of adjacent slices 400 and sampling grids 420-422 indicating which intensity values from individual image slices 410-412 are used as input to the machine trained model. For example, the sampling grid may be a pre-defined grid, e.g., using 256×256 sampling points at known intervals such as 1 mm apart. For example, when intensity values of three adjacent slices are input to the machine trained model, the model may be fed with 3×256×256 intensity values. Various other sampling grids may be defined as well, e.g., involving a different number of samples, a different spatial shape, a different number of slices, etc. In some embodiments, differences in intensity values between the slices may be used as input. Intensity values may be normalized, e.g., using a statistic from the complete stack, e.g., mean and standard deviation or percentiles of intensities.

[0126] For the example of pair-wise estimation of slice shifts, the following is noted. For N slices, a global shift may be applied to all slices without changing the relative alignment or mis-alignment. However, the relative positions between the N slices may be defined by N−1 relative shifts. Such relative shifts may be expressed in various ways, e.g., as shift of each slice with respect to a reference slide (e.g., the first or last slice in the slice stack), or as the relative shift between successive slices i and i+1. Overall, one may thereby estimate N−1 relative slice shifts for the N slices. For one of the slices (e.g., the first or last slice), a zero shift may be assigned. However, this may lead to an overall shift of the estimated relative shifts for the other N−1 slices as these may not sum up to zero. Moreover, as also described earlier, slice shift estimates may be imperfect and introduce drift and/or tilting. The earlier-described slice shift normalization step may therefore be applied to the series of shift values obtained by the slice shift estimation to remove bias, e.g., an offset (the aforementioned global shift) or linear trend, from the shift values to minimize the deviation from the z-axis (referring to the axis orthogonal to the in-slice axes).

[0127] In addition to image intensities, various other types of auxiliary information may be used as well as input to the machine trained/trainable model. For example, the anatomical content of the slices may vary from the apex to the “base” (transition from ventricles to atria) and thereby vary quite significantly throughout the slice stack. The machine trained/trainable model may benefit from knowing that ‘more apical’ or ‘more basal’ slices have to be aligned. In addition, since the slices may show the heart in varying rotations, angular information may help as well.

[0128] The following refers to the machine trained/trainable model as a neural network, but also applies to other types of machine trained/trainable models.

[0129] There are various ways to provide such additional positional information to the neural network. In one embodiment, positional information may be provided to the neural network by indicating that a particular slice is at a particular relative position in the slice stack, e.g., by specifying a percentage with 0% being most apical or even below the apex and 100% being most basal or above the ventricles. In another embodiment, a 3D coordinate may be specified per sample point. For example, the DICOM information of the acquisition may be used so that a relative position may be specified with respect to the volume center in so-called ‘patient’ (or ‘world’) coordinates, where ‘z’ is from foot to head, ‘x’ is from right to left, and ‘y’ is from front to back of the patient. Since the heart has a typical orientation within each patient, such DICOM-based positional information may already provide a coarse indication of the anatomical content of a particular slice/sample point. However, both the size and the precise orientation of the heart varies from patient to patient, so that a more specific coordinate system may help. In a more elaborate embodiment, an approximate segmentation of the heart may be used (e.g., with imperfect or absent slice alignment) to define a coordinate system based on cardiac landmarks, e.g., as also used to define 2-, 3- and 4-chamber plus short axis views. Since this may require a complete heart segmentation, an intermediate embodiment may use a coordinate system estimated by a Generalized Hough Transform (GHT) that may be applied with various scales and various rotations of its edge template. The anatomical positional information of a slice or sample point may then be determined in this coordinate system and provided to the neural network as input. Various types of pre-processing may be used as well before training or subsequently using the machine trained model. This may reduce the complexity of the learning problem and incorporate prior knowledge already in the input to the machine trained model. For example, the image data of the slice stack may be resampled in a new (anatomically aligned) coordinate system such that the cardiac anatomy follows a standard orientation in this coordinate system. Rather than sampling the image intensities on a grid aligned with the scan axes, the sampling grid may also be re-oriented in each individual slice using axes associated with anatomical directions. The only additional positional information input to the neural network may be the z-coordinate encoding the anatomical position of the slice.

[0130] Additionally or alternatively, a rough pre-segmentation or localization may be used to mask out irrelevant parts of the image data, such as the rib cage, which may exhibit a different transformation compared to the moving heart.

[0131] Knowledge of the orientation of the anatomy may also be used in recurrent architectures which predict a sequence of translations when the input consists of a sequence of successive (or even single) slices, for example always starting close to the apex and finishing around the basal parts of the heart. Even if there is no correlation between the translations from slice to slice, this may still exploit locational information along the heart's long axis. Namely, an internal state may be updated at each new slice, but information of the previous slices may be retained. Accordingly, the neural network may be aware of the current slice position which may make the output part of the neural network act differently. In this way, typical alignment curves across the population may be implicitly taken into account. Such an approach may also make use of so-called ‘bi-currence’, in which upward (from the apex) and downward (from the basal part) passes may be combined.

[0132] FIG. 6 shows two sets of adjacent image slices 400, 402 which are from two slice stacks which are each acquired during different cardiac phases, and in which sampling grids 420, 424 are defined in both sets of adjacent image slices. Namely, image intensities from different cardiac phases may be used as joint input to the neural network. In the example of FIG. 6, intensity values from two cardiac phases (e.g., ED and ES) may be sampled on the same grid in each cardiac phase, e.g., using sampling grids 420 and 424. As such, each sample point used as input to the neural network may provide intensity values from both cardiac phases.

[0133] FIG. 7 shows an image slice 500 of a cardiac structure and a cross-section of an adapted deformable surface model, and in particular of a surface 560 of the deformable surface model which is shown to misaligned with respect to the boundary of the cardiac structure due to a misalignment between image slices. FIG. 8 shows a result of the slice alignment as described in this specification, in that the image slice of FIG. 7 is shown after shifting the image slice within the slice stack while the deformable surface model is shown after another iteration of adapting the deformable surface model to the slice stack. This may yield an improved alignment of the surface 500 to the boundary of the cardiac structure in the image slice 502.

[0134] In general, the slice alignment as described in this specification may be used in various medical systems and apparatuses, including but not limited to 3D visualization systems, e.g., ‘orthoviewer’ visualization systems which show two and four-chamber multiplanar reformats in addition to the individual slices.

[0135] FIG. 9 shows a flow-diagram of a computer-implemented method 600 for slice alignment using a machine trained model. The method 600 may correspond to an operation of the system 100 of FIG. 1 when configured for slice alignment using a machine trained model. However, this is not a limitation, in that the method 600 may also be performed using another system, apparatus or device.

[0136] The method 600 may comprise, in an operation titled “ACCESSING IMAGE DATA OF SLICE STACK”, accessing 610 image data of an input set of image slices acquired using a short axis cardiac magnetic resonance cine protocol. The method 600 may further comprise, in an operation titled “ACCESSING DATA OF MACHINE TRAINED MODEL”, accessing 620 trained model data defining a machine trained model, wherein the machine trained model is trained on training data comprising image data of a training set of image slices acquired using a short axis cardiac magnetic resonance cine protocol, wherein one or more adjacent image slices are mutually misaligned, and wherein the training data further comprises shift values for reducing said mutual misalignment by shifting one or more of the image slices. The method 600 may further comprise, in an operation titled “APPLYING MACHINE TRAINED MODEL TO ADJACENT SLICES”, applying 630 the machine trained model to sets of adjacent image slices of the input set of image slices, thereby obtaining at least one shift value for at least one of the image slices of the sets of adjacent image slices, and in an operation titled “SHIFTING IMAGE SLICE(S)”, shifting 640 said image slice based on the shift value.

[0137] FIG. 10 shows a flow-diagram of a computer-implemented method 700 for slice alignment using a machine trained model. The method 700 may correspond to an operation of the system 100 of FIG. 1 when configured for slice alignment using a deformable surface model. However, this is not a limitation, in that the method 700 may also be performed using another system, apparatus or device.

[0138] The method 700 may comprise, in an operation titled “ACCESSING IMAGE DATA OF SLICE STACK”, accessing 710 image data representing a set of image slices acquired using a short axis cardiac magnetic resonance cine protocol. The method 700 may further comprise, in an operation titled “ACCESSING DATA OF DEFORMABLE SURFACE MODEL”, accessing 720 surface model data defining a deformable surface model for segmenting a cardiac structure in short axis cardiac MR cine slice stacks, wherein deformability of the surface model is constrained by shape regularization. The method 700 may further comprise, in an operation titled “ADAPTING THE SURFACE MODEL TO CARDIAC STRUCTURE”, adapting 730 the surface model to the cardiac structure by detecting boundary points of the cardiac structure in the image data and deforming the surface model towards the boundary points to obtain an adapted surface model which is adapted in shape to the cardiac structure in the image data. The method 700 may further comprise, in an operation titled “SHIFTING IMAGE SLICE(S)”, shifting 740 at least one image slice relative to other image slices so that the boundary points in the image slice obtain an improved match with a cross-sectional representation of the surface model in the respective image slice. Operations 730, 740 may be repeated in iterations 750.

[0139] It will be appreciated that, in general, the operations of method 600 of FIG. 9 and/or method 700 of FIG. 10 may be performed in any suitable order, e.g., consecutively, simultaneously, or a combination thereof, subject to, where applicable, a particular order being necessitated, e.g., by input/output relations.

[0140] The method(s) may be implemented on a computer as a computer implemented method, as dedicated hardware, or as a combination of both. As also illustrated in FIG. 11, instructions for the computer, e.g., executable code, may be stored on a computer readable medium 800, e.g., in the form of a series 810 of machine-readable physical marks and/or as a series of elements having different electrical, e.g., magnetic, or optical properties or values. The executable code may be stored in a transitory or non-transitory manner. Examples of computer readable mediums include memory devices, optical storage devices, integrated circuits, servers, online software, etc. FIG. 11 shows an optical disc 800. Alternatively, the computer readable medium 800 may comprise transitory or non-transitory data 810 representing the machine trained model as described elsewhere in this specification.

[0141] Examples, embodiments or optional features, whether indicated as non-limiting or not, are not to be understood as limiting the invention as claimed.

[0142] It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design many alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. Use of the verb “comprise” and its conjugations does not exclude the presence of elements or stages other than those stated in a claim. The article “a” or “an” preceding an element does not exclude the presence of a plurality of such elements. Expressions such as “at least one of” when preceding a list or group of elements represent a selection of all or of any subset of elements from the list or group. For example, the expression, “at least one of A, B, and C” should be understood as including only A, only B, only C, both A and B, both A and C, both B and C, or all of A, B, and C. The invention may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the device claim enumerating several means, several of these means may be embodied by one and the same item of hardware. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.