APPARATUS AND METHOD FOR SURGERY PREPARATION

Abstract

An apparatus for surgery preparation. The apparatus includes a tracking device for representing a surgical instrument, a display device for displaying a subject for surgery and one or more processors arranged to receive signals corresponding to the tracking device to determine the position of a virtual surgical instrument following the movement of the tracking device. A model is generated for three-dimensional representation of a tissue corresponding to the subject for surgery, the model having plurality of tissue elements arranged to together represent the tissue. A method for utilizing a tracking device for representing a surgical instrument is also disclosed.

Claims

1. An apparatus for surgery preparation, the apparatus comprising: a tracking device for representing a surgical instrument; a display device for displaying a subject for surgery; one or more processors arranged to receive signals corresponding to the tracking device to determine the position of a virtual surgical instrument following the movement of the tracking device; and one or more memories comprising computer program code, the one or more memories and the computer program code configured to cause the one or more processors to: generate a model for three-dimensional representation of a tissue corresponding to the subject for surgery, the model comprising plurality of tissue elements arranged to together represent the tissue; determine a spatial location for each of the tissue elements with a local/global solver for a state equation of motion for the tissue elements, which local/global solver alternatingly obtains a set of local solutions under one or more constraints for the tissue elements indicating optimization of an elastic potential for the tissue elements and determines the spatial locations which optimize the state equation when the elastic potential corresponds to the local solutions; and based on the position of the virtual surgical instrument with respect to the tissue elements, alter the one or more constraints to indicate that a cut has been made to the tissue with the virtual surgical instrument.

2. The apparatus according to claim 1, wherein a spatial location for a tissue element corresponds to a density distribution and the one or more constraints comprise a constraint for the density distribution.

3. The apparatus according to claim 2, wherein the set of local solutions is obtained under a constraint indicating that a deformation gradient for the tissue elements for indicating elasticity of the tissue substantially corresponds to a reference deformation gradient.

4. The apparatus according to claim 3, wherein obtaining the set of local solutions comprises determining the deformation gradient or an indication thereof, wherein rotation is removed from the reference deformation gradient or the indication thereof.

5. The apparatus according to claim 2, wherein the one or more memories and the computer program code are configured to cause the one or more processors to: determine a neighborhood for a tissue element of the plurality of tissue elements for indicating the group of tissue elements of the plurality of tissue elements that are close enough for interaction with the tissue element; and determine a constraint for the density distribution for the tissue element as a limitation for cumulative density for the neighborhood for providing incompressibility to the tissue.

6. The apparatus according to claim 5, wherein determining the constraint for density distribution comprises determining a scaling factor for scaling the distances between the tissue elements in the neighborhood so that the cumulative density for the neighborhood is limited.

7. The apparatus according to claim 2, wherein the one or more memories and the computer program code are configured to cause the one or more processors to: determine a neighborhood for a tissue element of the plurality of tissue elements for indicating the group of tissue elements of the plurality of tissue elements that are close enough for interaction with the tissue element; and determine the distances between the tissue elements in the neighborhood for determining weighing coefficients for interaction between the tissue elements.

8. The apparatus according to any preceding claim, claim 1 wherein the tracking device comprises an actuator for haptic feedback and the apparatus is arranged to use the model for generating a haptic feedback to the actuator.

9. A method comprising: generating a model for three-dimensional representation of a tissue corresponding to a subject for surgery, the model comprising plurality of tissue elements arranged to together represent the tissue; determining a spatial location for each of the tissue elements with a local/global solver for a state equation of motion for the tissue elements, which local/global solver alternatingly obtains a set of local solutions under one or more constraints for the tissue elements indicating optimization of an elastic potential for the tissue elements and determines the spatial locations which optimize the state equation when the elastic potential corresponds to the local solutions; receiving one or more signals for determining the position of a virtual surgical instrument following the movement of a tracking device; and based on the position of the virtual surgical instrument with respect to the tissue elements, altering the one or more constraints to indicate that a cut has been made to the tissue with the virtual surgical instrument.

10. The method according to claim 9, wherein a spatial location corresponds for a tissue element corresponds to a density distribution and the one or more constraints comprise a constraint for the density distribution.

11. The method according to claim 10, wherein the set of local solutions is obtained under a constraint indicating that a deformation gradient for the tissue elements for indicating elasticity of the tissue substantially corresponds to a reference deformation gradient.

12. The method according to claim 11, wherein obtaining the set of local solutions comprises determining the deformation gradient or an indication thereof, wherein rotation is removed from the reference deformation gradient or the indication thereof.

13. The method according to claim 10, comprising: determining a neighborhood for a tissue element of the plurality of tissue elements for indicating the group of tissue elements of the plurality of tissue elements that are close enough for interaction with the tissue element; and determining a constraint for the density distribution for the tissue element as a limitation for cumulative density for the neighborhood for providing incompressibility to the tissue.

14. The method according to claim 13, wherein determining the constraint for density distribution comprises determining a scaling factor for scaling the distances between the tissue elements in the neighborhood so that the cumulative density for the neighborhood is limited.

15. The method according to claim 10, comprising: determining a neighborhood for a tissue element of the plurality of tissue elements for indicating the group of tissue elements of the plurality of tissue elements that are close enough for interaction with the tissue element; and determining the distances between the tissue elements in the neighborhood for determining weighing coefficients for interaction between the tissue elements.

16. The method according to claim 9, comprising using the model for causing a haptic feedback to be generated.

17. A computer readable storage medium storing a non-transitory computer program product comprising instructions which, when the program is executed by a computer, cause the computer to carry out the method of claim 9.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The accompanying drawings, which are included to provide a further understanding and constitute a part of this specification, illustrate examples and together with the description help to explain the principles of the disclosure. In the drawings:

[0033] FIG. 1 illustrates an apparatus according to an example,

[0034] FIG. 2 illustrates a method according to an example,

[0035] FIG. 3 illustrates a first detailed method according to an example, and

[0036] FIG. 4 illustrates a second detailed method according to an example.

[0037] Like references are used to designate equivalent or at least functionally equivalent parts in the accompanying drawings.

DETAILED DESCRIPTION

[0038] The detailed description provided below in connection with the appended drawings is intended as a description of examples and is not intended to represent the only forms in which the example may be constructed or utilized. However, the same or equivalent functions and structures may be accomplished by different examples.

[0039] FIG. 1 shows an example of an apparatus 100 for surgery preparation. The apparatus 100 comprises a tracking device 110 for representing a surgical instrument. The tracking device 110 may be arranged to move in three dimensions for three-dimensional movement of a virtual surgical instrument 112. The surgical instrument may be a cutting instrument such as a scalpel. The tracking device 110 may comprise a handheld portion for representing the surgical instrument. For example, the tracking device 110 may comprise a pointed instrument such as a stylus for representing the surgical instrument. Alternatively or additionally, the tracking device 110 may comprise a wearable device such as a glove. Optionally, the tracking device 110 may comprise or be coupled to a base 114. This allows some or all of the electronics for the tracking device 110 to be housed in the base 114. The hand-held portion of the tracking device 110 may be coupled to the base 114 wirelessly and/or through an articulated arm. The hand-held portion may also be self-contained allowing the tracking device 110 to be moved freely without rigid connection to a fixed base 114. The tracking device 110 allows performing simufated movement for a surgical instrument and tracking the movement so that the position, and optionally the orientation, for the surgical instrument can be communicated forward.

[0040] The apparatus 100 also comprises a display device 120 for displaying a subject 130 for surgery.

[0041] The apparatus 100 may also be arranged to use the display device to display the virtual surgical instrument 112. The display device 120 may comprise one or more screens such as a computer screen and/or one or more projectors. Alternatively or additionally, the display device 120 may comprise a headset, such as a virtual reality or an augmented reality headset. This can be used to allow an immersive experience for the user of the apparatus 100 preparing for surgery. The apparatus 100 may further comprise one or more speakers 122, which may be used for example for providing instructions and/or sound for surgery.

[0042] The subject 130 for surgery may be, for example, a human body or a part of it, such as a wrist or an ankle. One or more cuts 132, such as incisions, may be displayed on the subject 130 for surgery in response to the virtual surgical instrument 112 cutting the subject 130 for surgery. The apparatus 100 may be arranged to use the display device 120 to simultaneously display the subject 130 for surgery and the virtual surgical instrument 112 so that the person using the apparatus 100 can observe their relative positions. For example, the subject 130 and the virtual surgical instrument may be simultaneously displayed on one screen and/or in the same virtual reality environment. One option is also to display the subject 130 for surgery in an augmented reality environment so that the person using the apparatus 100 can observe the relative position of the subject 130 for surgery and the tracking device 110 representing the surgical instrument.

[0043] The apparatus 110 comprises one or more processors 140, which are arranged to receive signals for determining the position, and optionally the orientation of the virtual surgical instrument 112. The signals may be received from the tracking device 110 directly and/or indirectly. The one or more processors 140 may comprise a central processing unit and/or a graphics processing unit. The one or more processors 140 may be arranged to be coupled to the tracking device 110 through one or more connections 150, which may comprise a wireless and/or a wired connection.

[0044] Similarly, the one or more processors 140 may be arranged to be coupled to the display device 120 through one or more connections 150, which may comprise a wireless and/or a wired connection. The tracking device 110 may comprise one or more actuators for haptic feedback. The one or more actuators may be arranged to provide haptic feedback in response to one or more signals from the one or more processors 140. The haptic feedback may be generated, for example, to a stylus and/or to a wearable device such as a glove.

[0045] The apparatus further comprises one or more memories 142, which comprise computer program code. Together, these may be configured to cause the one or more processors 140 to perform any or all of the methods of the subsequent examples.

[0046] FIG. 2 shows an example of a method 200. The method comprises several parts which may be performed independently from each other and/or in any order. The method can be used for replication of a surgical operation in a computerized environment. Consequently, it can be used for surgery preparation, such as training for surgery. The subject for surgery, including a target tissue to be surgically operated can be, for example, selected from a pre-existing database or generated when the replication of the surgical operation is started. While the subject can be any subject, the method has been found to be particularly applicable for real-time replication of human soft tissue.

[0047] The method 200 comprises receiving 210 one or more signals for determining the position, and optionally the orientation, of a virtual surgical instrument 112. The position, and optionally the orientation, of the virtual surgical instrument 112 may follow movement of a tracking device 110 representing a surgical instrument. Following the movement may include following changes in position and/or orientation.

[0048] The method 200 also comprises generating 220 a model for three-dimensional representation of a tissue corresponding to the subject 130 for surgery. The model comprises a plurality of tissue elements, which are arranged to together represent the tissue. In an advantageous embodiment, the tissue is soft tissue such as fat tissue, skin tissue, muscle tissue and/or ligaments. The tissue elements may be point-like elements in the sense that they have a single well-defined spatial location, which indicates the position of the tissue element. Generating the model may comprise generating one or more relations between the tissue elements, for example one or more relations between two adjacent tissue elements. While the method may also be applied to meshed structure of tissue elements, using a meshless structure of tissue elements provides particular advantages for the surgery application as it allows producing cuts in the tissue without having to redefine the mesh. An initial state may also be determined for the plurality of tissue elements. The initial state may correspond to a state before a cut or any cuts have been made to the tissue.

[0049] The initial state may comprise the spatial locations of the tissue elements and/or any values derivable therefrom.

[0050] The method 200 also comprises determining 230 a spatial location for each of the tissue elements with a local/global solver for a state equation of motion for the tissue elements. The local/global solver is arranged to provide a solution (hereafter “the solution”), which can be an approximate solution, to the state equation of motion for the tissue elements for determination of the spatial locations of the tissue elements. The local/global solver allows providing the solution by alternatingly performing a parallelizable local step and a global step utilizing local solutions obtained from the local step. Similarly, the steps can be alternated so that each global step is followed by a local step. The local/global solver alternatingly, in a local step, obtains a set of local solutions under one or more constraints for the tissue elements indicating optimization, e.g. minimization, of an elastic potential for the tissue elements and, in a global step, determines the spatial locations which optimize the state equation when the elastic potential corresponds to the local solutions. This allows turning the difficult problem of solving the full state equation into a constantly evolving optimization problem, where individual steps may even be analytically solved. The local/global solver may utilize Projective Dynamics for providing the solution. Examples of Projective Dynamics are given, for example in “Bouaziz et al.—Projective Dynamics: Fusing Constraint Projections for Fast Simulation; ACM Transactions on Graphics, 33-4, 2014” (here also “Bouaziz”). While the examples in Bouaziz utilize meshes, it is emphasized that in the present invention point-like tissue elements may also be used allowing particularly advantageous application for surgery. Even then, the local solutions may correspond to projections of Projective Dynamics. The local/global solver may be arranged to alternate between projection and distance optimization, e.g. minimization.

[0051] The state equation of motion for the tissue elements, for which the solution is provided, can be expressed as defining the spatial location and velocity of each tissue element, which spatial location and/or velocity may change over time. This allows the tissue elements to follow general laws of motion, such as the Newton's laws of motion. The state equation can be expressed as a matrix equation. The state equation may be defined in terms of a mass matrix M, which describes how the velocities v of the tissue elements at spatial locations x is affected by forces internal f.sub.int (x) and/or external f.sub.ext to the tissue, for example through the equation

v.sub.i+1=v.sub.i+hM.sup.−1(f.sub.int(x.sub.i+1)+f.sub.ext),   (1)

where M.sup.−1 is the inverse of matrix M and the values at subsequent time steps (i and i+1) are indicated as subscripts, whereas the length of the time step is denoted as h (this notation will also be used below). In general, the mass matrix may be constant so that it remains the same from one step to another. However, the mass matrix may be altered in response tissue elements being added to and/or subtracted from the model for three-dimensional representation of a tissue. As an example, the state of the tissue elements at subsequent time steps may then be related through an equation of the form

M(x.sub.i+1−x.sub.i−hv.sub.i)=h.sup.2(f.sub.int(x.sub.i+1)+f.sub.ext).   (2)

Such an equation can be converted (examples in Bouaziz, particularly in section 3.1, and “Martin et al.—Example-based Elastic Materials; ACM Transactions on Graphics, 30-4, 2011)” into an optimization problem of the type

x.sub.i+1=argmin (v.sub.mom+v.sub.el),   (3)

where the arguments of the minima is determined for the spatial locations x and terms that either correspond to the external forces or are proportional to the mass matrix can be contained in a momentum potential V.sub.mom, whereas terms corresponding to the internal forces of the tissue can be contained in an elastic potential V.sub.el.

[0052] In general, the solution to the state equation of motion for the tissue elements can be provided by solving an optimization problem, where the sum of a momentum potential and an elastic potential is optimized, e.g. minimized, with respect to the spatial locations x of the tissue elements. The elastic potential then corresponds to the internal forces of the tissue. The momentum potential then becomes the remaining part, or the part corresponding to the external forces and the mass matrix. The elastic potential can, in turn, be defined as a sum of a distance measure and an indicator function, the latter of which becomes infinite if a set of constraints is not satisfied. Examples are given in Bouaziz, particularly in section 3.2. While the exact form of the elastic potential may vary, one example is the elastic potential having a quadratic distance measure, examples of which are also given in Bouaziz, particularly in section 3.2. It is noted that while in Bouaziz constant matrices A and B are used in the distance measure, here they can change, for example in response to tissue elements being separated from each other. Correspondingly, the elastic potential may be changed in response to tissue elements being separated from each other, allowing surgical cuts to be efficiently made to the tissue.

[0053] In general, the elastic potential can be determined such that it decouples a distance measure and a constraint manifold. This allows the local/global solver to parallelize the optimization problem for providing the solution to the state equation of motion for the tissue elements. Examples are given in Bouaziz, particularly in section 3.3. The local step may even be arranged to correspond to optimization, e.g. minimization, of the elastic potential alone. The elastic potential can be optimized with respect to auxiliary variables to yield local solutions. In the local step, local solutions can be obtained due to one or more constraints defined for the tissue elements. The one or more constraints may comprise one or more constraints representing interaction between tissue elements, for example an interaction between two tissue elements, which can be expressed as a limitation for the relative spatial location between the two tissue elements. Correspondingly, the one or more constraints may comprise one or more constraints representing tissue elements being connected to each other. This allows a cut in the tissue to be represented by an alteration of the one or more constraints, in particular those representing tissue elements being connected to each other, for example by removing one or more constraints representing tissue elements being connected to each other. The one or more constraints can be defined to allow parallelization of the optimization. For example, such constraints may comprise constraints for defining absolute and/or relative spatial locations of one or more tissue elements for parallelizing the optimization of the elastic potential. For example, optimizing the elastic potential with respect to a single tissue element, whose interaction with other tissue elements is constrained only to a limited number of other tissue elements allows parallelizing the local step, so that the optimization in the local step is markedly simplified from that of the full optimization problem.

[0054] In the global step, the local solutions can then be used to simplify the many-body problem of internal interactions between the tissue elements. In particular, the local/global solver can be arranged to, in the global step, determine the spatial locations which optimize the state equation when the elastic potential corresponds to the local solutions. As an example, this may correspond to solving Equation 3, when the local solutions are inserted into the elastic potential. In some embodiments, for example with a quadratic distance measure of the elastic potential, the local/global solver can be arranged to be able to provide the solution with a single linear solve. It is noted that the local solutions in the elastic potential can make the global optimization problem analytically solvable. The system matrix in the global step may change, for example in response to tissue elements being separated from each other. A solution from the global step may directly correspond to the solution of the local/global solver. It may be provided numerically. Applicable methods are, for example, Jacobi method, Gauss-Seidel method and any form of conjugate gradient method.

[0055] The method 200 may further comprise altering 240 the one or more constraints to indicate that a cut 132 has been made to the tissue with the virtual surgical instrument 112. This can be done based on the position, and optionally the orientation and/or velocity, of the virtual surgical instrument 112 with respect to the tissue elements. The cut 132 can be made, and also be transmitted for visualization, substantially in real-time, allowing the method to be used for surgery preparation. In response to the cut 132, haptic feedback can also be generated. For example, the method may comprise a step where the model is used to generate a haptic feedback, for example by an actuator for haptic feedback, which may be comprised in the tracking device 110. The haptic feedback may be caused to be generated based on one or more changes for the tissue elements, for example in response to a cut 132 and/or a change in spatial location of one or more tissue elements. Alternatively or additionally, it may be caused to be generated in response to relatine movement of the virtual surgical instrument 112 and one or more tissue elements, for example in response to the virtual surgical instrument 112 being within a threshold distance from one or more tissue elements. The haptic feedback may comprise any combination a force, a vibration and a motion.

[0056] FIGS. 3 and 4 show two detailed examples of methods, which can be performed independently from each other or as a combined method. The apparatus 100 and/or a computer program product may be configured to perform either or both of the methods. In accordance with these examples, a method may comprise any or all features of the methods described above together with the following additional features, which may be included in any suitable order within a composite method.

[0057] A feature common for both cases is that tissue elements can be represented with density distributions. This allows the tissue to be replicated as a continuum instead of spatially discrete tissue elements. However, the tissue elements continue to be characterized by their spatial location and the spatial location for a tissue element thereby corresponds to the density distribution for the tissue element. Each tissue element can therefore correspond to both a spatial location and a density distribution. The spatial location for the tissue element may correspond, for example, a center point and/or a symmetry point for the density distribution. For example, the density distribution may be, at least substantially, rotationally symmetric or spherically symmetric with respect to the spatial location for the tissue element. However, other shapes for the density distribution are also possible. In any case, the density distribution is non-vanishing, or non-zero, also outside the spatial location for the tissue element. The density distribution has a total density, which can be determined as a sum or an integral of the density distribution over space. The density distribution can have a continuous part but, optionally, it can also have one or more discontinuous parts. The density distribution may vanish, or be zero, above a threshold distance from the spatial location. This allows the interactions between tissue elements to be limited to close neighbors. The density distribution may go to zero continuously, which can improve the stability of the local/global solver. The density distribution for two or more, or even all, tissue elements for the tissue may have at least substantially the same shape, allowing a simple definition of the tissue. However, density distributions of two or more different shapes can also be used to allow different types of tissue elements. Similarly, the total density of the density distribution may be constant for the tissue elements for the tissue or one or more tissue elements may correspond to a density distribution having a different total density. The density distribution may, for example, comprise a Gaussian shape, which may be limited away from the spatial location for the corresponding tissue element. The peak of the density distribution may correspond to the spatial location for the corresponding tissue element. Using both a distinct spatial location and a density distribution for a tissue element allows the tissue element to have a single well-defined spatial location while simultaneously simulating a continuum. In particular, maintaining the single well-defined spatial location allows the local/global solver to be used as described above. In addition, interactions between tissue elements can be determined using the one or more constraints for the local/global solver. The one or more constraints may comprise one or more constraints for the density distribution. These constraints may be, at least partially, common to several or all tissue elements. An example of a constraint is a threshold value for density, for example for cumulative density at a spatial position, such as the spatial location of a tissue element.

[0058] The use of density functions further allows integrating fluids into the method conveniently, which is a specifically advantageous feature for surgery. This further allows an improved process of forming a tissue by creating an empty space, causing tissue elements with liquid-like interactions fill the space, at least partially, and when the tissue elements have accumulated into form, create an elastic structure based on their spatial locations. This way, the distances between the tissue elements are optimized by liquid dynamics. Such a structure could be, for example, fat tissue. Consequently, a method for generating a tissue for surgery preparation can thus be provided.

[0059] FIG. 3 shows an example of a method 300, a combination of features which has been found to bring realistic elasticity into the tissue to allow improved real-time replication of a surgical operation.

[0060] Here, any method 200 described above has tissue elements represented 310 with density distributions. In addition, a deformation gradient or an indication of a deformation gradient (both of us referred below as “the deformation gradient”) is determined 320. Here, the deformation gradient indicates elasticity of the tissue and it can be formulated in accordance with the principles of strain theory or finite strain theory in particular. The deformation gradient can be used to indicate how the deformation of the tissue changes spatially. It can indicate the spatial change of deformation between an earlier state and a later state for the tissue elements. The deformation gradient may be represented with a matrix, or a tensor. It may act as a transformation matrix or tensor which transforms an infinitesimal line element from an initial position to the current position. Examples of a deformation gradient can be seen in “Peer et al.—An Implicit SPH Formulation for Incompressible Linearly Elastic Solids: Implicit Elastic SPH Solids; Computer Graphics Forum 37-6, 2017” (here also “Peer”), particularly in sections 3.1. and 3.3. of the reference. Accordingly, the deformation gradient can be a Smoothed Particle Hydrodynamics deformation gradient. As an indication of a deformation gradient, a displacement gradient or any other construct providing the corresponding indication of the elasticity of the tissue can be used.

[0061] Importantly, the set of local solutions for the local/global solver can be obtained 340 under a constraint that the deformation gradient substantially corresponds to a reference deformation gradient, which may be a constant reference deformation gradient. In particular, the reference deformation gradient may at least substantially correspond to a previously determined state for the tissue elements, such as the initial state for the tissue elements. The previously determined state may be a state for the tissue elements which has been determined before a cut 132 has been made to the tissue with the virtual surgical instrument 112. This can be achieved by having the reference deformation gradient correspond to an identity matrix.

[0062] The previously determined state may also be a state for the tissue elements prior to or at the beginning of the local step. Correspondingly, the deformation gradient or the reference deformation gradient may change during the replication of the surgical operation, for example in response to a cut 132 being made by the virtual surgical instrument 112. Even in this case, the deformation gradient can still be made to correspond to a reference deformation gradient over a single execution of the local step, for example by excluding any tissue elements separated by the cut 132 from the deformation gradient and the reference deformation gradient is determined anew without the excluded tissue element or elements. Irrespective of whether the previously determined state is the initial state or a later state, making the deformation gradient correspond to the reference deformation gradient over a local step has been found to allow improved real-time replication of a surgical operation, particularly when density functions are used for tissue elements. Using the constraint as one of the one or more constraints in the local step allows accommodating the constraint for the local/global solver.

[0063] The solution from the local/global solver may be provided utilizing the linear elasticity approximation. Also, it has been found that the reference deformation gradient can be co-rotated to significantly improve the real-time performance of the method. Examples of a co-rotated deformation gradient are shown in Peer, particularly in section 3.3. of the document. Removing 330 rotation from the reference deformation gradient has been found to be compatible with the local/global solver. It is noted that while a standard method for deformable elastic solids is to remove rotation from a system with a Cauchy-Green deformation tensor, here the rotation can be removed without squaring the deformation gradient. Instead, the rotation removal can be linear with respect to deformation gradient, for example using a rotation extraction method corresponding to that shown in Peer. The rotation can be removed so that the deformation gradient can directly be used as a linear strain or deformation tensor.

[0064] FIG. 4 shows an example of a method 400, a combination of features which has been found to bring realistic incompressibility into the tissue to allow improved real-time replication of a surgical operation.

[0065] Here, any method 200, 300 described above has tissue elements represented 310 with density distributions. In addition, a neighborhood can be determined 420 for one or more tissue elements. The neighborhood indicates the group of tissue elements (hereafter also “neighboring tissue elements”) that are close enough for interaction with the tissue element for which the neighborhood is defined (hereafter also “main tissue element”). As a matter of definition, the neighborhood may either include or exclude the tissue element with respect to which the neighborhood is determined. Correspondingly the group may consist of zero, one or more tissue elements. The neighborhood for a tissue element may be determined based on a threshold distance, for example so that neighboring tissue elements are those whose spatial location is within the threshold distance from the spatial location for the main tissue element. Any or all neighborhoods for tissue elements may be determined anew for subsequent local steps. This can be used to provide incompressibility after a cut 132 has been made to the tissue since the model can now, in real-time, determine the changing closeness relationships for tissue elements. A neighborhood may be defined for each tissue element so that each tissue element is the main tissue element for a neighborhood. Depending on the configuration of the tissue and the definition of the neighborhood, it may be possible, at least in some situations, that one or more tissue elements are not part of any neighborhoods. In addition to the matter of incompressibility, using the neighborhoods allows the tissue to be replicated without a separate collision detection system, significantly improving the real-time performance of the method. It is noted that when the neighborhoods are used, the initial state for the tissue elements needs not necessarily be defined or used. Instead, the reference deformation gradient may be separately defined for each neighborhood. These neighborhoods related by elasticity may change, even solely, in response to a cut 132. In contrast, neighborhoods related by incompressibility may change freely between time steps. Also, the deformation gradients for the neighborhoods may be made to correspond to reference deformation gradients, such as constant reference deformation gradients, for the neighborhoods over the local step, as described above, for example so that any or all reference deformation gradients for the neighborhood correspond to identity matrices. This can provide the abovementioned effect of robustness even for neighborhoods changing from one local step to another. It is noted that since the neighborhoods may change for subsequent local steps, the system matrix for subsequent global steps may change correspondingly.

[0066] A constraint for the cumulative density for the neighborhood can then be determined 430 to provide incompressibility to the tissue. The constraint can be used as one of the one or more constraints for the tissue elements in the local step. The cumulative density for the neighborhood can be defined for some or all spatial points of the neighborhood. It has been found that an effective way can be to actually define it for a single point within the neighborhood, which can be the spatial location for the main tissue element. As an example of a simple and effective constraint, the cumulative density for the neighborhood at the spatial location of the tissue element can be limited, for example with a target value, an upper boundary or a lower boundary. Generally, the cumulatine density can include a density for each of the neighboring tissue elements, e.g. as a sum density.

[0067] As another simple and effective constraint, it has been found that a scaling factor can be determined 440 for limiting the cumulative density. The scaling factor may be determined as the value with which distances of the neighboring tissue elements from the main tissue element need to be multiplied to have a threshold cumulative density for the neighborhood, for example at the spatial location for the main tissue element. The scaling factor may be determined in the local step. The scaling factor may be used in the local and/or global in different ways. One reliable and stable way to do this is to determine an instantaneous deformation gradient for the neighborhood in a similar way to how it is done for the reference deformation gradients. This instantaneous deformation gradient can then be multiplied by the scaling factor and the product can be used as the local solution, to which the global step will try to match the unscaled instantaneous deformation gradient. This may be performed with rotation extraction, for example as for the case of elasticity, but also without, since the neighborhoods related by incompressibility are free to change between time steps. This allows a very efficient way for providing incompressibility to the tissue. The scaling factor can be used to increase the likelihood that, during a single iteration round, the local/global solver moves those tissue elements which have the largest effect on the density of the neighborhoods.

[0068] By using density distributions that are functions of distance from the elements, the distances within a neighborhood can also be used to determine interaction strength between tissue elements. Based on the distance between the spatial locations of two tissue elements, a weighing coefficient may be determined for interaction. Decreasing distance between two tissue elements can correspond to a larger overlap in density distributions and, thus, a stronger interaction, which may be indicated by a larger weighing coefficient. The distances may be determined within a neighborhood with respect to the main tissue element so that the distance for each of the neighboring tissue element with respect to the main tissue element is determined. The weighing coefficient may be determined in the local step. The weighing coefficient may be used in the local and/or global step to weight the contributions of the tissue elements to the deformation gradients, for example based on how much their density distributions overlap or by some other indication for how strongly they interact.

[0069] The apparatus as described above may be implemented in software, hardware, application logic or a combination of software, hardware and application logic. The application logic, software or instruction set may be maintained on any one of various conventional computer-readable media. A “computer-readable medium” may be any media or means that can contain, store, communicate, propagate or transport the instructions for use by or in connection with an instruction execution system, apparatus, or device, such as a computer. A computer-readable medium may comprise a computer-readable storage medium that may be any media or means that can contain or store the instructions for use by or in connection with an instruction execution system, apparatus, or device, such as a computer. The examples can store information relating to various processes described herein. This information can be stored in one or more memories, such as a hard disk, optical disk, magneto-optical disk, RAM, and the like. One or more databases can store the information used to implement the embodiments. The databases can be organized using data structures (e.g., records, tables, arrays, fields, graphs, trees, lists, and the like) included in one or more memories or storage devices listed herein. The databases may be located on one or more devices comprising local and/or remote devices such as servers. The processes described with respect to the embodiments can include appropriate data structures for storing data collected and/or generated by the processes of the devices and subsystems of the embodiments in one or more databases.

[0070] All or a portion of the embodiments can be implemented using one or more general purpose processors, microprocessors, digital signal processors, micro-controllers, and the like, programmed according to the teachings of the embodiments, as will be appreciated by those skilled in the computer and/or software art(s). Appropriate software can be readily prepared by programmers of ordinary skill based on the teachings of the embodiments, as will be appreciated by those skilled in the software art. In addition, the embodiments can be implemented by the preparation of application-specific integrated circuits or by interconnecting an appropriate network of conventional component circuits, as will be appreciated by those skilled in the electrical art(s). Thus, the embodiments are not limited to any specific combination of hardware and/or software.

[0071] The different functions discussed herein may be performed in a different order and/or concurrently with each other.

[0072] Any range or device value given herein may be extended or altered without losing the effect sought, unless indicated otherwise. Also any example may be combined with another example unless explicitly disal-lowed.

[0073] Although the subject matter has been described in language specific to structural features and/or acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as examples of implementing the claims and other equivalent features and acts are intended to be within the scope of the claims.

[0074] It will be understood that the benefits and advantages described above may relate to one embodiment or may relate to several embodiments. The embodiments are not limited to those that solve any or all of the stated problems or those that have any or all of the stated benefits and advantages. It will further be understood that reference to ‘an’ item may refer to one or more of those items.

[0075] The term ‘comprising’ is used herein to mean including the method, blocks or elements identified, but that such blocks or elements do not comprise an exclusive list and a method or apparatus may contain additional blocks or elements.

[0076] Although the invention has been the described in conjunction with a certain type of apparatus and/or method, it should be understood that the invention is not limited to any certain type of apparatus and/or method. While the present inventions have been described in connection with a number of examples, embodiments and implementations, the present inventions are not so limited, but rather cover various modifications, and equivalent arrangements, which fall within the purview of prospective claims. Although various examples have been described above with a certain degree of particularity, or with reference to one or more individual embodiments, those skilled in the art could make numerous alterations to the disclosed examples without departing from the scope of this specification.