Method and apparatus for determining or predicting the position of a target

Abstract

A data processing method for determining the position of a target, comprising the steps performed by a computer: a) acquiring a target movement model specifying a movement cycle of the target; b) acquiring a target position signal representing a view of the target from a single direction and/or provided by a single imager; c) determining, based on the acquired target position signal and the target movement model, the position of the target.

Claims

1. A data processing method performed by a computer for determining a three-dimensional (3D) position of a target, comprising: a) acquiring, using a first imager having a first imaging plane and a second imager having a second imaging plane, a target movement model specifying a movement cycle of the target, wherein the target movement model specifies at least two sections that are associated with an inhale phase and an exhale phase; b) acquiring, using the first imager, a two-dimensional (2D) target position signal representing a view of the target in the first imaging plane of the first imager; c) determining the 3D position of the target based on the 2D target position signal acquired using the first imager, and the target movement model, wherein the determining the 3D position of the target comprises: determining, based on the 2D target position signal, an epipolar line in the second image plane of the second imager corresponding to the view of the target in the first imaging plane of the first imager; determining at least one intersection between the epipolar line in the second image plane of the second imager with a projected target trajectory that is projected onto the second image plane of the second imager based on the target movement model; and determining the 3D position of the target based on the at least one intersection between the epipolar line in the second image plane of the second imager with the projected target trajectory.

2. The data processing method of claim 1 for updating a correlation model correlating a surrogate signal with the 3D position of the target, the update being based on an update signal being the 2D target position signal, and including the determined 3D position information of the target and the target movement model, further comprising: d) acquiring the surrogate signal; and e) correlating the surrogate signal with the determined 3D position of the target to obtain the updated correlation model.

3. The data processing method of claim 1, wherein the first imager is a single imager of a stereoscopic imaging apparatus.

4. The data processing method of claim 2, wherein the target movement model acquired in step a) is acquired based on at least one of: latest target detection on an obstructed or not used imager of the first imager and the second imager; latest stereoscopic target detection; latest prediction of the 3D position of the target, the prediction being based on the updated correlation model and the surrogate signal; latest prediction of the 3D position of the target being projected onto the respective imaging plane of an obstructed or not used imager of the first imager and the second imager.

5. The data processing method of claim 1, wherein the specific movement phases are associated with a surrogate signal or relative marker positions.

6. The data processing method of claim 1, wherein information on the specific movement phases is used when determining the 3D position of the target.

7. The data processing method of claim 2, wherein the updated correlation model is based on a number of update points distributed over a movement cycle or breathing cycle or respiratory cycle of the target.

8. The data processing method of claim 2, further comprising updating a prediction model predicting the 3D position of the target in a body using the updated correlation model.

9. The data processing method of claim 2, further comprising updating a prediction model predicting the 3D position of the target in a body, wherein the prediction model is updated based on the updated correlation model, wherein a prediction of the 3D position of the target in the body is made by: predicting a future value of the surrogate signal or a future position of a surrogate element using a surrogate movement model; and determining the predicted 3D position of the target in the body based on the predicted future value of the surrogate signal or the predicted future position of the surrogate element, using the updated correlation model.

10. The data processing method of claim 1, wherein the target movement model is stored in a memory and can be read from the memory.

11. The data processing method of claim 1, wherein the 2D target position signal is acquired by the first imager during a condition that the second imager has an obstructed line of sight to the target.

12. A data processing method performed by a computer for determining a three-dimensional (3D) position of a target, comprising: a) acquiring, using a first imager having a first imaging plane and a second imager having a second imaging plane, a target movement model specifying a movement cycle of the target, wherein the target movement model specifies at least two sections that are associated with an inhale phase and an exhale phase, and wherein the first imager and the second imager are respective imagers of a stereoscopic X-ray imaging apparatus; b) acquiring, using the first imager, a two-dimensional (2D) target position signal representing a view of the target in the first imaging plane of the first imager; c) determining the 3D position of the target based on the 2D target position signal acquired using the first imager and the target movement model, wherein the determining the 3D position of the target comprises: determining, based on the 2D target position signal, an epipolar line in the second image plane of the second imager corresponding to the view of the target in the first imaging plane of the first imager; determining at least one intersection between the epipolar line in the second image plane of the second imager with a projected target trajectory that is projected onto the second image plane of the second imager based on the target movement model; and determining the 3D position of the target based on the at least one intersection between the epipolar line in the second image plane of the second imager with the projected target trajectory.

13. The data processing method of claim 12 for updating a correlation model correlating a surrogate signal with a positional information of the target, the update being based on an update signal being the 2D target position signal and including the determined 3D position of the target and the target movement model, further comprising: d) acquiring the surrogate signal; and e) correlating the surrogate signal with the determined 3D position of the target to obtain the updated correlation model.

14. The data processing method of claim 12, wherein the 2D target position signal is acquired by the first imager during a condition that the second imager has an obstructed line of sight to the target.

15. A system for determining a three-dimensional (3D) position of a target comprising: a target movement model generator configured to acquire, using a first imager having a first imaging plane and a second imager having a second imaging plane, a target movement model specifying a movement cycle of the target, wherein the target movement model specifies at least two sections that are associated with an inhale phase and an exhale phase; a target position acquiring element configured to acquire, using the first imager, a two-dimensional (2D) target position signal representing a view of the target in the first imaging plane of the first imager; and a determination section connected to the target movement model generator and the target position acquiring element and receiving information or signals therefrom to determine, the 3D position of the target based on the 2D target position signal acquired using the first imager, and the target movement model, wherein the determination section is configured to: determine, based on the target position signal, an epipolar line in the second image plane of the second imager corresponding to the view of the target in the first imaging plane of the first imager; determine at least one intersection between the epipolar line in the second image plane of the second imager with a projected target trajectory that is projected onto the second image plane of the second imager based on the target movement model; and determine the 3D position of the target based on the at least one intersection between the epipolar line in the second image plane of the second imager with the projected target trajectory.

16. The system of claim 15 for updating a correlation model correlating a surrogate signal with a positional information of the target, the update being based on an update signal being the 2D target position signal and including the determined 3D position of the target and the target movement model, the system being further configured to: acquire the surrogate signal; and correlate the surrogate signal with the determined 3D position of the target to obtain the updated correlation model.

17. The system of claim 15, wherein the first imager is a single imager of a stereoscopic imaging apparatus.

18. The system of claim 16, wherein the updated correlation model is based on a number of update points distributed over a movement cycle or breathing cycle or respiratory cycle of the target.

19. The system of claim 15, wherein the target position acquiring element is configured to acquire the 2D target position signal by the first imager in a condition that the second imager has an obstructed line of sight to the target.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

(1) In the following, the invention as described with reference to the enclosed figures which represent preferred embodiments of the invention. The scope of the invention is however not limited to the specific features disclosed in the figures, which show:

(2) FIG. 1 a depiction of the base line shift;

(3) FIG. 2 the relationship between the 3D target position and its 2D projection onto an imager plane;

(4) FIG. 3 a target detection based on a target movement model and a non-stereoscopic image;

(5) FIG. 4 a system for determining the position of a target according to a first aspect; and

(6) FIG. 5 a system for determining the position of a target according to a second aspect.

DETAILED DESCRIPTION

(7) FIG. 1 shows in an abstract form a positional change of a surrogate signal being an upper sinusoidal wave. The surrogate signal might be the detected positions of one or more markers being located on the body surface of a patient, the body surface moving, for example due to a breathing motion. As can be seen on the left side of FIG. 1, the surrogate signal is a periodic signal having repeated cycles 1, 2, . . . .

(8) Below the surrogate signal there is shown as another sinusoidal signal the position of a target also moving due to a vital motion, such as a breathing motion. Same as the above surrogate signal, the below target position performs a periodic movement.

(9) If the target's position at a specific time is correlated with the marker position or surrogate signal, then a target-marker-model TMM or in an abstract manner a correlation model can be defined specifying the relation between the detected surrogate signal (e.g. movement of the body surface) with the movement of the target. Thus, it is possible to determine a target's position based on only the surrogate signal alone, e.g. based on only the detected marker position(s). As can be seen, the relationship or distance between the surrogate signal and the target position may vary over a cycle, which variation can be included into the target-marker-model. The target-marker-model can e.g. be this stored relationship over one cycle.

(10) However, this relation may change due to for example sagging movement of the patient lying on a couch for a longer time. If for example the body surface to which the markers are attached sags down, then the target-marker-model TMM or correlation model changes. As can be seen on the right side of FIG. 1, even after the change in relationship, it is still possible to use a surrogate signal to determine the position of the target. However, the target-marker-model or correlation model has to be adapted to the change of the relation.

(11) FIG. 2 shows a target T moving in a simplified manner along a predefined path being a circle. The movement cycle of the target T defines a trajectory drawn in a solid line.

(12) An imager viewing the target's movement from a single specified direction has an imaging plane P. The trajectory of the target T is projected onto the imaging plane P of the imager when the target's movement is seen from the imager. The target's movement cycle being in the shown example a circle may be projected onto the imaging plane P being there an oval. The projected trajectory can itself be acquired and can for example be used as an embodiment of a target movement model. Advantageously, parts of the trajectory can be associated with specific movement cycles, such as inhale and exhale. The trajectory can have a specific direction (in the shown example: clockwise) as indicated by arrows.

(13) When the target T moving along the trajectory is imaged by a stereoscopic camera, i.e. by two cameras viewing the target from different viewing angles, the target's trajectory is projected onto each of the respective imaging planes and for every imager there can be made a target movement model as described above. If both imagers see the target, the target's position can be determined by back projection.

(14) According to an aspect, the target movement model (for example the target trajectory line projected onto the image plane) of the obstructed imager is used in combination with the image of the imager seeing the target for determining the target's position.

(15) A target movement model can thus for example be built by the projected trajectory of one or of both imagers.

(16) A target movement model can for example be built during an initial stereoscopically taken movement sequence, such as an X-ray fluoro sequence, and can be updated whenever images from both imagers or both X-ray panels are simultaneously available. The update of the target movement model can be a full update e.g. establishing a new target movement model in total, or can be a partial update, e.g. updating or modifying only a section of a prior known target movement model.

(17) For example, a series of shots can be taken at a predetermined phase, such as e.g. fully exhale, and depending on the determined position of the target the target movement model can be modified. If for example an offset is detected in a specific phase, the whole target movement model can be updated or shifted by the detected offset.

(18) Exemplary Solution

(19) If coordinates of target's projection are given with x.sub.i and y.sub.i, and predicted 3D target position with x, y and z then this model has the form:
f.sub.x(x.sub.i,y.sub.i).fwdarw.x(1)
f.sub.y(x.sub.i,y.sub.i).fwdarw.y(2)
f.sub.z(x.sub.i,y.sub.i).fwdarw.z(3)

(20) Dependency between 3D position and projection is modeled with linear functions whose parameters are calculated using linear regression and have the form:
x=a.sub.xx.sub.i+b.sub.xy.sub.i+c.sub.x(4)
y=a.sub.yx.sub.i+b.sub.yy.sub.i+c.sub.y(5)
z=a.sub.zx.sub.i+b.sub.zy.sub.i+c.sub.z(6)

(21) These equations can be written as:
{right arrow over (t.sub.3D)}=M.sub.3D{right arrow over (t.sub.2D)}(7)
where {right arrow over (t.sub.3D)}=(x,y,z).sup.T represents predicted target position, {right arrow over (t.sub.2D)}=(x.sub.i,y.sub.i,z.sub.i).sup.T represents projection's position and the actual 3D model is:

(22) $\begin{array}{cc}{M}_{3D}=\left(\begin{array}{ccc}{a}_x& b_x& c_x\\ a_y& b_y& c_y\\ a_z& b_z& c_z\end{array}\right)& \left(8\right)\end{array}$

(23) According to an alternative solution, the 3D position of the target can be predicted or determined out of an image taken from a single imager (2D detection), for example being based on the principles of epipolar geometry.

(24) This is illustrated in FIG. 3 which shows the imaging planes of two imagers, imager 1 and imager 2, which were able to both image the target when setting up the target movement model. Imager 2 is obstructed in the shown example, but from a prior measurement the estimated target's trajectory as being projected onto the imaging plane of imager 2 is shown in the dotted oval line. Imager 1 can detect a point corresponding to the position of the target T. The back projection of the detected position of target T onto the target T (i.e. the line of sight of imager 1) is projected as the epipolar line shown as a broken line onto the imaging plane of imager 2. The epipolar line can be interpreted as specifying all possible locations of target T as detected by imager 1 when projected onto the image plane of imager 2. Since from the prior measurement the target's projected trajectory is known, the epipolar line can be intersected with the target's trajectory, thus obtaining two possible solutions shown as two crosses within the imaging plane of imager 2 defining two possible locations of the target in space (which can be determined when back projecting the detected position of imager 1 and the estimated position being one of the possible solutions of imager 2, the intersection of both back projections being the position of the target). The absolute or relative position and/or orientation of the imagers being known.

(25) In case the specific movement cycles or specific phases of the target's movement are as well known (being for example part of the target movement model), then it can be determined which one of the two possible solutions of FIG. 3 is the correct solution, thus ruling out one of the intersection points. The current breathing phase is for example known from the surrogate signal, such as from an acquired infrared marker signal.

(26) By computing the movement phases, such as breathing phases, a plausibility check can additionally be performed to determine how valid the model for the obstructed imager still is.

(27) In order to predict the 3D position of the target at any given moment the target's projection onto the X-ray panel (2D position) at the same moment can be predicted as well. For this a correlation model between a surrogate signal position and the velocity and 2D position is determined. This model can be built using a given number of most recent 2D position detections and can be updated whenever there is a new 2D detection. Furthermore, newer 2D detections can be weighted more. There can be one such model for every X-ray panel.

(28) The couch on which a patient is located can be moved whenever a beam has to be shot from a plane other than the one in which the gantry is rotating (if the couch is in the starting position this is the axial plane). This way the angle between patient and X-ray panels changes and with it the projection, as well as the patient position in machine coordinate system, which makes existing models invalid. A straightforward solution for this problem would be to discard all previous data and make a new stereoscopic X-ray fluoro sequence, but that would extend treatment time, expose the patient to unnecessary radiation (skin dose) and could even lead to overheating.

(29) In order to solve the couch rotation problem, a transformation matrix is used to rebuild the models, which projects 3D target positions onto X-ray panels at a new angle. This way new models can be built without a need to make new stereoscopic X-ray fluoro sequences.

(30) FIG. 4 shows an embodiment where a movement cycle or in a more simplified manner a plane is determined as a movement model. The orientation of the plane being known when determining the target movement model. FIG. 4 shows two imagers 1 and 2 being able to provide a stereoscopic X-ray view of the target T moving along movement cycle C within plane P.

(31) In case imager 2 is blocked, the position of the target T can be determined based on only an image signal from imager 1 having a view on the target T, since the back projected image of target T being intersected with plane P and/or correlated or intersected with movement cycle C delivers the position of target T.

(32) A stereoscopic camera IR being able to detect a surrogate signal, such as an infrared marker signal (marker is not being shown in FIG. 4) also viewing the operation scene or couch can be connected to a computer also receiving the imager signals to perform the described method.

(33) FIG. 5 shows a further embodiment, wherein again imager 2 does not provide a positional signal of target T due to for example being blocked by a gantry.

(34) The target movement model is the estimated projection of the target movement cycle C onto the imaging plane of imager 2 being shown as projected trajectory C.

(35) The projected trajectory C is intersected with the epipolar line of the line of sight of imager 1 and the target's position is determined as described with reference to FIG. 3.