IMAGE BASED MOTION CONTROL CORRECTION

Abstract

The present invention relates to a method of adjusting control commands for moving a medical camera connected to a motorized support structure, wherein the adjustment is based on images provided by the camera. Based on a comparison of at least two images provided by the camera, an actual motion of the camera is determined and compared with an intended motion defined by a control command forwarded to the motorized support structure. In case a deviation between the intended motion and the actual motion is determined, a correction is applied to the control command such that the actual motion of the camera coincides with the intended motion.

Claims

1. A computer-implemented method of adjusting control commands for moving an associated medical imaging device coupled with an associated moving device that is operable to move the associated medical imaging device, the method comprising: acquiring first image data that describes a first image of an associated structure received by the associated medical imaging device; acquiring intended motion data that describes an intended motion of the associated medical imaging device, wherein the intended motion data is based on a control command received by the associated moving device; acquiring second image data that describes a second image of the associated structure received by the associated medical imaging device that has been moved by the associated moving device as a result of the associated moving device receiving the control command; determining actual motion data based on the first image data and the second image data, wherein the actual motion data describes an actual motion of the associated medical imaging device; and determining correction data based on the intended motion data and the actual motion data, wherein the correction data describes a correction to be applied to the control command in order to produce the intended motion of the associated medical imaging device.

2. The method according to claim 1, wherein the determining the correction data comprises correlating the intended motion data with the actual motion data in a coordinate system that is common to both the intended motion data and the actual motion data, the coordinate system comprising a coordinate system of an associated display device that is adapted to display images that are received by the associated imaging device.

3. The method according to claim 1, wherein the acquiring the intended motion data comprises acquiring intended motion data that is based on a control command that is effected by at least one of: a user control command input transmitted by an associated input device; and/or an automatic control command generated and transmitted by an associated control device.

4. The method according to claim 1, wherein the acquiring the intended motion data comprises acquiring intended motion data that describes: an intended direction of motion of the associated medical imaging device; and/or an intended velocity of motion of the associated medical imaging device.

5. The method according to claim 4, wherein the acquiring the intended motion data that describes the intended direction of motion of the associated medical imaging device comprises: acquiring intended motion data that describes an intended direction of motion of the associated medical imaging device that is parallel to a plane of at least one of the first image and/or the second image.

6. The method according to claim 1, wherein the acquiring the intended motion data that describes the intended velocity of motion of the associated medical imaging device comprises: acquiring intended motion data that describes an intended velocity of motion of the associated medical imaging device that comprising a function of a dimension of an associated display device comprising at least one of: a width of the associated display device, a height of the associated display device, a size of the associated display device, and/or a pixel number of the associated display device.

7. The method according to claim 1, wherein the determining the actual motion data comprises determining the actual motion data by image processing determining the motion of the associated medical imaging device relative to the associated structure displayed on an associated display device with a first image corresponding to the first image data and a second image corresponding to the second image data.

8. The method according to claim 1, wherein the determining the correction data comprises determining correction data to be applied to the control command to compensate for a difference between an actual position of the associated medical imaging device and an intended position of the associated medical imaging device that was caused by a deviation between the actual motion of the associated medical imaging device and the intended motion of the associated medical imaging device.

9. The method according to claim 1 wherein the determining the correction data comprises determining the correction data for every control command received by the associated moving device.

10. The method according to claim 1, further comprising determining, based on the correction data, control data describing a corrected control command to be transmitted to the associated moving device.

11. The method according to claim 1 further comprising controlling an associated display device to not show in a sequence of images a motion of the associated medical imaging device that deviates from the intended motion.

12. The method according to claim 1, wherein the determining the correction data comprises applying a plausibility check to determine whether the determined correction of the control command is reasonable, particularly wherein a reasonable correction of the control command does not exceed a maximum limit and/or a minimum limit.

13. The method according to claim 1, further comprising determining correction verification data, describing whether the control command has been validly corrected, particularly based on at least a third image received by the associated medical imaging device that has been moved as a result of the corrected control command.

14. A program logic stored in a memory device of a computer that when executed on the computer or when loaded onto the computer, causes the computer to perform a method of adjusting control commands for moving an associated medical imaging device coupled with an associated moving device that is operable to move the associated medical imaging device, the method comprising: acquiring first image data that describes a first image of an associated structure received by the associated medical imaging device; acquiring intended motion data that describes an intended motion of the associated medical imaging device, wherein the intended motion data is based on a control command received by the associated moving device; acquiring second image data that describes a second image of the associated structure received by the associated medical imaging device that has been moved by the associated moving device as a result of the associated moving device receiving the control command; determining actual motion data based on the first image data and the second image data, wherein the actual motion data describes an actual motion of the associated medical imaging device; and determining correction data based on the intended motion data and the actual motion data, wherein the correction data describes a correction to be applied to the control command in order to produce the intended motion of the associated medical imaging device.

15. A medical system comprising: a computer comprising a processor and a memory device, wherein the computer is operable to execute program logic stored in the memory device to perform a method of adjusting control commands for moving a medical imaging device coupled with a moving device that is operable to move the medical imaging device, the method comprising: acquiring first image data that describes a first image of an associated structure received by the medical imaging device; acquiring intended motion data that describes an intended motion of the medical imaging device, wherein the intended motion data is based on a control command received by the moving device; acquiring second image data that describes a second image of the structure received by the medical imaging device that has been moved by the moving device as a result of the moving device receiving the control command; determining actual motion data based on the first image data and the second image data, wherein the actual motion data describes an actual motion of the medical imaging device; and determining correction data based on the intended motion data and the actual motion data, wherein the correction data describes a correction to be applied to the control command in order to produce the intended motion of the medical imaging device; and a moving device operable to move a medical imaging device connected with the moving device wherein the computer is operably coupled to the moving device for issuing a control signal to the moving device for controlling the operation of the moving device based on the first image data and the second image data.

16. (canceled)

17. The method according to claim 1: wherein the determining the correction data comprises determining correction data to be applied to the control command to compensate for a difference between an actual position of the associated medical imaging device and an intended position of the associated medical imaging device that was caused by a deviation between the actual motion of the associated medical imaging device and the intended motion of the associated medical imaging device. further comprising determining, based on the correction data, control data describing a corrected control command to be transmitted to the associated moving device.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In the following, the invention is described with reference to the appended figures which give background explanations and represent specific embodiments of the invention. The scope of the invention is however not limited to the specific features disclosed in the context of the figures, wherein

[0060] FIG. 1 illustrates the basic steps of the control adjustment method in accordance with the present invention;

[0061] FIG. 2 shows a schematic system setup in accordance with the present invention along with a schematic illustration of a control command correction in accordance with the present invention;

[0062] FIG. 3 shows a schematic illustration of the system according to the fifth aspect.

DESCRIPTION OF EMBODIMENTS

[0063] FIG. 1 shows the basic steps of the inventive control adjustment method in accordance with the present invention. In a first step S1, first image data is acquired via at least one (endoscope) camera, whereupon, in second step S2, intended motion data is acquired from an input device. In this respect it should be noted that the steps of acquiring first image data and acquiring intended motion data may be also performed in a reverse order. In a third step S3, second image data is acquired via the (endoscope) camera, such that, in a fourth step S4, actual motion data can be determined from the first image data and the second image data. In a fifth step S5, the determined actual motion data is compared with the acquired intended motion data so as to determine correction data that describes a correction to be applied to the control command, such that the actual camera motion corresponds to the intended camera motion.

[0064] FIGS. 2 and 3 shows the basic components of a system setup in accordance with the present invention. A medical endoscopy system 1 comprises a monitor 8, a joystick 7 and a semi-robotic support arm 4 comprising a plurality of arm-sections which are connected to each other via motorized joints such that the arm-sections can be moved with respect to each other. At the distal end of the motorized support arm 4 an endoscopic camera 5 is held in place above a structure 9 (represented by a pyramid) to be observed via the endoscope camera 5, with the camera's line of sight being directed towards the structure 9.

[0065] Further, the system 1 comprises a control device 6 for performing the inventive control adjustment method, which is connected to the motorized support structure 4, the endoscopic camera 5, the joystick 7 and the monitor 8. The control device includes a computer 2 having a digital processor for performing the inventive control adjustment method, and a program storage medium 3 with a program that causes the computer to perform the method steps of the inventive method.

[0066] In a first step S1, the camera 5 acquires an image of the structure 9, wherein the image data is transferred via the control device to the monitor 8 which then shows an image of the structure 9. As can be seen in the frame of FIG. 2, which corresponds to step S1, the structure 9 is shown on the monitor in the bottom left corner of the displayed image.

[0067] A user who may not wish the structure 9 to be displayed on the bottom left corner of the display 8 may input a control command via the joystick 7 to cause the motorized support structure 4 to move the endoscope camera 5 with respect to the structure 9, such that structure 9 is then shown at the top left corner of the image shown by the monitor 8. In order to do so, the user may push the joystick in an “up”-direction as this is schematically shown in the frame which corresponds to step S2. After the control command which represents the intended motion of the camera 5 has been input and the motorized support structure 4 has accordingly moved, it may occur that the structure 9 is not shown as intended at the top left corner of the image, but instead at the top right corner of the image (cf. the frame which corresponds to step S3). By performing image processing techniques in order to compare the camera images acquired in steps S1 and S3, an actual motion of the support structure 4 is calculated in step S4 and it is found that the structure 9 has not moved in an “up”-direction in the image, but rather in an “up”-direction and a “right”-direction. Consequently, the actual camera motion (cf. frame corresponding to step S4) deviates from an intended camera motion (cf. frame corresponding to step S2). For example, this deviation may be the result of a rotational degree of freedom of the endoscope camera 5 along its line of sight and with respect to the support arm 4. In any case, the deviation of the actual camera motion from the intended camera motion is undesired as it causes confusion for the practitioner operating the endoscope camera 5 via the joystick 7 and the monitor 8.

[0068] In order to compensate for this unwanted deviation, the present invention determines a correction to be applied to the control command such that it produces the intended motion of the camera 5. As schematically shown in the frame according to method step S5, a transformation matrix M is calculated which transforms the vector which represents the actual motion to the vector which represents the intended motion. This calculated correction matrix M may then be applied to any control command which is transmitted to the support arm 5 subsequent to the second image, such that the camera 5 moves as intended and in accordance with the control commands input via the joystick 7 (cf. frame according to method step S6).

[0069] In case the correction matrix M is not valid anymore, for example if the camera 5 rotates with respect to the support arm 4 after the correction matrix M has been calculated, a further control correction can be performed in the same manner as previously described in order to calculate a new, updated correction matrix M.