METHOD AND ARRANGEMENT FOR ELECTROMAGNETIC RADIATION BASED NON-INVASIVE MONITORING OF A PERFORMANCE OF AN ANATOMIC OBJECT DURING AN OPERATION OR MEDICAL INTERVENTION

Abstract

An arrangement for electromagnetic radiation based non-invasive monitoring of an anatomic object during an operation includes at least one imaging device for obtaining at least two or more images of at least one surface point on a surface of the anatomic object over at least one fraction of a characteristic movement cycle from the surface of the anatomic object. In addition the arrangement includes an output for a display device or the display device, and a control unit for determining deformation based on movements of the at least one surface point over the at least one fraction of the cycle between the at least two or more images of the surface of the heart in function of time. Further changes in the deformation is determined or determined deformation is compared to reference deformation values to find any deviation in the performance or state of the deformation determined.

Claims

1. An arrangement for electromagnetic radiation based non-invasive monitoring or assessment of a performance of an anatomic object during an operation or during medical intervention or during medical examination of a subject, wherein the arrangement comprises: at least one imaging device or imaging input device for obtaining at least two or more images of at least one surface point on a surface of the anatomic object over at least one fraction of a muscle movement cycle from the surface of the anatomic object, an output for a display device or said display device, and a control unit for determining deformation based on movements of said at least one surface point over said at least one fraction of the cycle between said at least two or more images of the surface of the anatomic object in function of time and further determining changes in said deformation or for comparing determined deformation to reference deformation values to find any deviation in said performance or state of the deformation determined, and providing control data for controlling the display device to display the changes in said deformation or said deviation or state of the deformation determined in function of time.

2. The arrangement of claim 1, wherein the anatomic object is a heart, myocardium, right side of the heart, or lung, or other skeletal muscle, a back muscle or limb muscle.

3. The arrangement of claim 1, wherein the deformation is or comprises at least one of the following: variable distance of at least two surface points or local displacement of at least one surface point on the surface of the anatomic object over said at least one fraction of the cycle, such as a cardiac cycle, tissue displacement, velocity, or acceleration of at least one surface point over said at least one fraction of the cycle, contractility indicator based on strain rate, shortening fraction, principal shortening strain or deformation values determined from at least two surface points over said at least one fraction of the cycle, or contractility of the heart based on the contractility indicator and simultaneous measurement of pressures in a right atrium or right ventricle with a pulmonary artery catheter or pressure measurement, or variable surface area or volume defined by at least three surface points on the surface of the anatomic object over said at least one fraction of the cycle.

4. The arrangement of claim 1, wherein the control unit is configured to provide displacement vector for at least one surface point on the images as a time series function from the data representing a position of pixel kernels in space for determining said changes in said deformation.

5. The arrangement of claim 1, wherein the arrangement is configured to gather time series data from at least one additional external measuring device measuring vital signs of the subject, and wherein the arrangement is configured to provide at least one trigger pulse or a common time base for synchronizing or linking at least two or more images or measurement data from at least one of said imaging device or other device with each other and therefore determining said changes in said deformation between said two or more images, where said trigger pulse or the common time base is based on said time series data.

6. The arrangement of claim 5, wherein the arrangement is configured to set a zero-displacement reference frame for a first image gated by said trigger pulse or the common time base and compare a displacement vector provided for at least one second subsequent image obtained over at least one next fraction of a cycle to said zero-displacement reference frame and thereby determining said deformation or said changes in said deformation.

7. The arrangement of claim 5, wherein said trigger pulse or said common time base is based on the other physiological monitoring measures, electrocardiography (ECG), heart rate, blood pressure, central venous pressure, pulmonary artery pressure, pulmonary artery occlusion pressure, inspired and expired gases, oxygen saturation of the blood, cardiac output, ultrasound or tissue doppler imaging data or monitoring and quantification of cardiac cycle events in a right ventricle is provided into context of other vital sings and measures.

8. The arrangement of claim 1, wherein the arrangement further comprises or is arranged to provide control data for controlling life support machines, equipment or measures, cardiopulmonary bypass pump settings, ventilator or respirator settings or infusion pumps so that the deformation state or determined changes in said deformation is kept in or adjusted to a predetermined range or value or target range or value is achieved.

9. The arrangement of claim 1, wherein the arrangement is configured to visualize and quantify a deformation pattern characteristics with respect to amplitude, amplitude modulation, periodicity, aperiodicity, wavelength, frequency, phase, rhythm or combination thereof for representing said deformation or said changes in said deformation.

10. The arrangement of claim 9, wherein the arrangement is configured to determine a time lag between an event of an electrical activity measured by electrocardiography (ECG) and said deformation pattern for representing said deformation or said changes in said deformation.

11. The arrangement of claim 1, wherein the control unit is configured to provide a trend line control data for controlling the display device to display a trend line, said trend line representing a trend of changes in said deformation.

12. The arrangement of claim 1, wherein the control unit is configured to compare the value of the deformation or a trend of the changes in said deformation to a predetermined value or range, and whether said determined deformation or a trend deviates from the predetermined value or range, the control unit is configured to provide an alarm control data for controlling the display device or other device to display or generate the alarm indicating the deviating values or changes in said deformation.

13. The arrangement of claim 1, wherein the arrangement comprises or is at least arranged in a data communication with a database and is configured to provide data related to said deformation or changes in said deformation to said database or wherein said arrangement is configured to compare said determined deformation or changes in said deformation data to data previously stored in said database and thereby configured to identify possible match with said previously stored data and found a possible association with certain diseases or conditions inducing said deformation or deviating changes in said deformation.

14. The arrangement of claim 1, wherein the arrangement is configured to determine the state of said deformation or changes in said deformation by identifying common points or pixel kernels on images captured by at least two different imaging devices at the same point in time, and by tracing on each image of an imaging time sequence, where said tracing or a correlation is performed on images.

15. The arrangement of claim 1, wherein the arrangement is additionally configured to determine two- or three-dimensional coordinates of points on the surface of the anatomic object by measurement data obtained from the imaging device(s) using a coordinate system transformation.

16. The arrangement of claim 1, wherein the arrangement comprises a pattern providing device for providing a pattern on the surface of the anatomic object.

17. The arrangement of claim 1, wherein the arrangement further comprises or is arranged to receive a trigger signal from an ECG monitor and start data acquisition in case of detecting abnormal electrical activity.

18. An imaging device for electromagnetic radiation based non-invasive monitoring or assessment of a performance of an anatomic object during an operation or during medical intervention or during medical examination of a subject, wherein said imaging device is configured to obtain at least two or more images of at least one surface point on a surface of the anatomic object over at least one fraction of a cycle from the surface of the anatomic object, wherein the imaging device comprises: an output for a display device or said display device, and a control unit for determining deformation based on movements of said at least one surface point over said at least one fraction of the cycle between said at least two or more images of the surface of the anatomic object in function of time and further determining changes in said deformation or for comparing determined deformation to reference deformation values to find any deviation in said performance or state of the deformation determined, and providing control data for controlling the display device to display the changes in said deformation or said deviation or state of the deformation determined in function of time.

19. A method for electromagnetic radiation based non-invasive monitoring or assessment of a performance of an anatomic object during an operation or during medical intervention or during medical examination of a subject, wherein the method comprises steps of: obtaining at least two or more images of at least one surface point on a surface of the anatomic object over at least one fraction of a cycle from the surface of the anatomic object, and determining deformation based on movements of said at least one surface point over said at least one fraction of the cycle between said at least two or more images of the surface of the anatomic object in function of time and further determining changes in said deformation or for comparing determined deformation to reference deformation values to find any deviation in said performance or state of the deformation determined.

20. A non-transitory computer-readable medium comprising program code adapted to monitor or asses a performance of an anatomic object during an operation or during medical intervention or during medical examination of a subject, wherein the program code is adapted to perform steps of: obtaining at least two or more images of at least one surface point on a surface of the anatomic object over at least one fraction of a cycle from the surface of the heart, and determining deformation based on movements of said at least one surface point over said at least one fraction of the cycle between said at least two or more images of the surface of the anatomic object in function of time and further determining changes in said deformation or for comparing determined deformation to reference deformation values to find any deviation in said performance or state of the deformation determined, when said program code is run on a data processing device.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Next the invention will be described in greater detail with reference to exemplary embodiments in accordance with the accompanying drawings, in which:

[0056] FIG. 1 illustrates a principle of an exemplary arrangement for monitoring and assessment of a performance of a heart during an open cardiac surgery of a patient according to an advantageous embodiment of the invention,

[0057] FIG. 2 illustrates an exemplary method providing displacement vectors of the images in order to assess deformations and changes in the deformations according to an advantageous embodiment of the invention,

[0058] FIG. 3 illustrates an exemplary method for providing trigger pulse or a common time base according to an advantageous embodiment of the invention,

[0059] FIG. 4 illustrates an exemplary amplitude of monitored deformation change of the surface of the heart during a volume therapy according to an advantageous embodiment of the invention, and

[0060] FIGS. 5-7 illustrate examples of displacement and deformation data time series according to an advantageous embodiment of the invention.

DETAILED DESCRIPTION

[0061] FIG. 1 illustrates a principle of an exemplary arrangement 100 for monitoring and assessment of a performance of a heart 101 during an open cardiac surgery of a patient according to an advantageous embodiment of the invention. The arrangement 100 comprises imaging device 102, such as digital optical or other electromagnetic wave cameras (1, 2, or more), which can be CCD, CMOS, or other technology. The camera(s) is/are placed in three dimensional space at a/different position(s) (each) with a defined line of sight or radiation propagation direction to/from the object (heart). The camera position(s) in relation to the object and to each other are advantageously defined via a calibration procedure using a physical calibration object with defined geometrical features positioned at the intersection point of the different lines of sight/rays or by in-situ bundle adjustments, or other methods. Polarizing filters (not shown) can be placed in front of the OR-lights or dedicated polarized light sources can be used as well as polarizing filters in front of the camera to extinguish reflections or glare from the surface of the object.

[0062] The imaging devices 102 are configured to take two or more photographic images over the period of a fraction of a cardiac cycle or several cardiac cycles in defined time intervals ?t, whilst advantageously simultaneously synchronized time series data from an electrocardiography device and/or other external devices 103 measuring vital signs of the subject (patient undergoing open cardiac surgery) can be acquired. Camera timing is controlledand camera data captured via an electronic controller unit or processing unit 104 equipped with real time image capturing hardware whilst other external signals are captured digitally, using the dedicated communication protocols of the external devices, or analogue, using analogue-to-digital converters, where one of the signals can act as gating signal. Data is either processed after transfer to a computer unit, e.g., via Ethernet, or using an embedded graphics processing unit (GPU), as an example. In addition the arrangement 100 comprises advantageously a display device 106.

[0063] The processing unit 104 is advantageously configured to determine (by tracking) the deformation based on movements of surface points over at least one fraction of the cardiac cycle between said at least two or more images of the surface of the heart in function of time. In addition the processing unit 104 is configured to determine changes in the deformation to find any deviation in the performance or state of the deformation determined (or a feature changing during a cycle but in a normal state being regular and sequentially repetitive over several cycles). Alternatively determined deformation can be compared to reference deformation values to make said findings. Advantageously the processing unit 104 provides control data for controlling the display device 106 to display the changes in said deformation or said deviation or state of the deformation determined in function of time. The processing unit 104 can also provide control data or signal for controlling a life supporting machine 107 as is described elsewhere in this document.

[0064] In addition, the arrangement 100 may comprise or is at least arranged in a data communication 108 with a database 109, such as a cloud or other external service. Data related to, e.g., the deformation can be provided to the database, wherein said data can be compared to previously stored data in said database and thereby possible match with the previously stored data can be identified and possible associations with certain diseases or conditions inducing said deformation and/or deviating changes in said deformation found, such as e.g. valvular heart disease or problem with a mitral valve, tricuspid valve, ventricle or atrium, volume or pressure overload, arrhythmia, dysfunction, toxicity, ischemia, energy depletion, for example.

[0065] FIG. 2 illustrates an exemplary principle for providing displacement vectors 110 of the images in order to assess deformations and changes in the deformations according to an advantageous embodiment of the invention. The two- or three-dimensional coordinates 105 of points on the object surface are determined by measurements made with the camera(s) using a coordinate system transformation. Common points or pixel kernels are identified on images captured by the different cameras at the same point in time, and traced on each image of an imaging time sequence, where tracing or correlation can be performed on consecutive images or non-consecutive images, as can be seen in FIG. 2. The unique intensity distribution features within each pixel matrix of defined matrix size originate either from natural tissue texture or can be artificially applied. Due to the change of location (caused by respiration/ventilation of the subject and rigid body movement of the object due to competition between right and left ventricle for space in the pericardium as well as cardiopulmonary interaction) and shape (caused by cardiac activity) of the object, the position of pixel kernels in space changes over time t, from which the displacement vectors 110 of surface points can be determined as a time series function at sub-pixel resolution (translating into spatial sampling frequency of several micrometers) at time resolution between microseconds up to seconds. The above described Digital Image Correlation routine can herein refer to feature tracking, intensity tracking, or any other means of image registration or image alignment algorithms, where the trade-off is between processing time and spatio-temporal resolution.

[0066] According to an advantageous embodiment displacement vector(s) 110 is provided for at least one surface point 113 on the images as a time series function at sub-pixel resolution from the data representing a position of pixel kernels in space for determining said changes in said deformation. It is to be noted that due to the change of location and shape of the object, the position of pixel kernels in space changes over time, from which the displacement vectors of surface points can be determined as a time series function at sub-pixel resolution.

[0067] FIG. 3 illustrates an exemplary method for providing and using a trigger pulse 111 or a common time base according to an advantageous embodiment of the invention. The external time series data, such as ECG, can be gathered and used as said trigger pulse 111 or a common time base for synchronizing or linking at least two or more (sequential) images and/or or other measurement data from at least one of said imaging device or other device with each other. Using of the trigger pulse 111 or a common time base allows determining the changes in said deformation between the two or more (sequential) images either in combination or together with said other (external) measurement data. Said external time series data can be gathered, e.g., from at least one additional external measuring device measuring or monitoring vital signs of the subject so the patient undergoing open cardiac surgery. An example of the external measuring device is, e.g., an electrocardiography (ECG) device, but naturally it can also be any other suitable device measuring vital signs, from which the time series data can be gathered, such as devices for measuring or determining heart rate, blood pressure, central venous pressure, pulmonary artery pressure, pulmonary artery occlusion pressure, inspired and expired gases, oxygen saturation of the blood, cardiac output, ultrasound or tissue doppler imaging data or other devices for accurate monitoring and quantification of cardiac cycle events.

[0068] For an object changing its shape (and position) over time (such as heart beating), where the deformation cyclically increases and decreases as function of time t, the correlation coefficient r of the cross correlation function normally decreases with increasing deformation, making the determination of displacement vectors more inaccurate and prone to bias when using a fixed reference image. For a series of n images, the cross correlation coefficient r is typically minimized for consecutive image pairs n/n+1, which is utilized in the sum of differentials routine. However, errors in the determination of displacement vectors cumulate when using this routine, and the cumulative error therefore increases with increasing length of the measurement series. Using the sum of differentials routine and re-assigning the zero-displacement reference frame via an external gating signal, such as ECG, is an optional solution to minimize inaccuracy for an analysis time series exceeding the duration of one cardiac cycle, as is described now in FIG. 3.

[0069] FIG. 4 illustrates an exemplary correlation for an amplitude of monitored deformation change of the surface of the heart during a volume therapy according to an advantageous embodiment of the invention. Patients undergoing open heart surgery are routinely artificially ventilated, using a positive (or negative) pressure ventilator device. Adjustment of the ventilator settings with respect to frequency, gas mixture, and especially pressure affects right ventricle function. The most adverse outcome of non-optimal ventilator settings are right ventricular (RV) failure and dysfunction.

[0070] Ventilator level steering using both amplitude and amplitude modulation values of displacement or strain data delivered by the present invention improves the adjustment of optimum level and reduces the risk of choice of detrimental settings. Ventilator settings can be adjusted either manually based on the indicative parameters, or via an automated feedback control loop, where the targeted performance level of the right ventricle is set, and ventilator settings adjusted incrementally in order to reach the target.

[0071] The same principle applies also to an Intravenous Fluid Therapy, which is carried out to prevent dehydration, i.e., deficit of total body water. Fluid replacement is provided by intravenous infusion. Fluid overload occurs when fluids are given at a higher rate or in a larger volume than the system can absorb or excrete. Possible consequences include hypertension, heart failure, and pulmonary edema. The measurements and estimates provided by the embodiments of the present invention can generate accurate information about the effect of the fluid therapy on the patient's heart and indicate conditions of hypo/hyper infusion to the operating team, for instance, by indication of deviation from optimal fractional shortening values or principal strain values. The amplitude representing said deformation is illustrated in FIG. 4 in a function of volume, where it can be clearly seen that when the intravenous infusion is increased also the amplitude increased to a certain acceptable level, but beyond a certain point 112, where the fluids are given at a higher rate or in a larger volume than the system can absorb or excrete, the change of the deformation is not in the range or limit anymore and the amplitude representing the change of the deformation changes dramatically. This can be clearly seen in the point 112 in FIG. 4, which data can also be used for providing control data or signal to the infusion machine, for example.

[0072] FIGS. 5-7 illustrate examples of displacement and deformation data time series according to an advantageous embodiment of the invention. By linking the series of images through a common time base to other physiological monitoring measures such as electrocardiography (ECG), heart rate, blood pressure, central venous pressure, pulmonary artery pressure, pulmonary artery occlusion pressure, inspired and expired gases, oxygen saturation of the blood, cardiac output, cerebral activity, neuromuscular function, other ultrasound or tissue doppler imaging data etc., accurate monitoring and quantification of cardiac cycle events in the right ventricle is possible and can be put into context of other vital sings and measures. Extracting displacement data time series functions x.fwdarw.f(t) enables visualization and quantification of deformation pattern characteristics with respect to amplitude, amplitude modulation, periodicity, aperiodicity, wavelength, frequency, phase, rhythm or other features and their combination, which can be interpreted either visually-cognitively by a clinician in respect of clinical characteristics, possibly associated with certain diseases or conditions, or by applying an automated pattern recognition routine, and steer or guide the settings of life support machines, equipment and measures such as cardiopulmonary bypass pump settings, ventilator/respirator settings (volume, pressure, flow), infusion pumps etc.

[0073] In particular FIGS. 6 and 7 illustrate tissue displacement and velocity data gathered from the right ventricle (RV) by the current invention and from the left ventricle (LV) by the TEE tissue Doppler method. The waveforms of the invention resemble the waveforms obtained by TEE tissue Doppler, showing the same characteristic cardiac cycle events and amplitudes, whilst acquisition is faster, easier, and of higher accuracy.

[0074] The invention has been explained above with reference to the aforementioned embodiments, and several advantages of the invention have been demonstrated. It is clear that the invention is not only restricted to these embodiments, but comprises all possible embodiments within the spirit and scope of the inventive thought and the following patent claims. The features recited in dependent claims are mutually freely combinable unless otherwise explicitly stated.

[0075] For example, even if optical or near optical electromagnetic radiation cameras are described as an example, also other type of cameras can be used, such as infrared cameras, whereupon also oxygen saturation can be determined during the same measurements. In addition, the monitoring and assessment can be implemented even with one camera, e.g., of a Plenoptic type. Moreover, even if it is said that at least two images are taken, but in practice typically at least 100 images are taken in order to get better reliability. In addition, the images taken into the process can be consecutive, but also single images from the imaging sequence can be ignored. Still in addition it is to be noted that in accordance to an embodiment also minutes of the operation can be provided with the information and assessment data gathered by the current invention automatically.

[0076] In addition, even if the imaging device (102) is described, the arrangement may comprise just an imaging input device for receiving image data from the outer imaging device. Furthermore, even if the at least one surface point 113 is disclosed from which the two or more images are taken, in practise it is a pattern (comprising a number of points), which is imaged. However, it is to be understood that in some situations also one point to be imaged is enough, when e.g. the speed, acceleration and/or direction of said point is determined in function of time (cycles of the heart).

[0077] Still, in addition, it is to be noted that according to an embodiment the arrangement can be implemented by the imaging device 102 as such, such as by a camera (advantageously comprising same elements or devices as said arrangement described above, such as described in connection to FIG. 1), wherein said imaging device is configured to obtain at least two or more images of at least one surface point on a surface of the anatomic object over at least one fraction of a cardiac cycle from the surface of the heart. The imaging device comprises advantageously an output for a display device or even comprises said display device. In addition the imaging device comprises a processing unit (or interface to an external processing unit) for determining (advantageously by tracking) deformation based on movements of the at least one surface point over said at least one fraction of the cycle between said at least two or more images of the surface of the anatomic object in function of time. The imaging device may also comprise elements, such as software or hardware elements, for determining changes in the deformation or for comparing determined deformation to reference deformation values to find any deviation in said performance or state of the deformation determined (or a feature changing during a cycle but in a normal state being regular and sequentially repetitive over several cycles), and providing control data for controlling the display device to display the changes in said deformation or said deviation or state of the deformation determined in function of time. As can be understood the imaging device may function as a master imaging device and comprise interface for receiving additional image information from other imaging devices connected with.

[0078] Furthermore, even if the examples above are related to the determination of the heart, it is to be understood that also other kinds of anatomic or muscle deformations, such as lungs deformation due to breathing or muscle deformations due to muscle movements, can be tracked and determined in order to monitor and assess the performance of those anatomic objects.