Abstract
An apparatus to detect blood clots based on the analysis of the blood's chromatic properties is described. The chromatic property can be determined non-evasively when used in conjunction with ECMO systems. The red, green, and blue chromatic values of a clotting site and a reference site are compared to determine if a clotting event occurred. It was discovered that, at a minimum, only the red chromatic value needs to be tested and measured to determine if a clotting event had occurred. This system can be adopted to monitor clot formation in heart surgery, heart or lung transplants or patients coupled to ECMO requiring an ability to measure the clots to a blood depth of 20 mm. Once clots are detected, the system can introduce anti-coagulants into the blood stream to reduce the clot formation.
Claims
1. A system for a measurement of clot formation in blood comprising: at least one light source arrangement providing an electromagnetic radiation bandwidth operating in a visible spectrum range, an infrared spectrum range, or both spectrum ranges; a light transmission arrangement for channeling the electromagnetic radiation through, or reflected from, the blood; one or more light detection arrangements receiving the channeled electromagnetic radiation from the light transmission arrangement to capture amplitudes over a frequency range of the channeled electromagnetic radiation; and a computation device arrangement computing spatial, temporal, or spatial and temporal analysis on the captured amplitudes corresponding to a pixel data output value of the light detection arrangement, wherein when the pixel data output value exceeds a reference pixel data output value, an action is performed.
2. The system of claim 1, wherein the action that is performed is selected from the group consisting of injecting an anti-coagulant, changing blood flow rate, raising a flag, issuing an alarm, raising temperature, and lowering temperature.
3. The system of claim 1, wherein two or more light detection arrangements are positioned around a volume of the blood to collect the channeled electromagnetic radiation, each capable of detecting a clotting event, the two or more of the light detection arrangements each receives a different component of the channeled electromagnetic radiation.
4. The system of claim 1, wherein the blood being measured flows within either a cannula of an extracorporeal blood circulation system coupled to a patient, a vein of the patient, or an artery of the patient.
5. The system of claim 1, wherein hardware for the computation device arrangement is selected from the group consisting of a field programmable gate array (FPGA), a multi-core central processing unit (MC-CPU), a graphics processing unit (GPU), and a machine learning (ML) device.
6. The system of claim 1, wherein the light detection arrangement, further comprises: a detector comprised of a plurality of pixels arranged in rows and columns on a planar surface; and a lens to focus the channeled electromagnetic radiation onto the plurality of pixels, the radiation incident substantially perpendicular to the planar surface.
7. The system of claim 6, wherein each pixel is subdivided into a red, a green, and a blue sub-pixel, and blood clotting can be detected by using the red sub-pixel data of the pixel data output value from the light detection arrangement.
8. A system for a measurement of clot formation in blood of a patient comprising: at least one light source arrangement providing an electromagnetic radiation bandwidth operating in a visible spectrum range, an infrared spectrum range, or both spectrum ranges; a light transmission arrangement for channeling the electromagnetic radiation through, or reflected from, the blood of the patient; and two or more light detection arrangements are positioned around a volume of the blood to collect the channeled electromagnetic radiation, each capable of detecting a clotting event, the two or more of the light detection arrangements each receives a different component of the channeled electromagnetic radiation from the light transmission arrangement and capturing amplitudes within a frequency range of the channeled electromagnetic radiation, wherein when a pixel data output value of the radiation exceeds a reference pixel data output value within any of the light detection arrangements, a clotting event has been detected.
9. The system of claim 8, wherein once the clotting event has been detected, perform an action that is selected from the group consisting of injecting an anti-coagulant, changing blood flow rate, raising a flag, issuing an alarm, raising temperature, and lowering temperature.
10. The system of claim 8, wherein the blood being measured flows within either a cannula of an extracorporeal blood circulation system of the patient, a vein of the patient, or an artery of the patient.
11. The system of claim 8, wherein hardware to perform the comparator operation is selected from the group consisting of a field programmable gate array (FPGA), a multi-core central processing unit (MC-CPU), a graphics processing unit (GPU), and a machine learning (ML) device.
12. The system of claim 8, wherein the light detection arrangement, further comprises: a detector comprised of a plurality of pixels arranged in rows and columns on a planar surface; and a lens to focus the channeled electromagnetic radiation onto the plurality of pixels, the radiation incident substantially perpendicular to the planar surface.
13. The system of claim 8, wherein the computation device arrangement, further comprises: a memory to store a plurality of pixel data output values; and a computation device arrangement computing spatial, temporal, or spatial and temporal analysis on the captured amplitudes corresponding to pixel data output value of the light detection arrangement, a comparator to compare a reference pixel data output value and the channeled electromagnetic radiation of the blood of the patient.
14. A method for detecting and correcting a formation of a blood clot in blood comprising the steps of: providing at least one light source arrangement that provides an electromagnetic radiation bandwidth operating in a visible spectrum range, an infrared spectrum range, or both spectrum ranges; channeling the light through, or reflected from, the blood of the patient; capturing amplitudes over a frequency range of the channeled electromagnetic radiation; and computing spatial, temporal, or spatial and temporal analysis on the captured amplitudes corresponding to a pixel data output value of the light detection arrangement, wherein when the pixel data output value exceeds a reference pixel data output value, an action is performed.
15. The method of claim 14, wherein the action that is performed is selected from the group consisting of injecting an anti-coagulant, changing blood flow rate, raising a flag, issuing an alarm, raising temperature, and lowering temperature.
16. The method of claim 14, wherein two or more light detection arrangements are positioned around a volume of the blood to collect the channeled electromagnetic radiation, each capable of detecting a clotting event, the two or more of the light detection arrangements each receives a different component of the channeled electromagnetic radiation to detect a clotting event.
17. The method of claim 14, wherein the blood of the patient being measured flows within either a cannula of an extracorporeal blood circulation system, a vein of the patient, or an artery of the patient.
18. The method of claim 14, wherein hardware to computing spatial, temporal, or spatial and temporal analysis is selected from the group consisting of a field programmable gate array (FPGA), a multi-core central processing unit (MC-CPU), a graphics processing unit (GPU), and a machine learning (ML) device.
19. The method of claim 14, further comprising the steps of: arranging a plurality of pixels in rows and columns on a planar surface; and focusing the channeled electromagnetic radiation onto the plurality of pixels using a lens, the radiation incident substantially perpendicular to the planar surface.
20. The method of claim 19, wherein each of the plurality of pixels comprises at least a red, a green, and a blue sub-pixel, the pixel data output of the light detection arrangement, at a minimum, only requires the response corresponding to the output of the red sub-pixel (wavelengths 625-740 nanometers) to detect blood clotting.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
[0039] Please note that the drawings shown in this specification may not necessarily be drawn to scale and the relative dimensions of various elements in the diagrams are depicted schematically. The inventions presented here can be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art. In other instances, well-known structures and functions have not been shown or described in detail to avoid unnecessarily obscuring the description of the embodiment of the invention. Like numbers refer to like elements in the diagrams.
[0040] FIG. 1A depicts one embodiment of a static RGB Detector in accordance with the present disclosure.
[0041] FIG. 1B illustrates another embodiment of a static RGB Detector in accordance with the present disclosure.
[0042] FIG. 1C shows an embodiment of a dynamic RGB Detector in accordance with the present disclosure.
[0043] FIG. 1D presents a biological specimen coupled by blood flow to the embodiment presented in FIG. 1C in accordance with the present disclosure.
[0044] FIG. 1E depicts a block diagram of a blood clot sampling and corrective system in accordance with the present disclosure.
[0045] FIG. 2A shows the mechanical system used to measure coagulation in accordance with the present disclosure.
[0046] FIG. 2B presents a map of the clotting and reference sites in accordance with the present disclosure.
[0047] FIG. 2C illustrates a block diagram of the comparison system for FIG. 2A in accordance with the present disclosure.
[0048] FIG. 2D shows the RGB values of the reference site and the clotting site with a 2x diluted hemostatic agent in accordance with the present disclosure.
[0049] FIG. 2E depicts the RGB values of the reference site and the clotting site with blood depth of 5 mm in accordance with the present disclosure.
[0050] FIG. 2F presents the test results of the measured R (red) value in the reference sites and the clotting sites in accordance with the present disclosure.
[0051] FIG. 3A depicts the block diagram of the electronics used to measure coagulation in accordance with the present disclosure.
[0052] FIG. 3B illustrates a process flow of detecting and correcting for blood clots in accordance with the present disclosure.
[0053] FIG. 3C presents a block diagram of an ECMO system with anti-coagulant in accordance with the present disclosure.
[0054] FIG. 4 shows the RGB detector in relationship to the cannulas (tube) in accordance with the present disclosure.
[0055] FIG. 5A depicts the cross sectional view of one embodiment of flexible LED and flexible sensor arrangement in accordance with the present disclosure.
[0056] FIG. 5B illustrates the cross sectional view of another embodiment of multiple LED and multiple sensor arrangement in accordance with the present disclosure.
[0057] FIG. 5C presents the cross sectional view of an embodiment of an embedded LED and embedded sensor arrangement within the cannulas in accordance with the present disclosure.
[0058] FIG. 6 depicts the process flow of a self-correcting coagulant system in accordance with the present disclosure.
[0059] FIG. 7A presents a spatial view of a small blood clot within the field of view (FOV) in accordance with the present disclosure.
[0060] FIG. 7B illustrates the spatial view of the small blood clot a short time later within the field of view (FOV) in accordance with the present disclosure.
[0061] FIG. 7C depicts the spatial view of the small blood clot after another short time interval within the FOV in accordance with the present disclosure.
[0062] FIG. 8A illustrates a block diagram view of an embodiment of RGB database accessed temporally and spatially in accordance with the present disclosure.
[0063] FIG. 8B presents a block diagram view of another embodiment of an multi-time-space/computation blood clotting machine in accordance with the present disclosure.
[0064] FIG. 9 depicts a flow chart the RGB Detector using machine learning in accordance with the present disclosure.
[0065] FIG. 10 presents a block diagram of a machine training/learning system in accordance with the present disclosure.
DETAILED DESCRIPTION OF THE INVENTION
[0066] FIG. 1A illustrates a system 1-1 that transmits light 1-18 from a light source 1-2 emitting white light through a static blood sample 1-3 where static implies stationary blood. Separate color detectors (Red, Blue, and Green) are used to measure each of the three primary colors after the sample has been illuminated by the light source. The detectors 1-4 to 1-6 can provide information on a spatial, a temporal, or a combination of a spatial and a temporal matrix pattern. A first blood sample is measured and stored in memory as the reference sample. A second blood sample is introduced and then measured, but first, a coagulant is added to the second blood sample. A series of timed images are captured by the detectors and stored into memory. These timed images provide information on the condition of the blood. For example, at a frame rate of 750 fps, an image is captured every 1.3 ms. Each of these images can be compared to the reference to determine the condition of the blood. If the third image captured at 2.6 ms indicates that no significant difference in transmissivity exists when compared to the reference slide, then any effects of the coagulant on the blood sample have not been detected yet. However, after a quarter million frames (300 s), the transmissivity changed from 130 units to about 80 units. The extracted color information is provided to the computer 1-7 which can analyze the results using different algorithms. The spatial and temporal algorithms can perform moving averages, moving average over time, moving averages over space, and processes and techniques similar to those used in radar processing. All of the data can be stored in memory and the accumulated databases can be shared with the Internet or stored in the Cloud 1-8. The computer can also offload any algorithmic computations onto the Cloud, as well. The components within the dotted box 1-16: light source 1-2, blood sample 1-3, and the detectors 1-4 to 1-6 comprise an RGB Detector.
[0067] FIG. 1B depicts another embodiment of the RGB Detector 1-16 with a detector 1-10 and light diffuser 1-17. The light diffuser spreads out and diffuses the light preventing the camera from being saturated when there is too much light. The detector can be an area array of light sensors. The light sensors are arranged in rows and columns. For example, a CCD camera or a CMOS camera can be used as detectors. These cameras are fabricated in one of many semi-conductor processing lines, the pixels are arranged on the planar surface of the processed semi-conductor chip. Each x-y array pixel in these cameras is comprised of at least three different sub-pixels. In one embodiment, the first sub-pixel captures the R (red) color, the second sub-pixel captures the G (green) color, and the last sub-pixel captures the B (blue) color. Other sub-pixel combinations are used; some configurations depend on the manufacturer. The measured output of the detector is comprised of results of the three sub-pixels. The results of the RGB can be segregated from each other and analyzed independently or they can be combined in different proportions and analyzed together. The inventors have discovered that the R (red) color is highly correlated to the blood clotting event. The other two primary colors: G (green) and B (blue) show reduced response to the blood becoming clotted. However, the large changes to the R results correlate with the measured blood clotting events. In one embodiment, only the R pixels are sampled, viewed, and analyzed to make an assessment of the amount of blood coagulation or blood clotting that has occurred.
[0068] FIG. 1C shows another embodiment of the RGB Detector 1-16. The blood sample 1-11 is now dynamic and its image is focused using a lens 1-18. The focused image is captured by pixels within the array of the detector. The blood sample images comprise the blood while the blood is moving. In one embodiment, the blood is being circulated in a loop within the system. For example, in ECMO, the cannula, being transparent, visibly shows the moving blood stream. The blood stream is actively monitored for any clotting events. New samples of blood arrive continuously within the cannula. A computer adjusts the intensity of the light source and the detector registers the RGB color content instantaneously. The R detector (corresponding to the red pixel) is very sensitive to any clotting events forming in the stream. The change in the R color's transmissivity as a function of clotting events can be used to quickly identify blood coagulation.
[0069] FIG. 1D presents a biological specimen 1-12 coupled via blood flow 1-13 to the dynamic blood sample 1-11. The specimen's blood flow is monitored by the detector. In particular, the R reading is monitored to note if any coagulation is occurring. The system in FIG. 1D can be used to identify clotting events in a patient's blood flow. Once the R reading exceeds a pre-defined threshold level, the system is defined as experiencing blood clots.
[0070] FIG. 1E illustrates a detection and correction system to maintain the blood clot levels within a given tolerance range. The blood flow of the biological specimen is constantly monitored. The detector measures to determine when a pre-defined threshold level is exceeded. This indicates the onset of a blood clot. Once the blood clot has been detected, the computer applies a signal to the anti-coagulant substance 1-14 which injects 1-15 a set amount of anti-coagulant such as Heparin, xarelto, pradaxa, eliquis, lixiana, etc. into the blood stream. The anti-coagulant can be introduced into the patient through a newly formed intravenous port or use an intravenous port already in existence. After a short time period, the blood is monitored again. If the detector continues to measure an exceeded threshold level, inject another set amount of anti-coagulant and repeat the test. If the detector measures a value below the threshold value, the blood clotting events has been controlled. The system continues to monitor the blood flow for blood clot formation. Another embodiment of the system can include a comparison to decide if the clotting value is approaching one or more alarms. An upper threshold clotting level can be set to issue a critical alarm. This upper clotting level alarm indicates that the patient may be approaching death and imminent corrections are required.
[0071] FIG. 2A depicts one embodiment of a static blood clotting detector apparatus used to obtain measurements using a high frame rate camera on blood during the coagulation event to determine the changes in red, green, and blue (RGB) values. Each of the pixels in the camera array is sub-divided into a plurality of sub-pixels. In one embodiment, the pixel is sub-divided into three primary colors: red (R), green (G), and blue (B) sub-pixels. To detect the onset of coagulation, these three sub-pixel properties can be captured by a high frame rate camera, which offer high resolutions and a wide dynamic range. The high frame rate (750 f/s) camera can measure the response of each of the three primary colors individually to the coagulation event. The blood flows through the tubes very rapidlyup to several centimeters per second. To detect a small blood clot that is quickly passing through the field of view requires a detector (camera) having a high frame rate image capturing capability. At least two benefits occur by collecting the plurality of measured results. First, the data can be used to reduce the noise by using various techniques; such as averaging, and strengthen the signal, thereby improving the signal to noise ratio and overall measurements. Second, the data provides samples for a global database that can be used in improving the machine learning systems.
[0072] The apparatus includes a camera 2-5 and optic lens 2-10. The camera's output is provided to a computer 1-7. The camera is a Basler model number acA640-750 um which can take up to 750 frames per second. The sensor area of the CMOS camera is 3.1 mm2.3 mm while the pixel area occupies an area of 4.8 m4.8 m. An Edmund Optics Lens 59870 is used to focus the light. It has a Field of View (FoV) of 61.4 mm and a max sensor format of 30.9. Lampire Laboratories provided the sheep's blood 2-3 placed in the reflector tube 2-4, having a sodium heparin content of 1000 units/mL and a shelf life of 10 days at 4 C. The heparinized blood makes it possible to control the coagulation time and obtain results under various conditions (as explained below). This checks the consistency of the experiments. A white LED 2-1 provides the source lighting whose intensity can be controlled by a computer 1-7. A light diffuser 2-2 diffuses the light from the LED to avoid saturating the camera while the reflector tube helps to show greater blood depths.
[0073] Shown in FIG. 2B are five clotting sites within 250 m of each other and two reference sites that are 5 mm and 7.5 mm away from the clotting sites. For our experiments, a drop of the hemostatic agent is added to the blood samples in the five left clotting sites. The camera measures, in real time, the RGB values of the pixels in these sites. A reference site about 100 pixels (550 m) away is also measured using the RGB values. To reduce variations, an average of the clotting value is taken over 100 samples every 2 milliseconds.
[0074] FIG. 2C illustrates one embodiment of the test setup that was used. To start the coagulation, a Frenna hemostatic agent which has a 25% aluminum chloride content is added to the clotting site samples. Other clotting agents, such as Celox, have a granular form, which proved to have poor precision as compared to the Frena solution. The blood sample 1-3 is illuminated from below by a white 30-W LED light source 1-2.
[0075] In the one of the embodiments of the system, a compact optical device comprising the light source and detector will be designed to conserve space. The device can be miniaturized to a point where the device can be inserted into the cannula or into the patient's blood vessels.
[0076] The light passes through the static blood sample 1-3 and the altered Light is captured by the RGB pixels of the detector 1-10 as a clotted image. The altered light, or channeled electromagnetic radiation, is comprised of any transmitted components and any reflected components from the blood sample. A memory 2-7 holds the reference image of the reference site. This reference image is recalled from memory and compared in the comparator 2-6 against the clotted image from the detector. Software in the computer 1-7 or available in the Cloud/Internet 1-8 analyzes the results of the comparison and decides where the clotting value stands with regard to the threshold clotting level alarm. The clotting level alarm indicates that the patient has exceeded the allowable threshold level and that a blood clot formation has been found indicating that some action is required.
[0077] The system was tested in five conditions: with hemostatic agent without dilution, with 2x dilution, and with 4x dilution, as well as greater blood depths of 5 mm and 10 mm. In each case, the RGB values for both the clotting site and reference site were collected. Two sets of the comparative results are plotted in FIG. 2D and FIG. 2E. The vertical scale has a maximum of 255. The clotting agent is added at time zero, and the RGB values are captured for the next 1000 seconds. The objective was to see if there exists a clear and distinct difference between the clotting and reference sites in terms of red, green, and blue values. For greater blood depths, the light diffuser and reflector tube are adjusted to illuminate the blood optimally. As a result, the maximum red value varies to some extent across the measured results.
[0078] FIG. 2D presents measured results of a reference site and a clotting site after a 2x diluted hemostatic agent was added to the clotting site. Note that the Green and Blue results show little or no difference between the two cases. As illustrated in the lower part of the figure, the actual image of the reference site and the actual image of the clotting site are applied to the comparator 2-6. The computer 1-7 or Cloud (not shown) compares the two results and determines the clotting level based on the threshold value.
[0079] FIG. 2E illustrates another set of measurements. The results of the reference site and the clotting site after the light penetrated through 5 mm of blood. Note that the Green and Blue results show little or no difference between the two cases. The Red (R) case does show a difference. As illustrated in the lower part of the figure, the stored image of the reference site is extracted from memory 2-7 and the actual image of the clotting site are applied simultaneously to the comparator 2-6. The computer 1-7 or Cloud (not shown) compares the two results and determines the clotting level based on the threshold value.
[0080] Even in the scenario of 10 mm of blood depth, where the difference in red values between the clotting and reference sites was comparatively small, there was a contrast of around 40%. This depth was chosen to simulate the conditions of ECMO, where the tube radius is around 10 mm. This contrast can be improved by using a brighter LED or adding a lens between the LED and the diffuser.
[0081] Although only two sets of results were presented, data was monitored and collected from other sites as well. The additional tests conducted included using an undiluted hemostatic agent, a 4x diluted hemostatic agent, and blood depth tests of 10 mm. In all of these measurements, the observation was that the red value is a consistent indicator of coagulation and drops far more than the blue and green values. As can be observed in the graphs above, the time taken for the clotting site red value to drop is proportional to the dilution factor of the hemostatic agent. More importantly, the red in the clotting site undergoes a more significant change than that of the reference site. Results have shown that the transmissivity of the primary red color shows a strong relationship to the formation of blot clots. The transmissivity of the other two primary colors (green and blue), comparatively speaking, change little during the coagulation event.
[0082] FIG. 2F plots only the red values for the clotting and reference sites illustrated in FIG. 2B. The variation is about five percent for clotting sites and about seven percent for reference sites before the passage of 950 seconds. The two reference sites 2-8 have relatively constant red values of about 105. Note that the red values of the 5 clotting sites 2-9 experience a dramatic change in the red value starting at about 130 at zero time and ending at about 60 at about 350 seconds.
[0083] FIG. 3A illustrates the electronic block diagram 3-1 of the clot detecting system. The three components: the LED 3-3, the transmitted light after passing through the Blood Sample 3-4, and the Sensor 3-5 detecting the transmitted light make another embodiment of the RGB detector 1-16. The electronic system 3-2 comprises the control electronics 3-6 which activates, coordinates, and captures the data created between the LED and the sensor of the RGB detector. A flow chart illustrates the sensor 3-5 providing data to the R G B extraction block 3-7. The primary colors are extracted as a function of time and/or a function of position within the pixel array. The results are calibrated 3-8 against a reference. The reference could be a live image or a stored image as described earlier.
[0084] After calibration, the signal is filtered 3-9 to extract out the occurrence of coagulation from a space/time matrix result. The detection/decision 3-10 evaluates the filtering result and can provide the user with data about the size, position, and/or velocity of the blood clots. For example, in the comparison of one x-y array image against the reference image, various areas are darkened out indicating the locations of clots. Once the clots have been identified, each can be measured for location, width, height, closest neighbor, etc. Between additional timed measurements, the captured data can be used to calculate clot size, clot grouping size, clot velocity, etc. In one embodiment, the width of the detector spans the diameter of the cannula carrying the flow of blood.
[0085] A closed blood loop anti-coagulant system using the clot detecting system of FIG. 3A is depicted in the flowchart of FIG. 3B. The process starts 3-17, the RGB detector 3-18 comprising the light source (LED) and video recording source (Camera) continuously monitor the red, green, blue values of the transmitted light through the blood within the aperture of view. Note that earlier measurement results favor the red response as being the indicator of when blood clots occur.
[0086] The visible spectrum ranges from wavelengths from about 380 (violet) to 740 (red) nanometers. Approximately, Red occupies the wavelengths 625-740 nanometers, Green occupies 500 to 565 nanometers, and Blue occupies 450 to 485 nanometers. The measured results indicate that the Red response experiences the largest change in its transmissivity through the blood sample. These changes occur at the Red wavelengths of 625-740 nanometers. By monitoring the Red wavelength bandwidth of transmitted light through blood samples, comparative measurements against a reference can be performed to detect blood clot formations.
[0087] Similarly, the visible spectrum extends from frequencies of about 480 (red) to 680 (violet) terahertz. Approximately, Red frequencies in visible spectrum extend from 405-480 THz, the Green frequencies range from 530-600 THz, and the Blue frequencies range from 620-680 THz. A frequency range of the visible spectrum can be monitored, for example, the Red frequency range of 405-480 THz can be evaluated. By monitoring the Red frequency range of transmitted light through blood samples, comparative measurements against a reference can be performed to detect blood clot formations.
[0088] Once clotting has been detected, a decision 3-19 is necessary to check if the Red color exceeds the threshold value. If the threshold value is exceeded, apply anti-coagulant 3-20 that can comprise: heparin, xarelto, pradaxa, eliquis, lixiana, etc.
[0089] FIG. 3C depicts one embodiment of a closed loop RGB detector coupled to an ECMO that maintains the concentration of blood clots to remain in the vicinity of a pre-set threshold value. The ECMO system portion comprises the cannula A, the bladder box 3-16, the cannula B, the cannula Cl, cannula F, the blood pump 3-13, cannula G, the oxygenator 3-14, the cannula H, the heat exchange 3-15, the cannula I, and the patient (all dotted arrowed lines form a closed circular blood loop). Assume that the patient requires the ECMO system, for example, they are receiving a new lung. Blood is extracted from the patient on cannula A and moves into the bladder box 3-16. The fill capacity of the bladder box enables the blood pump. If the box is empty, there is nothing to pump, so disable the blood pump. If the box is full, there is much to pump, so enable the blood pump and move the blood through the remainder of the system. The bladder box prevents an excess negative pressure from occurring at the inlet to the blood pump. Since the patient is receiving a new lung, the oxygenator 3-14 is behaving like a lung by adding oxygen to the blood flow and removing carbon dioxide from the blood flow. The heat exchange 3-15 warms the blood to the correct body temperature. Finally, the oxygen rich blood is feed back to the patient using cannula I. Note the many access ports where an RGB Detector can be located via the many cannulas in the system.
[0090] The closed loop RGB detector/comparator is located in the lower left of FIG. 3C. Two of its components: the RGB Detector 1-16 and the injection port 3-12 have been inserted into the blood path between cannula B and cannula F. These two components are not required to be next to one another or need to be necessarily located between cannula B and cannula F. The RGB detector and the injection port can be located anywhere along the blood path where there is an accessible external cannula from which blood samples can be monitored or substances can be infused. The RGB detector monitors the clot formation in the blood flow and provides that information to the computer 1-7 and the comparator 2-6. The computer, the comparator, and the reference image from the memory 2-7 are used to decide what the clotting level currently is. If the blood clot level is less than the threshold level, do nothing. However, if the blood clot level is greater than the threshold level, activate the anti-coagulant substance 3-11 and inject some of this substance into the inject port 3-12 and into the blood flow. Continue monitoring the patient, wait till the anti-coagulant distributes in the patient, then follow the previous two steps.
[0091] The threshold level is set to detection of small clots of around a few hundred micrometers in size to determine the early detection of blood coagulation. To achieve consistent results, more than 40 experiments were performed in which the RGB data was collected and the system was refined. Similar experiment can be performed to detect smaller clots and larger clots and to use all this data to map out a threshold level versus clot size chart.
[0092] In critical operations, where the use of extracorporeal blood circulation system is required, the cannula is an external tube that is inserted into the patient's veins or arteries to provide an entry/exit blood port on the patient. Additional cannulas are used to couple external equipment together creating a blood loop between the entry/exit blood ports of the patient. These additional cannulas in the system (for example, see B, F, G, H, and I in FIG. 3C) provides easy external access to the blood flow of the patient. The easy access is for way to couple in both the injection port 3-12 and the RGB detector 1-16 into the blood loop. In one embodiment, the RGB Detector is coupled to a cannula while the injection port uses the patient's intravenous setup that already has been established for the patient's procedure.
[0093] In another embodiment, the injection port and RGB detector are combined into one unit, such that, an ingress exists for the introduction of an anti-coagulant into the blood flow and the blood flow can be observed using the light source and detector. Such a structure is illustrated in FIG. 4. The cannula 4-2 carries a blood flow 1-13 and the cannula is coupled to an RGB Detector 1-16. The dotted line 4-1 indicates the location of the cross sectional view as seen from the perspective of the arrow 4-3.
[0094] FIG. 5A-C illustrates several embodiments of these views along the dotted line 4-1 presented in FIG. 4 detailing the cross section between the LED, sensor, and the blood flow. The cannula 4-2 is a transparent tube that allows the transmission of visible light.
[0095] FIG. 5A illustrates a ring fixture 5-2 around the cannula and mounted on the ring fixture is a flexible LED 5-1 and flexible RGB light detector 5-3. The flexible material is a flexible plastic substrates that carries all required semiconductor electronics (LEDs, camera, amps, op amps, etc.) formed on a polyimide or a transparent conductive polyester film. In another embodiment, the flexible LED and RGB detector can be attached directly to the outside surface of the cannula (not shown) conserving space and removing the need for the ring fixture. Note that the detector and the LED are positioned on opposite out-sides of the cannula. The RGB detector can be a camera having some of the standard pixel sizes as used in industry. The tail of the arrow 1-13 indicates that the blood is flowing into the page. A wireless system to detect alarms and allow for additional controls to the system can be added to the detector.
[0096] The operation of FIG. 5A follows. As blood passes between the LED 5-1 and the sensor 5-3, light from the LED passes through the material of the cannula 4-2 and directly into the path of the blood flow. The transmitted light is altered after passing through the blood if the blood flow contains blood clots. The altered light then passes through the material on the opposite side of the cannula and into the sensor 5-3. The sensor analyzes the image for temporal and/or spatial information that may indicate the occurrence of blood coagulation. The comparator 2-6 compares the measured results with the reference extracted from the memory 2-7. The results are applied to the computer 1-7 which uses one of several algorithms to determine the characteristics of the one or more blood clots being analyzed. These characteristics include clot size, clot cluster size, clot position, clot size growth, clot cluster size growth, clot velocity, etc. Cluster size is a grouping of individual clots into tight clusters.
[0097] Another embodiment of the RGB electronic components combining with the cannula is presented in FIG. 5B. The ring fixture can be used to hold the components, or the components can be placed onto flexible material and then attached to the outside circumference of the cannula. The plurality (n) of LED-Sensors can be interleaved around the circumference of the cannula. FIG. 5B illustrates one case of where n=2. The light from LED 5-1 can be detected by the sensor at 5-3, the sensor 5-5, or both simultaneously. Similarly, the light from the second LED 5-4 can be detected by the sensor at 5-3, the sensor 5-5, or both simultaneously. One or more of the plurality of LEDs can be different than all the rest (red color, blue color, power, efficiency, etc.). One or more of the plurality of sensors can be different (technology types: CCD, CMOS).
[0098] Another embodiment of the RGB electronic components combining with the cannula is presented in FIG. 5C. The LED 5-1 and the sensor 5-3 are inserted directly into the cannula 4-2. In another embodiment, the flexible circuits can be used (not shown) to print the LED and sensor onto the inside surface of the cannula. Note that the detector and the LED are positioned on opposite in-sides of the cannula. In another embodiment, the flexible circuit may contain a wireless power supply. By applying a varying magnetic field outside the cannula, the wireless power supply generates a voltage to power the LED and sensor. The placement of the plurality of LEDs and sensors can also be interwoven along the inner circumference of the cannula. Algorithms within the computer can be used to collect the data from all the interwoven sensors, and use the data to extract information about the clot formation, its size, the number of members within a group, etc. An additional wireless system can be included to detect alarms. The wireless system can also allow for additional control capability to the system via a smart phone or tablet.
[0099] FIG. 6 presents a flow chart describing the operation of an ECMO controlled by a closed loop RGB detector to maintain the clot level at or below a specified threshold. At start 6-1, connect the patient's input/output ports to external equipment 6-2 using cannula. Next, enable the ECMO system 6-3. Once enabled, the blood will start flowing 6-4 in its closed blood loop. The blood loop comprises the output port, the bladder, the oxygenator, heater, the input port, and the internal veins or arteries. Once blood flows, enable the RGB system 6-5, then start monitoring the R, G, and B values 6-6. In some embodiments, only the R value is monitored. The image is photographed and analyzed for the occurrence of blood clots 6-7. Has clotting exceeded the threshold value 6-8? If yes, inject patient with anti-coagulant 6-9 such as, heparin, xarelto, pradaxa, eliquis, lixiana, etc. At wait 6-14, the patient waits till the anti-coagulant has distributed into its system, once complete, move to monitoring 6-6. However, if the threshold had not been exceeded, determine if the procedure is complete 6-10. If not, continue to monitor patient 6-6. Otherwise, if procedure is complete, stop analyzing 6-1, disconnect patient from equipment 6-12 and end 6-13.
[0100] FIG. 7A-C depicts a blood clot 7-2 as it is changing in space and in time within an array 7-1. The array is a two dimensional pixel matrix representing the location of each pixel in the camera. Each pixel is further sub-divided into sub-pixels, where the sub-pixels comprise at least R, G, and B. The pixels in the array are numbered left to right, top to bottom, using Cardinal numbers: 1, 2, 3, . . . . For example, 4 is in the top left while 16 is in the bottom right. The 44 array is a very simple array, as there is a plurality of different pixel sized arrays. For example, the Basler camera used in the present experiments has a pixel array sized as 640 pixels480 pixels. The accuracy of detection, measurement, and position of the clots improves as the array size is increased. The blood clot in FIG. 7A is located in pixels 14-15.
[0101] A threshold value is used in the system to identify a non-clotting patient from a clotting patient. The threshold level can be measured on each patient prior to the procedure and stored in a database. A threshold value is determined by the ratio of the area of the blood clot to the overall area of the total array. For example, the blood clot 7-2 had an area equal to about a of a pixel then the threshold value would be equal to 1/48. Assume that this threshold value identifies the boundary between the patient clotting or not clotting. For any readings below 1/48, do nothing. For any reading above 1/48, inject an anti-coagulant.
[0102] FIG. 7B presents the blood clot at a first later time. The blood flow is in the positive y-direction and the blood clot moved about 1.25 pixels and can be found within pixels 10-11. FIG. 7C presents the blood clot at a second later time. The blood clot moved about 1.66 pixels and can be found within pixels 2-3.
[0103] The photos of FIGS. 7A-C can be translated into number of different databases. One database can include the pixel number and a Boolean result (True if clot in pixel, False if not). This database represents spatial and temporal pixel information. Various algorithms can be applied to the database to better understand the formation of the clot, the size of the clot, the movement of the clot, the structure of the clot, etc. As a though experiment for one possible algorithm, use the positions of the blood clot in FIGS. 7A-C with the following assumption: the sequences presented are equally spaced in time. Since the separation between the previous clot and the current clot is increasing, it appears that the fluid flow is experiencing an increase in acceleration. Working with the pixel locations and time, one can calculate velocities (pix/sec), interpolate positions as a function of time, etc.
[0104] One simple example of a spatiotemporal matched filtering method is a delay and sum approach. Consider the system in FIG. 7. The clot is moving in the y direction. Instead of making the decision (on the presence of the clot) based on a single frame (e.g. FIG. 7A), spatiotemporal matched filtering, in an example embodiment, could combine the signals from multiple frames after taking into account the delay and location shifts due to the movement of the clot along the y axis. The simple example assumes laminar flow of the particle and at a constant velocity, so knowing the time difference of the frames, we could apply the expected shifts in locations (here the pixels) before combining the signals. The shift is due to locational changes across time difference between frames.
[0105] In the embodiment above, assuming laminar flow and with constant known velocity (related to the flow rate), we can combine the signals from pixels 14 (and 15) in FIG. 7A with pixels 10 (and 11) in frame in FIG. 7B, and so on. After the combination of all respective pixels are done, a threshold for detection with multiple frames can be applied, which now has better SNR and lower false alarm probability. Constant False Alarm Rate (CFAR) techniques, similar to ones applied to radar detection, could be used to further enhance the detection method.
[0106] FIG. 8A presents a sequence using the matrix information and arriving at clotting data. The matrix image RGB database 8-1 includes stored and currently viewed images. The pixels can be selected by time 8-2 or by position 8-3 or by both simultaneously. Once the information is combined, the information is filtered. The filtering is a data processing manipulation that uses algorithms and provides results. For example, the data can be filtered in a Matched Filter or a Moving Target filter. The data is sampled, noise averaged, and compared in a threshold detector 8-5. The data that was sampled and measured comprises the clotting data 8-6 can be stored away. Then, in future searches, Machine Learning can use these databases to improve the overall system.
[0107] FIG. 8B depicts another embodiment of a computing machine that can detect and counteract clots events. The matrix image RGB database 8-1 includes stored and currently viewed images. The pixels can be selected temporally or spatially or both simultaneously. The information is then applied to a computation machine 8-7. The computation machine is a special propose machine that is designed to preform many multiplies and adds simultaneously. Some examples of the special purpose machines comprises: a field programmable gate array (FPGA), a multi-core central processing unit (MC-CPU), a graphics processing unit (GPU), and a machine learning (ML) device. The computation machine can use this hardware to perform feature extraction 8-8, statistical processing 8-10, image processing 8-11, array processing 8-9 and video processing 8-12. The result of these calculations creates a database of the clotting data 8-6. The database can be used to determine if a clotting match occurs 8-13. If there is no match, redo the process again. If a match occurred, counteract the clotting event by adding an anti-coagulant 8-14 to reduce the clotting event.
[0108] FIG. 9 illustrates a blood clotting system that uses machine learning to make decisions of blood clotting events. Once the system starts 9-1, the RGB detector 9-2 enables the light source and camera to collect images. These images may be used immediately or stored in memory and recalled from memory at a later date. The images are collected to form a collection. The collection is a large database that the user or others can add their images to and use. This collection forms the basis of learning 9-3 where the data reveals the characteristics of a blood clot. That knowledge is used in the decision making 9-4 step to reduce the clotting event.
[0109] FIG. 10 shows a Supervisory Machine Learning (ML) Block diagram that uses the Gradient Decent 10-11 to adjust the weights and bias 10-9. A successful ML project requires the manipulation of large databases. The ML device performs specific tasks without proving explicit instructions. The ML device uses the massive database to help it to learn what is needed. The databases 10-2 obtains its data from local cache, on-chip RAM, tape, disc, Memory array, server, Cloud, Internet 10-1. The database comprises data to train the network and more data to test the network. In supervisory learning, data and the answer is provided to the ML device. In one embodiment, the answer is a Boolean (True or False) and represents if the pixel is seeing a clot or not. So, for example, if the answer is True, data 10-3 representing a clot is applied to the input layer 10-5. Boolean implies that this is classification problem (fraud: Yes or No; Spam Mail: True or False; Clot: True or False). During training, the weights and bias 10-9 change their values 10-12 and are continually applied and updated for each new input value applied to the neural network. The neural network comprises the input layer 10-5, the interconnect (with weights and bias), hidden layers 10-7 (one or more), another interconnect (with weights and bias), and an output layer 10-8. As time passes, the neural network makes a better and better estimate 10-10 that is compared to the answer 10-4 while the system is performing the gradient decent 10-11 and training the network.
[0110] A neural network (NN) is an adjustable-weighted network used with machine learning algorithms that can be used to classify inputs based on a previous training process. In a first embodiment of clot detection, a neural network is first trained on a first set of clotted and non-clotted images to detect clotting. Once the NN has been trained to classify images as either containing a blood clot or not, the NN is switched from test mode to run mode and used to identify blood clot events in patients.
[0111] The classification problem of finding the clot identifies the appearance of a clot. Once all the clots in an image have been identified, the next step is to determine the amount (or concentration) of the current state of clotting. One embodiment to find the concentration is to check each pixel that contains the clot. Next, find the average clot area, count all checked pixels and divide by the total pixel count. This creates a scale ranging from 0 (no clots) to 1 (filled with clots). Next, a threshold level is set that will be used to trigger an event. Assume a threshold level of 10%, so once the average clot area exceeds 0.1, an alert is posted. The alert can be used to perform a step, i.e., inject anti-coagulant to decrease the average clot area.
[0112] Once the neural network (NN) has been trained, the NN is ready for use in ECMO or for any other life threating need. Similar neural networks exist that are non-supervisory. These ML devices learn how to group events into clusters by just using data (no answers). There are many approaches to ML: Supervised learning, unsupervised learning, reinforcement learning, and feature learning, etc. There are also a number of models: NN, decision trees, support vector machine, etc.
[0113] It is understood that the above descriptions are only illustrative of the principle of the current invention. Various alterations, improvements, and modifications will occur and are intended to be suggested hereby, and are within the spirit and scope of the invention. This invention can, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that the disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the arts. It is understood that the various embodiments of the invention, although different, are not mutually exclusive. In accordance with these principles, those skilled in the art can devise numerous modifications without departing from the spirit and scope of the invention. The principles of clot detection as described above, can be applied to other cases, for example, in the case where blood clots can form in the blood stream even without injury. A biological specimen is defined to be a mammal. A mammal comprises humans, pigs, goats, cats, dogs, etc. The cannula is basically a tube that is inserted into the body to remove/add fluids. When inserted into veins or arteries, the fluid is blood and this blood, once removed externally, can be processed and returned to the body. The clot density is a measurement of the number of visible clots in a given area. The blood clot is also known as a thrombus. Thrombus has two components: a plug formed of platelets and red blood cells and a mesh of fibrin protein. In one embodiment, the RGB Detector components can be comprised of a light source, a light diffuser, a reflector tube, a lens, and a detector. In other embodiments, the RGB Detector can be comprised of a light source, a lens, and a detector. In some embodiments, wireless plays an important role. This inventive technique can be extended to monitor blood in the infrared spectrum range. One embodiment includes a CMOS camera and filter to detect an action in either the infrared or visible spectrum range. The action can be injecting an anti-coagulant, changing the blood flow rate, raising a flag, issuing an alarm, raising temperature, and lowering temperature. Anticoagulants may include variants of heparin, direct thrombin inhibitors, or anti-platelet drugs, Wireless communication techniques to control and interact with systems and the ability to power or charge an independent self-standing system wirelessly are well understood.
[0114] Some further definitions are provided. The detector's output presents pixel data (i.e., location, amplitude, color, etc.) where the location corresponds to the x-y position of the pixel in the array and the amplitude corresponds to the intensity of the detected light in a given frequency range. Data from all the pixels in the x-y array of the detector is collected and comprises one full scan of the detector's output. Each new full scan is an image. The image or part of the image provides the pixel data output value. The different pixels within the array capture the amplitude value of the three colors within the light that is incident at each of the different pixels for the entire array. Various algorithms can be used to find grouping of the different color intensities potentially indications a blood clot, and to assign a value to assign to the overall result which will be compared to the threshold value. A light sensitive array fabricated on a planar surface can be used to generate the pixel data output value. A reference view of blood is presented to the detector and comprises the threshold value indicating the start of blood clotting. The threshold value of the reference view of blood can be determined by calculating the numbers of pixels that the clot occupies then divide this number by the total number of pixels in the army. When the detector is presented a threshold value scene, the detector's output provides the reference pixel data output value. The term channeled electromagnetic radiation is comprised of the source light after being modified by passing through or reflected from the blood. Note that the light can also be reflected from internal sections of the blood sample. The source light illuminates the blood sample and the transmitted light and the reflected light from the blood sample comprises the channeled electromagnetic radiation. Each light detection arrangement receives a different component or fraction of the total channeled electromagnetic radiation. As a result, different components of the channeled electromagnetic radiation may correlate to different volumes within the blood. In one embodiment, a first light detection arrangement receives a component of the channeled electromagnetic radiation that the second light detection arrangement cannot resolve, while the second light detection arrangement receives a different component of the channeled electromagnetic radiation that the first light detection arrangement cannot resolve. The electromagnetic radiation bandwidth is any range of frequencies selected within the visible spectrum range, an infrared spectrum range, or both spectrum ranges. For example, one embodiment of the electromagnetic radiation bandwidth can be a sub-set of the visible spectrum or 405-480 THz which corresponds to the color Red.
[0115] Additionally, although the present invention is well suited for extracorporeal blood membrane oxygenation (ECMO), ECMO is but one of many possibilities of extracorporeal blood circulation systems where the present invention can be used. Furthermore, as capabilities of manufacturing the RGB detector will improve over time, sub-miniaturization techniques can be incorporated into the manufacturing of the detector until the entire unit can be reduced in size and be inserted entirely into one of the blood vessels of a patient. For example, in this case, assume 4-2 in FIG. 5C is either a vein or an artery. The RGB detector unit is attached to, coupled to, pressed against, or surgically connected to the inner walls of the vein or artery. The detector can then wirelessly communicate to the patient's mobile unit (phone) to monitor, determine and measure the blood clot formation. Power can be inductively coupled to the unit from outside the patient. The inductive coupling can transfer energy to the coils that are within the unit. The coils capture the energy.
[0116] The systems and methods disclosed herein can especially benefit from a computational machine that perform multiples and adds in parallel to significantly speed up performance. The systems and methods disclosed herein can use conventional purpose computer (but would take up to 2 orders of magnitude longer to calculate) when compared to the computational machines. These computational machines are special purpose computers that may be embedded in servers or other programmable hardware devices programmed through software, or as hardware or equipment programmed through hard wiring, or a combination of the two. A computational machine can comprise a single machine or device (a computer with multi-cores; a neural network; a gradient decent machine; machine learning device), or can comprise multiple interacting machines or processors (located at a single location or at multiple locations remote from one another). A computer-readable medium can be encoded with a computer program, so that execution of that program by one or more computers causes the one or more computers to perform one or more of the methods disclosed herein. Suitable media can include temporary or permanent storage or replaceable media, such as network-based or Internet-based or otherwise distributed storage of software modules that operate together, RAM, ROM, CD ROM, CD-R, CD-R/W, DVD ROM, DVD.+.R, DVD.+.R/W, hard drives, thumb drives, flash memory, optical media, magnetic media, semiconductor media, or any future storage alternatives. Such media can also be used for databases recording the information described above.
[0117] It is intended that equivalents of the disclosed exemplary embodiments and methods shall fall within the scope of this disclosure or appended claims. It is intended that the disclosed exemplary embodiments and methods, and equivalents thereof, may be modified while remaining within the scope of this disclosure or appended claims.
