Burn severity identification and analysis through three-dimensional surface reconstruction from visible and infrared imagery

Abstract

An apparatus and method to assist making treatment decisions for burn injuries. The apparatus will leverage computational imaging methodologies with conventional thermographic analysis techniques. Using infrared sensors, computational image analysis, and burn assessment using thermographic imaging, a complete burn assessment imaging device can be fabricated entirely from commercially-available components. This device will use advanced software paired with a smartphone-mounted infrared camera to perform a detailed thermographic analysis using a burn triage algorithm.

Claims

1. An apparatus for analyzing burn injuries, comprising: a smartphone having a visible light camera; an IR camera; the smartphone collecting visible light video and IR signal video of a burn injury while moving the cameras; a computing device using the video to create a 3D surface model; the computing device overlaying thermal information from the IR camera onto the 3D surface model; the computing device calculating burn area using machine vision methods on the 3D surface model and analyzing temperature with thermal information on the 3D surface model.

2. The apparatus according to claim 2, wherein the computing device is remote from the smartphone.

3. The apparatus according to claim 1, wherein the computing device is provided within the smartphone.

4. A method for analyzing burn injuries, comprising the steps of: using a smartphone having a visible light camera and an IR camera, running a pre-installed application on the smartphone to collect video and IR signal video of burn injury while moving the cameras; transmitting video data to backend software or an external computing device; using video input to an algorithm to create a 3D surface model; using IR video to overlay thermal information on the 3D surface model; performing calculations on the 3D surface model for predicted burn depth and thermal volume; calculating burn area using machine vision methods on the 3D surface model; performing temperature analysis with thermal information on the 3D surface model; combining metrics calculated above using threshold values to yield values between 0 and 1; calculating a statistical value, such as an average, of the above values to reach a triage decision.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0021] FIG. 1 is a method step flow diagram that shows the process for burn injury triage.

[0022] FIG. 2A and FIG. 2B are screen shots from a burn triage software showing exemplary output from the burn triage software from two readings, demonstrating how triage recommendations can change with later observation.

[0023] FIG. 3 is an exemplary schematic view of the graphical user interface (GUI) that might be used for the application.

[0024] FIG. 4A is a typical screen image captured by an IR camera.

[0025] FIG. 4B is a rear perspective view of an exemplary embodiment burn scanning apparatus according to the invention.

[0026] FIG. 4C is a front perspective view of the exemplary embodiment burn scanning apparatus of FIG. 4B.

[0027] FIG. 5A is a photographic perspective view of an overlay of the IR and visible images of a chilled burn area on a subject's arm showing a clear IR signature.

[0028] FIG. 5B is an enlarged photographic perspective view taken from FIG. 5A showing a plotting line drawn on the IR image of chilled skin.

[0029] FIG. 5C is a plot of the relative intensity vs distance along the chilled area.

DETAILED DESCRIPTION

[0030] While this invention is susceptible of embodiment in many different forms, there are shown in the drawings, and will be described herein in detail, specific embodiments thereof with the understanding that the present disclosure is to be considered as an exemplification of the principles of the invention and is not intended to limit the invention to the specific embodiments illustrated.

[0031] This application incorporates by reference U.S. Provisional Application No. 63/156,456 filed Mar. 4, 2021 in its entirety.

[0032] A burn triage procedure is described herein. First, the injury will be imaged and mapped as a 3D surface and various predictions made of the burn depth and thermal volume. Next, calculations are made of the burn area using machine vision methods (edge and contour detection with thresholding). This burn area will be of the 3D surface, and not from simple 2D images. Following the calculation of burn area, a temperature analysis is completed. This provides more metrics of burn damage: the absolute temperature of the burn overall and at its center, the spatial relative temperature following approximately the three Jackson zones of injury obtained statistically by their distribution, and the change in temperature over time through multiple scans of the same burn which has predictive power in the healing potential and damage extent. The temperature analysis is performed statistically and gives a relative percent of the body that is burned, the percent of each zone of injury, and the statistical mean and standard deviation of temperature changes in space and time.

[0033] Each metric from this large set of data yields a Triage Level value from 0 to 1, where 0 indicates very low healing potential from this metric, indicative of necrosis and the need for surgery, and 1 indicates high healing potential through a likely more superficial burn. At present, a composite score from the different metrics is computed as a simple average. This value can then be mapped for each burn, and can be mapped over the 3D model and related 2D images, giving an easy color-code for the extent of burn damage in a given area. This composite score will then be summed over the detected burn areas and an overall triage recommendation given to the user indicating the potential for healing spontaneously in 21 days, or indicating the need for surgical intervention for proper healing of the injury. This rather simplistic approach could be enhanced and validated using future clinical studies. Calculation of some example metrics are listed below.

[00001]$$\begin{gathered} T{L}_{absT}=\{\begin{array}{c}1\mathrm{if}{T}_{ce\mathrm{nter}}>33^{\circ }C.\\ 0\mathrm{if}{T}_{ce\mathrm{nter}}<31^{\circ }C.\\ \frac{\left(T_{ce\mathrm{nter}}-31\right)}{2}\mathrm{otherwise}\end{array} \\ T{L}_{\partial T/\partial x}=\{\begin{array}{c}1\mathrm{if}{T}_{healthy}-T_{ce\mathrm{nter}}\le 0^{\circ }C.\\ 0\mathrm{if}{T}_{healthy}-T_{ce\mathrm{nter}}>3^{\circ }C.\\ \frac{\left(T_{healthy}-T_{ce\mathrm{nter}}\right)}{3}\mathrm{otherwise}\end{array} \\ T{L}_{Zonesize}=\{\begin{array}{c}1\mathrm{if}{A}_{>0.25T_{\max }}/A_{tot}\le 30\%\\ 0\mathrm{if}{A}_{>0.25T_{\max }}/A_{tot}>60\%\\ 2-\frac{A_{>0.25T_{\max }}/A_{tot}}{0.3}\mathrm{otherwise}\end{array} \\ TL=\frac{1}{N}\underset{0}{\overset{N}{.Math.}}T{L}_N \end{gathered}$$

[0034] Using this technique, the first responder or combat medic can quickly assess the burn damage via the automated analysis, or they may inspect and analyze the 2D or 3D triage recommendation data themselves. As more data is collected with this system through animal studies or clinical trials, the automated algorithm can be further improved by a variety of means, including a new method of creating a composite score such as a weighted average, creation of new burn damage metrics with more predictive power, or through machine learning methods via a simple neural network classifier from the entire dataset.

[0035] Thus, a burn triage recommendation can be made immediately following a scan, with increasing confidence in the recommendation by adding repeated scans during recovery. Scans themselves take less than 10 seconds, with data processing on the order of minutes, allowing for very rapid triage when needed.

[0036] FIG. 1 is an exemplary flow diagram that shows the software-implemented process for burn injury triage.

[0037] Step 1: Start with Smartphone and IR camera apparatus, and a burn injury site.

[0038] Step 2: Run pre-installed application on smartphone, collect visible light video and IR signal video of burn injury while moving the cameras.

[0039] Step 3: Transmit video data to backend software or an external computing device.

[0040] Step 4: Use video input to an algorithm to create a 3D surface model.

[0041] Step 5: Use IR video to overlay thermal information on 3D surface model.

[0042] Step 6: Perform several calculations on 3D surface model for predicted burn depth and thermal volume.

[0043] Step 7: Calculate burn area using machine vision methods on 3D surface model.

[0044] Step 8: Perform temperature analysis with thermal information on the 3D surface model.

[0045] Step 9: Combine metrics calculated above using threshold values from the literature to yield values between 0 and 1.

[0046] Step 10: Calculate statistical value, such as an average, of the above values to reach a triage decision.

[0047] FIG. 2A and FIG. 2B are screen shots from a burn triage software showing exemplary output from the burn triage software from two scans, demonstrating how triage recommendations can change with later observation. Both were observed on the 2.sup.nd day of observation. The change in temperature from healthy to the center reduced from Δ3.1° C. to Δ2.1° C., and subsequently the triage level rose from 0.63 to 0.7.

[0048] FIG. 3 is an exemplary schematic view of a graphical user interface (GUI) 50 that might be used for the software application. The GUI 50 provides a display of instructions, controls and calibrations. It can be provided on a desktop or other computer or on the device shown in FIGS. 4B and 4C.

[0049] FIG. 4A is a typical screen image captured by an IR camera providing a thermal image.

[0050] FIG. 4B is a rear perspective view of an exemplary embodiment burn scanning apparatus 100 according to the invention. The apparatus 100 includes a screen 104 mounted on a handle 108. The screen 104 can be part of a smart phone 112.

[0051] FIG. 4C is a front perspective view of the exemplary embodiment burn scanning apparatus 100 of FIG. 4B. An IR camera 120 is mounted on the handle 108. A camera 126 for capturing visible images and videos can be provided, such as being provided on the smart phone 112.

[0052] FIG. 5A is a photographic perspective view of an overlay of the IR signature captured by the IR camera 120 and visible images captured by the camera 126, of a chilled burn area 150 on a subject's arm 156 showing a clear IR signature.

[0053] FIG. 5B is an enlarged photographic perspective view of the chilled burn area 150 taken from FIG. 5A showing a plotting line 160 drawn on the IR image of chilled skin.

[0054] FIG. 5C is a plot of the relative intensity vs distance along the plotting line 160 of the chilled area 150.

[0055] From the foregoing, it will be observed that numerous variations and modifications may be effected without departing from the spirit and scope of the invention. It is to be understood that no limitation with respect to the specific apparatus illustrated herein is intended or should be inferred.