Handheld devices for wound assessment using multi-modal imaging
11471696 · 2022-10-18
Assignee
Inventors
Cpc classification
A61N2005/0626
HUMAN NECESSITIES
H04N23/45
ELECTRICITY
A61B5/0035
HUMAN NECESSITIES
A61B2560/0431
HUMAN NECESSITIES
H04N23/10
ELECTRICITY
H04N13/239
ELECTRICITY
H04N23/90
ELECTRICITY
A61B5/443
HUMAN NECESSITIES
International classification
H04N13/239
ELECTRICITY
A61B5/1455
HUMAN NECESSITIES
A61B5/01
HUMAN NECESSITIES
Abstract
Multi-modal, portable, handheld devices for tissue assessment (e.g., wound assessment) are provided, as are methods of fabricating and methods of using the same. The devices can be used for virtual medicine (VM)-based wound management, such as VM-based diabetic foot triage (DFT) and management. The device can be used to take physiological measurements of temperature and/or tissue oxygenation of a wound to assess the wound, for example in a remote setting environment. The device can also be used to provide therapy for tissue repair and/or wound healing, apart from the multi-modal imaging of the tissue surface of the patient. For example, light therapy, such as low-level light therapy (LLLT) can be provided via one or more light emitting diodes (LEDs) and/or laser diodes.
Claims
1. A device for assessing a tissue of a patient by multi-modal imaging, the device comprising: a plurality of sensors comprising a near-infrared (NIR) camera, a thermal camera, and a visible light camera; a plurality of first light sources; a processor; a memory in operable communication with the processor; a machine-readable medium in operable communication with the processor and the memory; and a software application stored on at least one of the memory and the machine-readable medium, the software application comprising instructions that, when executed by the processor, perform the following steps: receiving data related to the tissue from the plurality of sensors; generating a plurality of maps based on the data related to the tissue received from the plurality of sensors, the plurality of maps comprising a heat map of the tissue and a tissue oxygenation map of the tissue; and providing the plurality of maps to a display in operable communication with the processor; the device being a portable, handheld device; the plurality of first light sources comprising at least four first light sources configured to provide low-level light therapy (LLLT) to the tissue, each of the first light sources providing light at a different wavelength from each of the other first light sources; the software application being configured to allow a user to control the plurality of first light sources to provide LLLT to the tissue; the device further comprising: a smart device that is a smartphone or a tablet; and an add-on module configured to attach to and communicate with the smart device; the NIR camera, the plurality of first light sources, and the thermal camera being disposed on the add-on module; and the display in operable communication with the processor being disposed on the smart device.
2. The device according to claim 1, the plurality of sensors further comprising a distance sensor.
3. The device according to claim 1, the plurality of first light sources being further configured for auto-fluorescence imaging.
4. The device according to claim 1, the add-on module comprising at least one of a clip-on attachment and a magnetic attachment configured to attach to the smart device.
5. The device according to claim 1, the processor, the memory, and the machine-readable medium being disposed in the smart device.
6. The device according to claim 1, the plurality of sensors further comprising a stereoscopic camera disposed on the add-on module.
7. The device according to claim 1, the plurality of sensors further comprising a stereoscopic camera.
8. The device according to claim 1, further comprising at least one second light source configured for auto-fluorescence imaging.
9. The device according to claim 1, each first light source being a light emitting diode (LED) or a laser diode.
10. The device according to claim 1, the plurality of maps comprising a visible light map of the tissue, a melanin map of the tissue, and a water map of the tissue.
11. The device according to claim 1, the heat map of the tissue being a two-dimensional (2D) heat map, and the tissue oxygenation map of the tissue being a 2D tissue oxygenation map.
12. A method for assessing a tissue of a patient, the method comprising: scanning the tissue with a device that is portable and handheld, the device comprising: a plurality of sensors comprising a near-infrared (NIR) camera, a thermal camera, and a visible light camera; a plurality of first light sources, the plurality of first light sources comprising at least four first light sources configured to provide low-level light therapy (LLLT) to the tissue, each of the first light sources providing light at a different wavelength from each of the other first light sources; a smart device that is a smartphone or a tablet; an add-on module configured to attach to and communicate with the smart device; a processor; a memory in operable communication with the processor; a machine-readable medium in operable communication with the processor and the memory; and a software application stored on at least one of the memory and the machine-readable medium, the software application being configured to generate a plurality of maps based on data related to the tissue received from the plurality of sensors, the plurality of maps comprising a heat map of the tissue and a tissue oxygenation map of the tissue, the software application being configured to allow a user to control the plurality of first light sources to provide LLLT to the tissue; assessing the tissue based on the plurality of maps displayed on a display in operable communication with the processor; and using the software application to control the plurality of first light sources to provide LLLT to the tissue; the plurality of sensors further comprising a stereoscopic camera; the NIR camera, the thermal camera, the plurality of first light sources, and the stereoscopic camera being disposes on the add-on module; the display in operable communication with the processor being disposed on the smart device.
13. The method according to claim 12, the plurality of sensors further comprising a distance sensor; the plurality of first light sources being further configured for auto-fluorescence imaging; the plurality of sensors further comprising a stereoscopic camera; the plurality of maps comprising a visible light map of the tissue, a melanin map of the tissue; and a water map of the tissue, the heat map of the tissue being a two-dimensional (2D) heat map; and the tissue oxygenation map of the tissue being a 2D tissue oxygenation map.
14. The method according to claim 12, the plurality of sensors further comprising a stereoscopic camera; and the stereoscopic camera being disposed on the add-on module.
15. The method according to claim 12, the device further comprising at least one second light source configured for auto-fluorescence imaging.
16. The method according to claim 12, each first light source being a light emitting diode (LED) or a laser diode.
17. A device for assessing a tissue of a patient by multi-modal imaging, the device comprising: a plurality of sensors comprising a near-infrared (NIR) camera, a thermal camera, a visible light camera, a stereoscopic camera, and a distance sensor; a plurality of first light sources, the plurality of first light sources comprising at least four first light sources configured to provide low-level light therapy (LLLT) to the tissue, each of the first light sources providing light at a different wavelength from each of the other first light sources; at least one second light source for auto-fluorescence imaging; a processor; a memory in operable communication with the processor; a machine-readable medium in operable communication with the processor and the memory; and a software application stored on at least one of the memory and the machine-readable medium, the software application comprising instructions that, when executed by the processor, perform the following steps: receiving data related to the tissue from the plurality of sensors; generating a plurality of maps based on the data related to the tissue received from the plurality of sensors, the plurality of maps comprising a heat map of the tissue, a tissue oxygenation map of the tissue, a visible light map of the tissue, a melanin map of the tissue, and a water map of the tissue; and providing the plurality of maps to a display in operable communication with the processor; the device being a portable, handheld device; the software application being configured to allow a user to control the plurality of first light sources to provide LLLT to the tissue; each first light source being a light emitting diode (LED) or a laser diode; the device comprising: a smart device that is a smartphone or a tablet; and an add-on module configured to attach to and communicate with the smart device; the NIR camera, the thermal camera, the plurality of first light sources, and the distance sensor being disposed on the add-on module; the display in operable communication with the processor being disposed on the smart device; and the add-on module comprising at least one of a clip-on attachment and a magnetic attachment configured to attach to the smart device.
Description
BRIEF DESCRIPTION OF DRAWINGS
(1)
(2)
(3)
(4)
(5)
(6)
(7)
(8)
(9)
DETAILED DESCRIPTION
(10) Embodiments of the subject invention provide novel and advantageous multi-modal, portable, handheld (e.g., smartphone, tablet, or separate standalone device) imaging technology that can be used for tissue assessment such as wound assessment (e.g., remote wound assessment, such as remote diabetic wound assessment), both visual and physiological, for virtual medicine (VM)-based wound management (e.g., VM-based diabetic foot triage (DFT) and management). Embodiments also provide methods of fabricating the devices and using the devices. The device can be used to take physiological measurements of temperature and/or tissue oxygenation (TO) of a target area (e.g., a wound or wound site) of a patient (e.g., a human patient) to provide assessment (e.g., sub-clinical assessment) to replace, or complement the visual gold-standard clinical assessment (i.e., wound size, color, and epithelization), especially in a remote setting environment. The device can also be used to provide therapy for tissue repair and/or wound healing, apart from the multi-modal imaging of the tissue surface of the patient. For example, light therapy, such as low-level light therapy (LLLT) can be provided (e.g., via one or more light emitting diodes (LEDs), such as one or more LEDs at specific wavelengths (either individually or respectively)). Embodiments also provide a software application that can be used to control the multi-modal imaging and/or therapy provided by the device. The software application can be stored (e.g., on memory and/or a (non-transitory) machine-readable (i.e., computer readable) medium) on the smartphone, tablet, add-on device, or separate standalone device.
(11) Remote temperature sensing (e.g., via a forward looking infrared radar (FLIR) sensor) as a physiologic measure can be used for early warning (or triage) for diabetic foot ulcers (DFUs) in high-risk patients (see also; Lazo-Porras et al., “Implementation of foot thermometry plus mHealth to prevent diabetic foot ulcers: study protocol for a randomized controlled trial,” Trials, 17(1): 206, 2016; and Lin et al., “Assessment of lower extremity ischemia using smartphone thermographic imaging,” J Vasc Surg Cases Innov Tech, 3(4):205-208, 2017, doi:10.1016/j.jvscit.2016.10.012; both of which are hereby incorporated by reference herein in their entireties). Thermal measurements is associated with perfusion through an area, and a change in temperature (e.g., within 2.2° C.) at the wound interface can be indicative that flow has reduced in that region and the rate of oxyhemoglobin dissociation and oxygen availability may be drastically reduced (see also, Hallock, “Dynamic infrared thermography and smartphone thermal imaging as an adjunct for preoperative, intraoperative, and postoperative perforator free flap monitoring,” Plast Aesthet Res, 6:29, 2019, dx.doi.org/10.20517/2347-9264.2019.029; Magnin et al., “Use of infrared thermography to detect early alterations of peripheral perfusion: evaluation in a porcine model,” Biomed. Opt. Express, 11, 2431-2446, 2020; and Frykberg et al., “Challenges in the Treatment of Chronic Wounds,” Adv Wound Care, 4(9):560-582, 2015, doi:10.1089/wound.2015.0635; all of which are hereby incorporated by reference herein in their entireties). This can result in a decrease in cell proliferation and can therefore slow down the formation of new tissue necessary for wound healing.
(12) Near-infrared (NIR) optical imaging is advantageous in its ability to image subcutaneous tissue oxygenation, which is pertinent to wound healing. In addition to perfusion shown from thermal sensing, oxygen supply to wounds is a vital factor for successful healing as demonstrated from transcutaneous oxygen measurement (TCOM) studies (see also, Ballard et al., “A prospective evaluation of transcutaneous oxygen measurements in the management of diabetic foot problems,” Journal of Vascular Surgery, 22(4): 485-492, 1995, which is hereby incorporated by reference herein in its entirety). However, these measurements are obtained from point-based locations and can take up to 20 minutes to capture. Some oxygenation measuring devices can obtain two-dimensional (2D) measurements without contact. Such 2D tissue oxygenation maps can be beneficial in assessing wound progression, and this is not only the case with correlation between visible color maps and oxygenation maps that are coregistered and correlated, but also oxygenation and thermal images (e.g., using NIR and infrared (IR) imaging), which has clinical significance in chronic wound care management. (see also, Lucas et al., “Wound Size Imaging: Ready for Smart Assessment and Monitoring,” Advance in Wound Care, 2020, doi: 10.1089/wound.2018.0937; Zhang et al., “Multimodal imaging of cutaneous wound tissue,” J Biomed Opt, 20(1):016016, 2015 doi: 10.1117/1.JB0.20.1.016016. PMID: 25604545; PMCID: PMC4300315; Xu et al., “Dual-mode imaging of cutaneous tissue oxygenation and vascular function,” Journal of visualized experiments: JoVE, (46): 2095, 2010; and Tewari et al., “Methods for registering and calibrating in vivo terahertz images of cutaneous burn wounds,” Biomed Opt Express, 10(1):322-337, 2018, doi:10.1364/BOE.10.000322; all of which are hereby incorporated by reference herein in their entireties). Embodiments of the subject invention provide multi-modal imaging from a visual perspective and a physiological perspective, which can augment standard DFU management, especially for remote virtual assessment of DFUs.
(13) Embodiments of the subject invention provide physiological measurements of temperature and tissue oxygenation provide assessment (e.g., sub-clinical assessment) to replace or complement the visual gold-standard clinical assessment (i.e., wound size, color, and epithelization), especially in a remote setting environment. Detection of temperature fluctuations can be effective for early diagnostics of diabetic feet because ulceration risks are linked to an increase in plantar temperatures. Also, oxygen supply to wounds is a vital factor for successful healing and measurement of oxygen supply to a wound can predict potential to heal sooner than visual assessment alone. Unlike a device that can only obtain 2D wound images or maps, embodiments of the subject invention provide portable, handheld devices capable of measuring (and configured to measure) thermal- and oxygenation-based physiological parameters. Embodiments can also provide digitized visual assessment (e.g., wound size and/or color) using a multi-modal imaging approach, thereby giving multiple clinical assessments of wounds (e.g., diabetic wounds) on a patient.
(14) In addition to physiological assessment of wound healing status, embodiments of the subject invention can also accelerate the wound healing process therapeutically. Generally, wound healing therapies include wound dressings, autografts, allografts, topical agents, stem cell therapy, hyperbaric oxygen treatment, and low-level light therapy (LLLT). LLLT can use LEDs to promote tissue repair in chronic wounds through a thermal photoactive process, exhibiting a biphasic dose response curve. Monochromatic LEDs in particular can be used as therapeutic agents due to their ability to influence cellular sub-structures, in a process known as photobiomodulation (PBM) (see also, De Freitas et al., Proposed Mechanisms of Photobiomodulation or Low-Level Light Therapy, IEEE journal of selected topics in quantum electronics: a publication of the IEEE Lasers and Electro-optics Society, 22(3), 7000417, 2016, doi:10.1109/JSTQE.2016.2561201; which is hereby incorporated by reference herein in its entirety). Photodynamic therapy is a Food and Drug Administration (FDA)-approved treatment in which an extracellular photosensitizing topical agent is applied to a wound bed and, following illumination in the NIR region, forms reactive oxygen species (ROS) (see also, Lubart et al., Photochemistry and photobiology of light absorption by living cells, Photomed Laser Surg, 2006, 24:179-85; which is hereby incorporated by reference herein in its entirety). The ROS are key for cell signaling, regulation of cell cycle, enzyme activation, and nucleic acid/protein synthesis, which are essential for tissue reconstruction.
(15) LLLT is effective in promoting similar wound bed conditions by targeting intracellular photosensitive chromophores that absorb NIR light, leading to an increase in ROS. While the idea that light can stimulate biomolecules sounds intriguing, there is a limit to the magnitude of energy transfer in target elements. Some challenges associated with LLLT are the large variation in reported doses applied when using LEDs and the highly biphasic response observed (see also, Huang et al., Biphasic dose response in low level light therapy, Dose-response: a publication of International Hormesis Society, 7(4), 358-383, 2009, doi:10.2203/dose-response.09-027; which is hereby incorporated by reference herein in its entirety). Ionizing radiation cancer therapies can also exhibit biphasic dose responses largely due to heat dissipation, and theranostics is an attempt to understand the extent of biological response (using biomarkers) for optimized therapy (see also, Choudhury et al., Personalized and Precision Medicine in Cancer: A Theranostic Approach, Curr Radiopharm, 2017, 10(3):166-170, doi: 10.2174/1874471010666170728094008, PMID: 28758574; which is hereby incorporated by reference herein in its entirety). While LLLT is non-ionizing, the process of energy transfer needs to be assessed to prevent or inhibit overworking of cellular components. An LLLT device can use a range of power outputs and wavelengths, and non-contact imaging devices measure changes in oxygenation and blood flow beneath the skin surface. Embodiments of the subject invention can include both LLLT and non-contact imaging (e.g., to measure changes in oxygenation and/or blood flow beneath skin surface), allowing for complete non-contact treatment and objective assessment of therapeutic efficacy for use in complicated chronic wound cases.
(16) In many embodiments, a handheld, portable device can be configured to provide LLLT, for example for the purpose of administering variable dose therapy with the capacity to track physiological changes in tissue oxygenation and temperature in response to applied dose without contact. The LLLT capability can be included in an add-on device for a smartphone or tablet or as a portable hand-held device. Biomarkers for healing predictions can be used to overcome challenges associated with dose delivery in LLLT and identify effective doses for chronic cases.
(17)
(18)
(19)
(20)
(21)
(22)
(23)
(24) In an embodiment, a smartphone-based or tablet-based add-on device (or a standalone device) can perform diagnostics via physiological assessment of thermal changes and/or tissue oxygenation changes of tissue. An LLLT module can optionally be included, such that the device would also be a theranostics device. The device can perform multi-modal imaging (e.g., NIR imaging, IR imaging, thermal imaging, white light imaging, RGB imaging, and/or auto fluorescence imaging) and/or LLLT. The device can include light sources in specific or broad-spectrum visible light and NIR wavelengths, thermal or IR sensors (or detectors), an NIR sensor (or detector), a distance sensor, drivers for light sources and/or sensors, optical filters, polarizers, diffusers, reflectors, and/or a wireless (e.g., Bluetooth and/or Wi-Fi) platform.
(25) If an add-on device or module is used, the smartphone's (or tablet's) own camera(s) can be used along with the add-on device/module (as can an NIR related filter), and these can be synced for multi-modal imaging via a software application for data acquisition. Alternatively, a separate NIR-sensitive camera can be included in the add-on device/module, and the smartphone/tablet can be used for data acquisition and/or analysis via a software application. In some embodiments, a completely independent handheld standalone device can have all modules/functionalities (e.g., thermal imaging, NIR imaging to obtain tissue oxygenation measurements, and/or LLLT module), one or more of which can be integrated into the device.
(26) A custom software application can be provided/utilized for data acquisition from one or more of the imaging approaches (NIR imaging, thermal imaging, RGB imaging, white light imaging, and/or auto fluorescence imaging at specific wavelength(s) of choice); the images obtained from different imaging approaches can also be coregistered. The application can be synced with the add-on device/module and the smartphone's/tablet's camera (e.g., via Bluetooth, Wi-Fi, a cable, or similar technology). The application can be used for data pre-processing and/or analysis from one or more of the images (e.g., single- or multi-wavelength NIR images, thermal images, RGB images, white light images, and/or auto-fluorescence images at specific wavelength(s) of choice) that are displayed individually or co-registered onto each other. The application can be used for processing multi-wavelength images to generate tissue oxygenation maps (e.g., in terms of oxy-hemoglobin, deoxy-hemoglobin, total hemoglobin, and/or oxygen saturation) along with thermal maps, auto-fluorescence maps, RGB maps, and/or white light maps. The application can be used to perform co-registration and/or image segmentation of each or any of these images with respect to each other, or all images into a single co-registered and/or segmented image. Segmentation can be manual or can be done using artificial intelligence (e.g., deep learning, machine learning, and/or convolutional neural networks). The device (e.g., add-on device/module and/or standalone device can include one or more LLLT-based LEDs or laser diodes (which can be at specific wavelengths) to allow therapeutic application in tissue repair and/or wound healing, apart from multi-modal imaging of the tissue surface. The software application can be used to control the LLLT (e.g., the one or more LLLT-based LEDs) for therapeutic applications, and the controls can include controlling dosage, power, intensity, and/or time.
(27) United States Patent Application Publication No. 2019/0008387 and United States Patent Application Publication No. 2020/0352515 both described devices that may have some features in common with certain aspects of embodiments of the subject invention, and both are hereby incorporated by reference herein in their entireties. United States Patent Application Publication No. 2020/0352515 describes a smartphone-based device that can physiologically measure for tissue oxygenation changes in wounds apart from providing 2D visual wound images (see also, Kaile et al., “Development and validation of a smartphone based near-infrared optical imaging device to measure physiological changes in-vivo,” Micromachines 10(3): 180, 2019, doi: 10.3390/mi10030180; Kaile et al., “Low-cost smartphone-based imaging device to detect subsurface tissue oxygenation of wounds,” SPIE Proceedings, Vol. 10869, Optics and Biophotonics in Low-Resource Settings V, San Francisco, US, doi: 10.1117/12.2510425, 2019; and Kaile et al., Development of a Smartphone-Based Optical Device to Measure Hemoglobin Concentration Changes for Remote Monitoring of Wounds, Biosensors, 2021, 11(6):165, doi.org/10.3390/bios11060165; all of which are hereby incorporated by reference herein in their entireties). The mechanism for tissue oxygenation sensing described in United States Patent Application Publication No. 2020/0352515 can be utilized for tissue oxygenation sensing in devices of embodiments of the subject invention.
(28) The transitional term “comprising,” “comprises,” or “comprise” is inclusive or open-ended and does not exclude additional, unrecited elements or method steps. By contrast, the transitional phrase “consisting of” excludes any element, step, or ingredient not specified in the claim. The phrases “consisting” or “consists essentially of” indicate that the claim encompasses embodiments containing the specified materials or steps and those that do not materially affect the basic and novel characteristic(s) of the claim. Use of the term “comprising” contemplates other embodiments that “consist” or “consisting essentially of” the recited component(s).
(29) When ranges are used herein, such as for dose ranges, combinations and subcombinations of ranges (e.g., subranges within the disclosed range), specific embodiments therein are intended to be explicitly included. When the term “about” is used herein, in conjunction with a numerical value, it is understood that the value can be in a range of 95% of the value to 105% of the value, i.e. the value can be +/−5% of the stated value. For example, “about 1 kg” means from 0.95 kg to 1.05 kg.
(30) The methods and processes described herein can be embodied as code and/or data. The software code and data described herein can be stored on one or more machine-readable media (e.g., computer-readable media), which may include any device or medium that can store code and/or data for use by a computer system. When a computer system and/or processor reads and executes the code and/or data stored on a computer-readable medium, the computer system and/or processor performs the methods and processes embodied as data structures and code stored within the computer-readable storage medium.
(31) It should be appreciated by those skilled in the art that computer-readable media include removable and non-removable structures/devices that can be used for storage of information, such as computer-readable instructions, data structures, program modules, and other data used by a computing system/environment. A computer-readable medium includes, but is not limited to, volatile memory such as random access memories (RAM, DRAM, SRAM); and non-volatile memory such as flash memory, various read-only-memories (ROM, PROM, EPROM, EEPROM), magnetic and ferromagnetic/ferroelectric memories (MRAM, FeRAM), and magnetic and optical storage devices (hard drives, magnetic tape, CDs, DVDs); network devices; or other media now known or later developed that are capable of storing computer-readable information/data. Computer-readable media should not be construed or interpreted to include any propagating signals. A computer-readable medium of embodiments of the subject invention can be, for example, a compact disc (CD), digital video disc (DVD), flash memory device, volatile memory, or a hard disk drive (HDD), such as an external HDD or the HDD of a computing device, though embodiments are not limited thereto. A computing device can be, for example, a laptop computer, desktop computer, server, cell phone, or tablet, though embodiments are not limited thereto.
(32) It should be understood that the examples and embodiments described herein are for illustrative purposes only and that various modifications or changes in light thereof will be suggested to persons skilled in the art and are to be included within the spirit and purview of this application.
(33) All patents, patent applications, provisional applications, and publications referred to or cited herein are incorporated by reference in their entirety, including all figures and tables, to the extent they are not inconsistent with the explicit teachings of this specification.