INFRARED BASED CORE BODY TEMPERATURE SENSING SYSTEM AND METHOD
20210156749 · 2021-05-27
Assignee
Inventors
Cpc classification
G01J5/026
PHYSICS
A61B3/0025
HUMAN NECESSITIES
A61B3/14
HUMAN NECESSITIES
International classification
G01K13/20
PHYSICS
A61B5/01
HUMAN NECESSITIES
Abstract
A non-invasive and a non-contact core body temperature monitoring method and system including an infrared camera, a processor, and a non-transitory computer readable medium with a program for analyzing thermal images of the human eye. The method for using the system that includes isolating ocular thermal images from a facial image; extracting thermal information about the surface of the eye; transitioning the thermal information to a selected model of the eye; estimating a temperature at the back of the eye; optimizing the estimated temperature at the back of the eye, and transmitting the optimized temperature information to an output device.
Claims
1. An infrared temperature estimating method comprising: transitioning temperature data from a thermal image of an ocular surface of an eye to an ocular surface of a model of the eye; and estimating a core body temperature at the back of the model of the eye.
2. The method of claim 1, further comprising: transmitting the estimated core body temperature to an output device.
3. The method of claim 1, further comprising: obtaining the thermal image of the ocular surface of the eye; and extracting the temperature data from the thermal image of the ocular surface using a processor.
4. The method of claim 3, further comprising: detecting at least part of a facial surface comprising the ocular surface of the eye; acquiring a facial thermal image of the at least part of the facial surface; and transmitting the facial thermal image to the processor, where the facial thermal image comprises the thermal image of the ocular surface of the eye.
5. The method of claim 1, where the step of estimating the core body temperature comprises: using at least one model of heat transfer to map between core temperature and temperatures on the ocular surface; and performing an optimization to estimate the core body temperature at the back of the eye.
6. The method of claim 1, wherein the transition step comprises: applying temperature data from pixel positions of the thermal image of the ocular surface of the eye to an ocular surface of a discretized model of the eye.
7. The method of claim 6, wherein the discretized model includes a finite element mesh.
8. The method of claim 1, further comprising: constructing the model of the eye based on thermophysical properties and boundaries of a cornea, an anterior humor, a vitreous humor, a lens, a retina, a choroid, an iris, a ciliary body, and a sclera of an eye.
9. The method of claim 1, where the core temperature is estimated using the equation: .sup.d), Γ.sub.1 is part of the eye surface that is not exposed to the surrounding air, and Γ.sub.2∪Γ.sub.3 is part of the eye surface that is exposed to the surrounding air, T is the ocular surface temperature at Γ.sub.2∪Γ.sub.3 and the estimated core body temperature (the surface temperature at the back of the eye) at Γ.sub.1, T.sub.∞ is ambient temperature, ε is emissivity, E.sub.v is tear evaporative heat flux, Q.sub.eff is the effective volumetric heat generation rate due to metabolism, T.sub.d_core is the default core body temperature (typically set by the user), h.sub.∞ is convective heat transfer coefficient between the tear film and the surrounding air, and h.sub.b is the convective heat transfer coefficient between the vascular region (R.sub.1) and the surrounding blood.
10. A system for estimating a core body temperature, comprising: a non-transitory computer-readable medium in communication with a processor and comprising a program that, when executed, performs a method, the method comprising: transitioning temperature data from a thermal image of an ocular surface of an eye to the ocular surface of a model of the eye; and estimating the core body temperature at the back of the eye model.
11. The system of claim 10, wherein the model of the eye is based on thermophysical properties and boundaries of a cornea, an anterior humor, a vitreous humor, a lens, a retina, a choroid, an iris, a ciliary body, and a sclera of an eye.
12. The system of claim 10, further comprising: an infrared camera in communication with the processor.
13. The system of claim 10, further comprising: an output device in communication with the processor.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
[0021] The organization and manner of the structure and operation of the invention, together with further objects and advantages thereof, may best be understood by reference to the following description taken in connection with the accompanying drawings wherein like reference numerals identify like elements in which:
[0022]
[0023]
[0024]
[0025]
[0026]
[0027]
[0028]
[0029]
DETAILED DESCRIPTION OF THE DISCLOSURE
[0030] While this invention may be susceptible to embodiment in different forms, specific embodiments are shown in the drawings and will be described herein in detail with the understanding that the present disclosure is to be considered an exemplification of the principles of the invention and is not intended to limit the invention to that as illustrated.
[0031]
[0032]
[0033] In step 330, the facial thermal image 250 may be transmitted from the infrared camera 110 to the processor 120. In step 340, the processor 120, executing the program on the memory of a local machine or cloud computer 130, may extract the thermal image of the right eye 260r and/or the left eye 2601. Step 340 may be performed automatically or may be manually performed by a person controlling the isolation of the ocular surface thermal images 260 of one or both eyes. In some embodiments, step 340 may include facial feature detection where facial landmarks are identified to localize and isolate the eye regions 260 from the facial thermal image 250. The processor 120 may first identify the location of a face from the image 250 transferred from the infrared camera 110 using one or more of: Haar cascades, pre-trained Histogram of Oriented Gradients and Linear and support vector machine object detector, and deep learning-based algorithms. Once the location of the face is confirmed, then the processor 120 may execute a program for facial landmark detection to first identify eye regions, and, then, isolate the ocular surface thermal image 260 from the facial thermal image 250. The facial landmark detection may be based on pre-trained facial landmark images where ensemble regression trees are trained to estimate the facial landmark positions directly from pixel intensities. The facial landmark detection may be performed using a location of 68 (x, y) coordinate points iBUG 300-W dataset. The use of the iBug-W dataset is exemplary and illustrative only, as other suitable datasets known to persons of skill in the art may also be used, such as a 194 coordinate points HELEN dataset. The facial landmark detector may also be equipped with eye blink detection feature, where it can distinguish open eyes from closed once. The purpose of the blink feature is to track the eyes and facilitate automatic capture of the ocular surface images from open eyes from a recorded or live streaming infrared thermal image.
[0034] In step 345, one of the optical image and the thermal image is mapped onto the other in order to correct for differences (image sizes, mounting orientations, fields of view, etc.) due to multiple cameras 110 being used to capture the optical and thermal images. This step is optional and may not be necessary when using a single camera that can capture both optical and infrared images. Once step 345 is complete, future performance of the method 300 may use the mapping information from previous performances, thus, step 345, if needed, is only performed once per set up of the cameras 110. Ocular landmark points (pixel locations) in the optical images may be used as a reference points to locate the ocular landmark points in the thermal images. This mapping process may use homography or geometric-based mapping.
[0035] Homography mapping is performed by selecting 4-8 points in the optical image and the corresponding thermal image to set up an initial pixel correspondence between the two cameras 110. Then a homograph may be constructed based on those data points that can then be used to map pixel locations in the optical image onto the thermal image. Geometric-based mapping is performed by using the inner canthi locations in both the optical and thermal images as anchor points (or fixed points of reference). In the thermal image, the inner canthi are most consistently warmest part of the facial surface; hence, different methods can be used to locate the inner canthi, such as 1) a localized search of the pixels around the eye region with the warmest temperature values and 2) use of machine learning to identify location of the inner canthi. In the optical image, the location of the inner canthi may be obtained from the ocular landmark points. Following the localization of the inner canthi, the geometry of both images and the relationship between the image dimensions and Field of Views of the chosen cameras may then be used to map the ocular landmark points from the optical image onto the thermal image. Both homography and geometric based mapping require that the cameras 110 be in close proximity to each other during the taking of the optical and thermal images, usually between 0.5 and 2.5 inches, in order to reduce the effect of parallax.
[0036] In step 350, temperature data may be extracted from the ocular surface thermal images 260 of the one or both eyes. Temperature data extraction may include filtering temperature values within segmented regions of the ocular surface 160.
[0037] In step 360, a model 400 of the eye may be constructed with the considerations of different regions: cornea, anterior and vitreous humor, lens, retina, choroid, iris, ciliary body, and sclera, grouped based on their thermophysical properties. In some embodiments, step 360 may be optional, especially if the eye model 400 has been previously constructed or provided. The eye model 400 selected to be used to establish boundary conditions may differ based on the patient. Different sizes of eye models, ranging from an infant to an adult size, may be constructed using CAD and meshing software. The goal is to approximately match the eyeball sizes of different age groups for an accurate estimate of core body temperature. The globes of the eye models are assumed to be axisymmetric with respect to the optical axis. The anterior and posterior regions of the eye 500 are composed of vascular and avascular tissue structures (See
TABLE-US-00001 TABLE 1 Thermophysical properties of eye tissues (see FIG. 5) Thermal Eye conductivity, Specific heat, Density, ρ Regions Tissue k [W/m-K] C.sub.p [J/kg-K] [kg/m.sup.3] R.sub.1 Sclera/Retina/ 1.0042 ± 0.04 4200 1032 Iris/Choroid R.sub.2 Cornea 0.58 ± 0.06 3515 1052 R.sub.3 Aqueous 0.58 ± 0.01 3997 1003 R.sub.4 Lens 0.43 ± 0.05 3133 1076 R.sub.5 Vitreous 0.59 ± 0.01 4047 1005 Blood 0.53 ± 0.04 3617 1050
[0038] Temperature distribution within the eye may be formulated using an energy equation where the vascular region of the eye includes two energy equations, while the avascular region is treated as a solid tissue. In the vascular region 510 of the eye (R.sub.1), for blood:
For tissue:
In the avascular region 520, 530, 540, 550 of the eye (R.sub.i):
[0039] where R.sub.1 and R.sub.i (i=2 . . . 5) represent the vascular and avascular regions (Table 1) in the eyeball, respectively; k.sub.i, k.sub.s and k.sub.b are the thermal conductivities of the avascular tissue (in R.sub.i, i=2 . . . 5), vascular tissue (in R.sub.I) and blood, respectively; φ is the porosity of R.sub.1; T.sub.b and T.sub.s are the temperature of the blood and tissue in R.sub.1, respectively; T.sub.i is the temperature of the avascular tissue in R.sub.i (i=2 . . . 5); h.sub.tb is the convective heat transfer coefficient between blood and tissue structure; a.sub.tb is the specific surface area; Q.sub.m is the volumetric heat generation rates due to metabolism; ρ.sub.s, c.sub.p.sub.
[0040] The governing equations are simplified by assuming steady state. For the vascular region, the two energy equations are coupled by the convective heat transfer term, h.sub.tb a.sub.tb (T.sub.b−T.sub.s):
∇.Math.(φk.sub.b∇T.sub.b)+∇.Math.((1−φ)k.sub.s∇T.sub.s)−φρ.sub.bc.sub.p.sub.
[0041] The above equation may be further simplified by assuming the capillary vessels within the vascular tissue are thermally insignificant; meaning, there is no temperature difference between tissue and blood, i.e. T.sub.s=T.sub.b, leading to:
{right arrow over (v)}.sub.b.Math.∇T.sub.b≈ω.sub.b(T.sub.s−T.sub.b)=0 in R.sub.1 (5)
[0042] where ω.sub.b is the perfusion rate of blood, defined as volume flow rate of blood per unit volume of the vascular tissue, in mL-blood/[(mL-tissue)-sec]. The temperatures T.sub.s and T.sub.b collectively are represented as T.sub.1 (T.sub.s=T.sub.b=T.sub.1). The vascular equation can then be represented as:
∇.Math.((φk.sub.b∇T.sub.1)+∇.Math.((1−φ)k.sub.s∇T.sub.1)−(1−φ)Q.sub.m=0, in R.sub.1 (6)
∇.Math.(∇T.sub.1(φk.sub.b+(1−φ)k.sub.s))−(1−φ)Q.sub.m=0, in R.sub.1 (7)
k.sub.eff=φk.sub.b+(1−φ)k.sub.s and Q.sub.eff=(1−φ)Q.sub.m (8)
∇.Math.(k.sub.eff∇T.sub.1)−Q.sub.eff=0, in R.sub.1 (9)
[0043] where k.sub.eff is the effective value of the thermal conductivity of eye tissue and Q.sub.eff is the effective volumetric heat generation rate due to metabolism
[0044] The avascular region 520, 530, 540, 550 may be expressed as:
∇.Math.(k.sub.1∇T.sub.i)=0, in R.sub.i i=2 . . . 5 (10)
Table 2 shows the design parameters considered in the eye model. Although some of the parameters, such as convective heat transfer coefficient of the blood, porosity, and emissivity, are assumed to be constant, the ambient temperature, convective heat transfer coefficient of the air are expected to vary with the environmental conditions. The parameter values shown in Table 2 are designed to represent eye model in a controlled indoor environment.
TABLE-US-00002 TABLE 2 Values of parameters used in eye model Parameter Description/Unit Value h.sub.∞ Convective heat transfer coefficient 10 ± 2.5 between the tear film and the surrounding air [W/m.sup.2-K]* h.sub.b Convective heat transfer coefficient 110 ± 10 between the vascular region (R.sub.1) and the surrounding blood [W/m.sup.2-K] σ Stefan Boltzmann constant [W/m.sup.2-K4] 5.67 × 10.sup.−8 ϵ Emissivity 0.975 ± 0.021 E.sub.v Tear evaporative heat flux [W/m.sup.2]* 40 ± 2.0 T.sub.∞ Ambient temperature [° C.]* 23 ± 1.0 φ Porosity of the healthy vascular tissue 0.3 ± 0.03 Q.sub.m Metabolic heat generation [W/m.sup.3] *Subject to change based on the environmental condition
[0045] Combination the vascular and avascular equations will provide the mathematical formulation necessary for solving the core body temperature as the boundary condition of the sclera surface that is not exposed to the surrounding air.
∇.Math.(k.sub.eff∇T.sub.1).Math.∇.Math.(k.sub.i∇T.sub.i)−Q.sub.eff=0, in R.sub.i i=2 . . . 5 (11)
[0046] The boundary condition for the vascular surface of the sclera not exposed to the surrounding air 610 (See
[0047] where h.sub.b is the convective heat transfer coefficient between the vascular region 510 (R.sub.1) and the surrounding blood, and
is me vascular temperature gradient in the direction of outward unit vector (n.sub.1) normal to Γ.sub.1 of R.sub.1. T.sub.d_core is the default core body temperature and T.sub.1 is the vascular tissue temperature distribution of R.sub.1.
[0048] The boundary condition for the vascular surface of the sclera exposed to the surrounding air 620, Γ.sub.2 is:
[0049] The vascular surface 620 is subjected to both radiative and convective heat transfer, as well as evaporation from the tear film to the surrounding air.
is me vascular temperature gradient in the direction of outward unit vector (n.sub.1) normal to Γ.sub.2 of R.sub.1, h.sub.∞ is the convective heat transfer coefficient between the tear film and the surrounding air, σ is the Stefan-Boltzmann constant, ε is the ocular surface emissivity, T.sub.∞ is the ambient air temperature, E.sub.v is the tear evaporative heat flux. Values of the parameters used in the above equations, including the thermophysical properties, are summarized in Tables 1 and 2.
[0050] The interfacial conditions between two contiguous regions are described by the continuity condition:
[0051] where IR.sub.1i is the interface between the vascular region R.sub.1 and the neighboring avascular regions R.sub.i (i=2 . . . 5); k.sub.i and T.sub.i are the thermal conductivity and the temperature distribution of R.sub.i (i=2 . . . 5), respectively.
[0052] The boundary condition avascular surface 630 of the cornea Γ.sub.3 (
[0053] where k.sub.3 and T.sub.3 are the thermal conductivity and temperature of the cornea tissue of the avascular region R.sub.2, respectively,
is the avascular temperature gradient in the direction of outward unit vector (n.sub.3) normal to Γ.sub.3 of R.sub.2.
[0054] where the eye domain R∪.sup.d the unknown temperature of the tissue T∈C.sup.n (R) (where C.sup.n is a continuous function of n derivatives), and a test function w such that w∈C.sup.n(R). Equation 19 is the final mathematical model that estimates the core body temperature.
[0055] In some embodiments, an approximate solution to this mathematical model is computed numerically using techniques that may include fixed-point iteration, Newton's method, or a quasi-Newton method for handling the nonlinear radiative term, and techniques that may include direct solvers or preconditioned iterative solvers for systems of linear equations.
[0056] In step 370, the temperature data may be transitioned from their pixel position in the ocular surface thermal image 260 to corresponding Cartesian coordinates 410 (see
[0057] where N.sub.col and N.sub.row are the column and row numbers, respectively; P.sub.y and P.sub.z are the pixel locations in the column and row locations, respectively; Y.sub.min is the least-valued point of the y axis from the sensor data of the ocular surface; Y.sub.max is the most-valued point; Z.sub.min is the least-valued point of the z axis from the sensor data point of the ocular surface; Z.sub.max is the most-value point of the z axis from the sensor data point of the ocular surface; and Y.sub.cor and Z.sub.cor are the sensor locations in the eye model corresponding to the P.sub.y and P.sub.z pixels of the ocular surface thermal image 260.
[0058] Zero or missing values may be validated. Once the temperature values are mapped from the 2D of the thermal image to 3D sensor data points of the ocular surface of the eye model 400, the eye model 400 may be electronically stored as an observed input value that may be used to estimate the temperature at the back of the eye using inverse analysis and optimization. In some embodiments, models of heat transfer may be used to map between core temperature and temperatures on the ocular surface.
[0059] In step 380, the temperature at the back of the eye is estimated using the eye model 400. The estimate may be obtained using a gradient-based or non-gradient-based optimization procedure to estimate the core body temperature at the back of the eye that produces the ocular surface temperature profile that best fits the data according to some merit function. For the temperature estimation in step 380, the temperature data may be evaluated using a combination of forward and backward methods, otherwise known as inverse analysis. In one embodiment, the mathematical formulations of the eye model which may include a governing equation, formulated in Equation. 19, along with the goodness of fit test and non-gradient optimization are used to evaluate the surface temperature at the back of the eye. The inverse problem may be set up as a minimization function of temperature (T), where the analysis can be performed by iterative minimization of an object function. As an initial starting point, guess the value of T at the surface boundary of Γ.sub.1 and solve the ocular surface temperature T at the surface boundary of Γ.sub.2 and Γ.sub.3 using a forward method. The estimated temperature data of the ocular surface (i.e. T at Γ.sub.2 and Γ.sub.3) may then undergo the goodness of fit test with the ocular surface temperature data obtained from the infrared camera where the convergence to the minimum of the objective function is assessed. Depending on the result of the fit, the initial guess may then be updated, and a new estimate of ocular surface temperature is evaluated. The iterative process continue until the objective function is satisfied, i.e. best core body temperature value is attained. To handle the iterative process, non-gradient-based optimization may be performed using Brent's method, golden section search method, downhill simplex, or a pattern search method; alternatively, a gradient-based technique such as steepest descent, conjugate gradient, the Broyden-Fletcher-Powell-Reeves method, or Newton's method may be used. The chi-square test may be used to assess the goodness of fit test between the estimated and infrared measured ocular surface temperature data.
[0060] Equation 21 shows formulation of the reduced chi-square (χ.sub.red.sup.2) equation, in which χ.sub.red.sup.2 is given by:
[0061] where T.sub.IR represents the ocular surface temperature acquired from the thermal image; T.sub.i (where i=1 and 3) represents the estimated ocular surface temperature evaluated using Equation 19; r is the position of the temperature sensors, a is the optimization parameter described as a function of the core body temperature; and N is the total number of sensors located on the ocular surface area of the eye model. Herein T.sub.IR(r; α) is fitted to N number of numerically generated ocular surface temperature data (T.sub.i). Since χ.sub.red.sup.2 is formulated to account the noise inherited from the instrumentation error, (T) is added to account for the noise equivalence difference temperature of the camera provided by the manufacturer. It should be noted that, since
(T) is uniform across the image, the method produces a core body temperature estimate that is independent of the accuracy rating of the camera.
[0062] For a single model assessment, the accuracy of the fitness is established based on the following criteria: [0063] λ.sub.red.sup.2>1—Poor fit. [0064] λ.sub.red.sup.2≈1—Good fit. [0065] λ.sub.red.sup.2<1—Over fit.
[0066] In step 390, the estimated core body temperature and (optionally) the model 400 may be transmitted to the output device 140.
[0067]
[0068] While embodiments in the present disclosure have been described in some detail, according to the preferred embodiments illustrated above, it is not meant to be limiting to modifications such as would be obvious to those skilled in the art.
[0069] The foregoing disclosure and description of the disclosure are illustrative and explanatory thereof, and various changes in the details of the illustrated apparatus and system, and the construction and the method of operation may be made without departing from the spirit of the disclosure.