GENERATION OF A TEMPERATURE MAP

Abstract

Generating an accurate and sensitive temperature map over a non-planar surface by capturing a polarimetric thermal image from a non-planar surface. The non-planar surface includes a surface element which emits thermal electromagnetic emission responsive to a temperature of the surface element. Thermal electromagnetic emission is directed to an image element of an image sensor. A polarisation state and intensity are measured of the ray of thermal electromagnetic emission from the surface element. Responsive to the polarization state and the intensity, an angular orientation of the surface element is determined. A temperature of the surface element is determined by estimating the thermal emission normal to the surface element responsive to the angular orientation of the surface element and the intensity of the ray of thermal electromagnetic emission emitting from the surface element.

Claims

1. A method comprising: capturing a polarimetric thermal image from a non-planar surface, wherein the non-planar surface includes a surface element, wherein the surface element emits thermal electromagnetic emission responsive to a temperature of the surface element, wherein a ray of the thermal electromagnetic emission is directed to an image element of an image sensor; measuring a polarization state and an intensity of the ray of thermal electromagnetic emission emitting from the surface element; responsive to the polarization state and the intensity, determining an angular orientation of the surface element; and determining a temperature of the surface element by estimating the thermal emission normal to the surface element responsive to the angular orientation of the surface element and the intensity of the ray of thermal electromagnetic emission emitting from the surface element.

2. The method of claim 1 comprising: said determining a plurality of temperatures respectively for a plurality of surface elements of the non-planar surface thereby generating a map of temperatures over the non-planar surface.

3. The method of claim 2, wherein the non-planar surface is living biological tissue.

4. The method of claim 2, further comprising: presenting a surface contour of the non-planar surface and the map of temperatures superimposed on the presented surface contour.

5. The method of claim 2, wherein the map of temperatures is independent of the respective angular orientations of the surface elements.

6. The method of claim 1, further comprising: prior to determining the temperature of the surface element, performing a calibration by measuring thermal electromagnetic radiation emittance or reflectance from a calibration surface as a function of angle; and said determining the temperature of the surface element further responsive to the calibration.

7. The method of claim 1, wherein said determining the temperature includes: extrapolating from the intensity of thermal emission as measured from the surface element emitting at a non-zero emission angle to an intensity of thermal emission expected at a normal to the surface element.

8. A system comprising a polarimetric camera and a processor connectible thereto, the system operable to: capture a polarimetric thermal image from a non-planar surface, wherein the non-planar surface includes a surface element, wherein the surface element emits thermal electromagnetic emission responsive to a temperature of the surface element, wherein a ray of the thermal electromagnetic emission is directed to an image element of an image sensor; measure a polarization state and an intensity of the ray of thermal electromagnetic emission emitting from the surface element; responsive to the polarization state and the intensity, determine an angular orientation of the surface element; determine a temperature of the surface element by estimating the thermal emission normal to the surface element responsive to the angular orientation of the surface element and the intensity of the ray of thermal electromagnetic emission emitting from the surface element.

9. The system of claim 8, further operable to: determine a plurality of temperatures respectively for a plurality of surface elements of the non-planar surface to generate a map of temperatures over the non-planar surface.

10. The system of claim 8, wherein the non-planar surface is living biological tissue.

11. The system of claim 9, further comprising a display; the system further operable to: present on the display a surface contour of the non-planar surface and the map of temperatures superimposed on the presented surface contour.

12. The system of claim 9, wherein the map of temperatures is independent of the respective angular orientations of the surface elements.

13. The system of claim 8, further comprising: prior to determining the temperature of the surface element, performing a calibration by measuring thermal electromagnetic radiation emittance or reflectance from a calibration surface as a function of angle; and said determining the temperature of the surface element further responsive to the calibration.

14. The system of claim 8, further operable to extrapolate from the intensity of thermal emission as measured from the surface element emitting at a non-zero emission angle to an intensity of thermal emission expected at a normal to the surface element.

15. A computer readable medium having software instructions stored therein to: input a polarimetric thermal image captured from a non-planar surface, wherein the non-planar surface includes a surface element, wherein the surface element emitted thermal electromagnetic emission responsive to a temperature of the surface element, wherein a ray of the thermal electromagnetic emission was directed to an image element of an image sensor; wherein the polarimetric thermal image includes a polarization state and an intensity of the ray of thermal electromagnetic emission emitted from the surface element; responsive to the polarization state and the intensity, determine an angular orientation of the surface element; determine a temperature of the surface element by estimating the thermal emission normal to the surface element responsive to the angular orientation of the surface element and the intensity of the ray of thermal electromagnetic emission emitted from the surface element.

16. The computer readable medium, of claim 15 having further software instructions stored therein to: determine a plurality of temperatures respectively for a plurality of surface elements of the non-planar surface to generate a map of temperatures over the non-planar surface.

17. The computer readable medium, of claim 16, wherein the non-planar surface is living biological tissue.

18. The computer readable medium of claim 16, having further software instructions stored therein to: present a surface contour of the non-planar surface and the map of temperatures superimposed on the presented surface contour.

19. The computer readable medium of claim 16, wherein the map of temperatures is independent of the respective angular orientations of the surface elements.

20. The computer readable medium of claim 15, having further software instructions stored therein to: input a calibration previously performed by measuring thermal electromagnetic radiation emittance or reflectance from a calibration surface as a function of angle; and determine the temperature of the surface element further responsive to the calibration.

21. The computer readable medium of claim 15, having further software instructions stored therein to: determine said temperature by extrapolating from the intensity of thermal emission as measured from the surface element emitting at a non-zero emission angle to an intensity of thermal emission expected at a normal to the surface element.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0007] The invention is herein described, by way of example only, with reference to the accompanying drawings, wherein:

[0008] FIG. 1 illustrates a simplified system drawing according to embodiments of the present invention; and

[0009] FIG. 2 illustrates a simplified flow diagram, according to features of the present invention.

[0010] The foregoing and/or other aspects will become apparent from the following detailed description when considered in conjunction with the accompanying drawing figures.

DETAILED DESCRIPTION

[0011] Reference will now be made in detail to features of the present invention, examples of which are illustrated in the accompanying drawings, wherein like reference numerals refer to like elements throughout. The features are described below to explain the present invention by referring to the figures.

[0012] By way of introduction, aspects of the present invention are directed to use of a thermal camera in order to generate a high resolution and accurate thermal map of biological tissue, e.g. female human breast, in order to screen for breast cancer, by way of example. It is well known that every object at a given temperature emits thermal electromagnetic radiation. In the case of a black body of emissivity =1, thermal radiation is modelled by the well known with Planck radiation law. However, for a general object, emissivity is less than one and not necessarily known exactly for all wavelengths of interest. Wavelength of maximum radiation in the temperature range of 30-40 degrees Celsius is in the range of 9.2-9.5 micrometers.

[0013] According to Kirchhoffs law of radiation, for an arbitrary body emitting and absorbing thermal radiation in thermodynamic equilibrium, absorptance and emittance are equal and that for a smooth and opaque surface, emittance is directly related to reflectance r,

()=1r()

[0014] Moreover, as can be understood from the equations above, emissivity decreases according to emission angle , as measured from the surface normal so that a temperature map generated by directly measuring intensity of thermal radiation is sensitive to angle of emittance. In order to conventionally overcome this drawback, it may be recommended to perform multiple thermal scans of the same object from different angles and only consider temperature of surfaces with emission close to the normal. Aspects of the present invention are related to use of polarization information of the emitted thermal radiation in order to account for the angle of emittance and generate a true temperature map.

[0015] According to features of the present invention, a thermal bolometric camera appropriate for detecting thermal radiation of wavelength 8-14 micrometer includes an image sensor of vanadium oxide or amorphous silicon which absorbs radiation of wavelength 8-14, and changes electric resistance of the image sensor elements by heating. Image sensors of different materials may become available for use in thermal bolometric cameras.

[0016] Commercially available thermal cameras may reach sensitivity and accuracy to sense thermal changes up to 30-40 millidegree Kelvin. A thermal camera suitable for implementing embodiments of the present invention is a division of focal plane imaging polarimeter with 640512 image elements, i.e. pixels, by way of example: Pyxis, of Polaris Sensor Technologies, Inc., Huntsville, AL and described in U.S. Pat. No. 9,829,384

[0017] The image sensor includes polarizing filters which may include a mosaic of four different polarization direction0, 90, 45 and 45 degrees. Each pixel measures an intensity with a different polarization, e.g. linear polarization at 0, +45, 45 and 90 degrees. An image may be reconstructed including polarization information from four adjacent pixels at the expense of spatial resolution. According to aspects of the present invention, the polarization information and the measured intensities at various emission angles are used to determine the thermal emission from the surface at zero emission angle, i.e. along the surface normal which may be used to determine a true temperature map of the surface.

[0018] Referring now to the drawings, reference is now made to FIG. 1, which is a simplified system drawing according to embodiments of the present invention. A thermal polarimetric camera 10 is shown imaging a surface 12. Surface 12 may be skin of a human female breast, by way of example. A Cartesian xyz coordinate system is shown with origin O at a target point of surface 12. An electromagnetic ray 14 is shown thermally emitted and entering camera 10 for simplicity along the z axis. A normal vector N to the surface at the target point is shown. Two Euler angles may be used to define the orientation of surface 12 at the target point. Target azimuthal angle is shown as a first rotation about the z axis with x, y axes rotating to x, y axes as shown until the normal vector N is in the yz plane. Target pitch angle is shown as a second rotation about the x axis so that z axis rotates to the z axis which is now collinear with the normal vector N.

[0019] Emitted ray 14 may be directed to and absorbed by an image sensor element (not shown) equipped with polarising filters as described above. Thermal polarimetric camera 10 may be configured to directly output polarization information or polarization information may otherwise be conventionally derived from the output of thermal polarimetric camera 10. Specifically, polarisation information may be in the form of the well known Stokes parameters (S.sub.0, S.sub.1, S.sub.2, DoLP) as follows:

[00001]$S_0=I_0+I_{90},\mathrm{or}{S}_0=\sqrt{I_0^2+I_{90}^2+I_{45}^2+I_{-45}^2}$$S_1=I_0-I_{90}$$S_2=I_{45}-I_{-45}$$\mathrm{DoLP}=\frac{\sqrt{S_1-S_2}}{S_0}$

[0020] where: I.sub.0, I.sub.90, I.sub.45, I.sub.45 are corrected measured intensities at an image element of thermal emission after transmission respectively through polarizing filters at 0, 90 +45 and 45 degrees. DoLP is an acronym for degree of linear polarization.

[0021] Polarisation information, e.g. Stoke's parameters may be directly related to the orientation of the surface element of surface 12 being imaged at image element of the image sensor of thermal polarimetric camera 10. Specifically, the target pitch angle is derivable from the third Stoke's parameter DoLP the degree of linear polarisation, as follows:

[00002]$\mathrm{DoLP}\left(\right)=\frac{\left(1-{.Math.r_{.Math.}\left(\right).Math.}^2\right)-\left(1-{.Math.r_{}\left(\right).Math.}^2\right)}{\left(1-{.Math.r_{.Math.}\left(\right).Math.}^2\right)+\left(1-{.Math.r_{}\left(\right).Math.}^2\right)}$

[0022] where r.sub. and r.sub. are reflectance values given by the well known Fresnel's equations for reflectance, with n equal to the complex index of refraction.

[00003]$r_{.Math.}\left(\right)=\frac{n^2\cos -\sqrt{n^2-{\sin }^2}}{n^2\cos +\sqrt{n^2-{\sin }^2}}$$r_{}\left(\right)=\frac{\cos -\sqrt{n^2-{\sin }^2}}{\cos +\sqrt{n^2-{\sin }^2}}$

[0023] Subscript .sub. indicates polarisation parallel to the plane of emittance and the subscript denotes polarisation perpendicular to the plane of emittance. The plane of emittance in FIG. 1 is the plane defined by emitted ray 14 and the normal N to surface 12.

[0024] The target azimuthal angle is derivable from Stokes parameters S.sub.2 and S.sub.1. Twice target azimuthal angle 2 is within an additive constant of:

arctan(S.sub.2/S.sub.1)

[0025] Further details with respect to determining orientation of a surface element of surface 12 from Stoke's parameters are previously disclosed in U.S. Pat. No. 9,609,238.

Extrapolating to Zero Angle to Find True Temperature

[0026] According to Kirchhoff's law of radiation as shown previously, for an arbitrary body emitting and absorbing thermal radiation in thermodynamic equilibrium, absorptance and emittance are equal and that for a smooth and opaque surface, emittance is directly related to reflectance r for each polarisation,

.sub.()=1r.sub.()

.sub.()=1r.sub.()

[0027] Using these relations the emittance at zero angle or surface normal N may be extrapolated from emittance measured at target pitch angle .

[0028] In other words, thermal intensity at zero degrees (in the direction of the normal N) may be estimated from the measured intensity at target pitch angle . The thermal intensity at zero degrees may relate to the true temperature of the surface element.

Effect of Skin Roughness, Non-Uniformity and Colour

[0029] It is known that surface roughness may effect the polarization state of thermally emitted radiation. D. L. Jordan and G. D. Lewis, Measurements of the effect of surface roughness on the polarization state of thermally emitted radiation, Optics Letter, pp. 692-4, May 1994.

[0030] In this case, Fresnel's law may not strictly apply. Moreover, non-uniformity in complex index of refraction and differences in skin colour may change intensity and polarisation state within surface 12 from person to person. However, these deviations from Fresnel's law if significant may be measured. A measurement of one or more Stoke's parameters, such as emitted intensity and/or DoLP as a function of angle may be used to calibrate the deviations from Fresnel's law as a function of target angle prior to generating a temperature map, according to features of the present invention.

[0031] Calibration may alternatively be performed by measuring reflectance as a function of angle in the far infrared wavelength, such as with a 10.6 micron wavelength CO.sub.2 laser sufficiently attenuated as to not burn the skin. Kirchhoff's law may then be applied to determine emittance.

[0032] Reference is now made to FIG. 2, which illustrates a simplified flow diagram 20, according to features of the present invention. A polarimetric image is captured (step 21) from a non-planar surface such as biological tissue, e.g. female human breast. In step 23, a polarisation state and a respective intensity are determined for thermal emission emitted from a surface element. The angular orientation, e.g. normal N to the surface element, is determined (step 25) responsive to the polarisation state and intensity. The temperature of the surface element is determined (step 27), responsive to the angular orientation of the surface element, the measured polarisation state and the measured intensity by estimating the thermal emission normal to the surface element. The temperature map generated is independent of the angular orientations of the surface elements

Reflectance Noise

[0033] It is known that the skin tends to specularly reflect far infrared radiation.

[0034] C. Villaseor-Mora, F. Snchez-Marin and S. Calixto-Carrera, An indirect skin emissivity measurement in the infrared thermal range through reflection of a CO2 laser beam, Revista mexicana de fisica, vol. 55(5), pp. 387-392, 1 Oct. 2009

[0035] Therefore, reflection of ambient far infrared radiation is a source of reflectance noise.

[0036] It can readily be understood from Kirchhoff's law and Fresnel's law of reflection that while reflected light is partially polarised parallel to the plane of incidence, thermal emission is partially polarised perpendicular to the plane of incidence. Thus, capturing of polarimetric images of thermal emissions tends to increase signal from thermal emission relative to noise due to reflectance of background radiation.

Ellipse Fit

[0037] As an alternative to the Stoke's vector formulation, the polarisation intensities, for each image element may be fit to a polarisation ellipse such as according to:

[0038] Numerically Stable Direct Least Squares Fitting of Ellipses, R Hal.Math.r, J Flusser, Proc. 6th International Conference in Central Europe, 1998.

[0039] The equation used to fit a polarisation ellipse is:

AX.sup.2+BXY+CY.sup.2+DX+EY+F=0

[0040] where A, B, C, D, E and F are parameters to be fit, and (X,Y) are points in the ellipse.

[0041] under the constraint that: B.sup.24AC<0

[0042] The centre of the ellipse (X.sub.0,Y.sub.0) is assumed to be at the origin.

[0043] Ellipse eccentricity e is given by:

[00004]$e=\frac{c}{a}=\frac{\sqrt{a^2-b^2}}{a}=\sqrt{1-\frac{b^2}{a^2}}$$\mathrm{where}a,b=\frac{-\sqrt{\begin{array}{c}2({\mathrm{AE}}^2+{\mathrm{CD}}^2-\mathrm{BDE}+\\ \left(B^2-4AC\right)F)\left(\left(AC\right)\sqrt{{\left(A-C\right)}^2{B}^2}\right)\end{array}}}{B^2-4AC}$

[0044] where a is given by selecting the + sign and b is given by selecting the sign.

[0045] Target pitch angle may be calculated from the ellipse eccentricity and a previously defined function between target pitch angle and ellipse eccentricity.

[0046] Ellipse area which relates to intensity is:

[00005]$A_{\mathrm{ellipse}}=\mathrm{ab}=\frac{2}{\sqrt{4AC-B^2}}$

[0047] Ellipse azimuth is given by:

[00006]$=\{\arctan \left(\frac{1}{B}\left(C-A-\sqrt{{\left(A-C\right)}^2+B^2}\right)\right)\mathrm{for}B0$

[0048] For each image element or pixel a fit of a polarisation ellipse is performed. The ellipse area represents intensity. Ellipse azimuth and eccentricity are extracted. The ellipse eccentricity is related to target pitch angle. Ellipse azimuth and target pitch angle are used to construct a surface contour in three dimensions.

[0049] The ellipse azimuth and the target pitch angle may be converted, as in Stoke's parameters, to Euler angles pitch, yaw with roll=0. A surface contour or mesh may be built from the pitch and yaw Euler angles for multiple surface elements.

[0050] A calibration may be performed which determines, e.g. by polynomial regression, a relation between ellipse eccentricity and target pitch angle which may account for specific skin parameters such as colour and roughness and temperature. A thermal emission at the surface element normal and a corresponding temperature may be related to the thermal emission at different target pitch angles for the various skin types.

[0051] The term angle as used herein is relative to the normal of a surface element.

[0052] The term emission angle is an angle between a ray of thermal emission and the normal of a surface element.

[0053] The term image element as used herein refers to a super-pixel of multiple pixels equipped with polarisation filters as earlier described.

[0054] The term independent referring to a temperature map being independent of the angular orientation of surface elements refers to after surface temperatures have been determined, according to embodiments of the present invention, from the estimation of thermal emission normal to the surface elements.

[0055] The embodiments of the present invention may comprise a general-purpose or special-purpose computer system including various computer hardware components, which are discussed in greater detail below. Embodiments within the scope of the present invention also include computer-readable media for carrying or having computer-executable instructions, computer-readable instructions, or data structures stored thereon. Such computer-readable media may be any available media, transitory and/or non-transitory which is accessible by a general-purpose or special-purpose computer system. By way of example, and not limitation, such computer-readable media can comprise physical storage media such as RAM, ROM, EPROM, flash disk, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other media which can be used to carry or store desired program code means in the form of computer-executable instructions, computer-readable instructions, or data structures and which may be accessed by a general-purpose or special-purpose computer system.

[0056] The transitional term comprising as used herein is synonymous with including, and is inclusive or open-ended and does not exclude additional, unrecited elements or method steps. The articles a, an is used herein, such as a polarisation state or a ray have the meaning of one or more that is one or more polarisation states, one or more rays.

[0057] All optional and preferred features and modifications of the described embodiments and dependent claims are usable in all aspects of the invention taught herein. Furthermore, the individual features of the dependent claims, as well as all optional and preferred features and modifications of the described embodiments are combinable and interchangeable with one another. Although selected features of the present invention have been shown and described, it is to be understood the present invention is not limited to the described features.