System and method for obtaining thermal image data of a body part and thermal imager

Abstract

A computer-implemented method for obtaining thermal image data of a body part and a thermal imager are disclosed. The method comprises receiving image data of a body part, the image data including thermal image data, the thermal image data including thermal image data of the body part and background thermal image data, applying a classifier to the image data to detect a profile of the body part, the classifier being configured to use image data other than the thermal image data to perform the detection, generating a mask from said profile, and applying the mask to the thermal image data to remove the background thermal image data and obtain thermal image data of the body part.

Claims

1. A computer-implemented method for obtaining thermal image data of a body part, the method being executed in a handheld device including a display and a thermal imager and comprising: receiving image data of a body part, the image data including thermal image data, the thermal image data including thermal image data of the body part and background thermal image data; applying a classifier to the image data to detect a profile of the body part, the classifier being configured to use image data other than the thermal image data to perform the detection; generating a mask from said profile; applying the mask to the thermal image data to remove the background thermal image data and obtain thermal image data of the body part, wherein the image data includes visual image data captured from the same scene as the thermal image data, the classifier detecting the profile of the body part from the visual image data, wherein the image data includes visual stereo image data captured from the same scene as the thermal image data, the method further comprising displaying via the display a live scene display of the scene to be captured by the thermal camera to guide positioning of the handheld device by the user such that the body part is in image.

2. The method of claim 1, wherein the step of applying a classifier includes applying an artificial neural network trained from image data of body parts to the received image data to detect the profile of the body part.

3. The method of claim 1, further comprising determining a spatial orientation of the body part by the classifier from the visual stereo image data and modifying spatial orientation of the obtained thermal image data of the body part in dependence on the determined spatial orientation to match a default spatial orientation.

4. A handheld thermal imager including a thermal camera, a display, and a processor, the processor being configured to execute computer program code for executing a classifier, including: computer program code configured to receive image data of a body part, the image data including thermal image data from the thermal camera, the thermal image data including thermal image data of the body part and background thermal image data; computer program code configured to apply a classifier to the image data to detect a profile of the body part, the classifier being configured to use image data other than the thermal image data to perform the detection; computer program code configured to generate a mask from said profile; computer program code to apply the mask to the thermal image data to remove the background thermal image data and obtain thermal image data of the body part, further comprising a visual imager configured to capture visual image data on the same scene as the thermal image data, the computer program code configured to apply the classifier detecting the profile of the body part from the visual image data, wherein the visual imager includes a stereo visual imager, the display configured to display a live scene display of the scene to be captured by the thermal camera.

5. The thermal imager of claim 4, wherein the computer program code configured to apply a classifier includes computer program code configured to apply an artificial neural network trained from image data of body parts to the received image data to detect the profile of the body part.

6. The thermal imager of claim 4, wherein the processor is further configured to execute computer program code to determine a spatial orientation of the body part by the classifier from stereo image data from the stereo visual imager and modify spatial orientation of the obtained thermal image data of the body part in dependence on the determined spatial orientation to match a default spatial orientation.

7. A computer-implemented method for obtaining thermal image data of a body part comprising: receiving image data of a body part, the image data including thermal image data, the thermal image data including thermal image data of the body part and background thermal image data; applying a classifier to the image data to detect a profile of the body part, the classifier being configured to use image data other than the thermal image data to perform the detection; generating a mask from said profile; applying the mask to the thermal image data to remove the background thermal image data and obtain thermal image data of the body part, wherein the image data includes visual image data captured from the same scene as the thermal image data, the classifier detecting the profile of the body part from the visual image data, wherein the image data includes visual stereo image data captured from the same scene as the thermal image data, the method further comprising generating a 3D model of the body part and projecting the thermal imaging data into the 3D model.

8. The thermal imager of claim 4, further comprising a visual alignment tool to guide alignment of the body part.

9. The thermal imager of claim 8, wherein the visual alignment tool comprises an on-screen overlay on the display to guide alignment of the body part.

10. The thermal imager of claim 8, wherein the visual alignment tool comprises a light emitter configured to project visual alignment guides from the handheld device to be positioned at or around the body part.

11. A handheld thermal imager including a thermal camera, a visual imager configured to capture visual image data on the same scene as imaged by the thermal camera wherein the visual imager includes a stereo visual imager, and a processor, the processor being configured to execute computer program code for executing a classifier, including: computer program code configured to receive image data of a body part, the image data including thermal image data from the thermal camera, the thermal image data including thermal image data of the body part and background thermal image data; computer program code configured to apply a classifier to the image data to detect a profile of the body part, the classifier being configured to use image data other than the thermal image data to perform the detection; computer program code configured to generate a mask from said profile; computer program code to apply the mask to the thermal image data to remove the background thermal image data and obtain thermal image data of the body part, the computer program code configured to apply the classifier detecting the profile of the body part from the visual image data, the processor being further configured to execute computer program code to generate a 3D model of the body part and project the thermal imaging data into the 3D model.

12. The method of claim 7, wherein the step of applying a classifier includes applying an artificial neural network trained from image data of body parts to the received image data to detect the profile of the body part.

13. The method of claim 7, further comprising determining a spatial orientation of the body part by the classifier from the visual stereo image data and modifying spatial orientation of the obtained thermal image data of the body part in dependence on the determined spatial orientation to match a default spatial orientation.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

(1) Embodiments of the present invention will now be described by way of example only with reference to the accompanying drawings in which:

(2) FIG. 1 is a schematic diagram of a thermal imager according to an embodiment of the present invention;

(3) FIG. 2 is a schematic diagram of a thermal image processing system according to an embodiment of the present invention;

(4) FIG. 3 is a schematic diagram illustrating aspects of the system of FIG. 2 in more detail;

(5) FIG. 4 is a schematic diagram illustrating aspects of the system of FIG. 2 in more detail; and,

(6) FIG. 5 is a schematic diagram illustrating alternate aspects of the system of FIG. 2 in more detail.

DETAILED DESCRIPTION

(7) FIG. 1 is a schematic diagram of a thermal imager according to an embodiment of the present invention.

(8) The thermal imager 10 is preferably of a hand-held form factor and includes a thermal camera 20, a battery power source 30, a processor 40, a data repository 50 and a touchscreen display 60.

(9) The thermal imager also includes a thermal image processing system 100 that is executed by the processor 40 to obtain thermal image data on a body part from raw thermal image data captured by the thermal camera 20. Further details on the thermal image processing system are set out below with reference to FIGS. 2-4.

(10) In operation, the user positions the thermal imager such that the body part to be imaged is within the field of view of the thermal camera. Output of the thermal camera may be displayed on-screen to aid this.

(11) Preferably, the thermal imager provides the user with a live scene display and the imager is orientated by the user such that the object of interest (foot) is in image. Alignment tools are preferably used such as a distance pointer/on screen overlays (akin to visual camera systems) etc. When the user is happy with orientation a button is depressed (or screen tap) and an image is captured.

(12) Optionally, alignment guides may be projected from the device using LED's or the like. In such an arrangement, the user points the alignment guides at specific features of the target area(s) of the body part or alternatively positions the body part/thermal imager such that the body part falls inside the projected alignment guides.

(13) A thermal image of the body part is then produced by the thermal image processing system 100 and written to the data repository 50. This may be displayed on screen, compared to prior thermal images of the body part in the data repository 50, to an image of the opposite foot and/or used to trigger an alert and/or highlight areas of potential concern exhibiting temperatures indicative of inflammation.

(14) Where inflammation may arise in different parts/sides of a body part, the thermal imager 10 may include capture protocols in the form of computer program instructions that are executed by the processor 40 and guide the user (for example by on-screen prompts on the display 60) to follow a predetermined measurement protocol to capture predetermined views of the body part. In the case of a foot, these view may include (in no particular order): left foot (sole—plantar), right foot (sole—plantar), left foot (top—dorsal), right foot (top—dorsal), left foot (outer side—lateral), right foot (outer side—lateral), left foot (inner side—medial), right foot (inner side—medial).

(15) Preferably, the thermal imager is a micro-bolometer array operating at 8-14 μm. The micro-bolometer array is preferably a staring focal plane array in which its pixel elements are micro-bolometers that are sensitive to infrared radiation/temperature. In one embodiment, the pixels are each 17-25 μm in size. Infrared radiation is emitted by every object above absolute 0. Infrared radiation has a strong (T.sup.4) relationship with temperature—such that temperature of an object can be determined from measuring its emitted infrared radiation. A thermal image is captured by measuring the infrared radiation received at each pixel of the array. This data may be encoded/compressed for storage efficiency (such as run length encoding) and can be handled using image processing in a similar way to a pixel-based visual image.

(16) It will be appreciated that the touchscreen display could be replaced by another user interface such as a display and keypad or it could be replaced by some other arrangement such as a Bluetooth connection to a smartphone or the like (which then acts as a remote user interface for controlling the thermal imaging system 10 and viewing imagery).

(17) The thermal imager 10 may also include an I/O system for wired or wireless communication of obtained thermal images or other data. The battery power source 30 may be rechargeable and/or replaceable.

(18) FIG. 2 is a schematic diagram of a thermal image processing system for obtaining thermal image data of a body part according to an embodiment of the present invention.

(19) The thermal image processing system 100 is configured to receive thermal image data 110. The thermal image data includes both thermal image data of the body part 111 and background thermal image data 112.

(20) In order to extract the thermal image data of the body part 111, the thermal image processing system 100 applies a classifier 120 to detect a profile of the body part. The classifier uses image data 130 other than the thermal image data 110 to perform the detection, examples of which are described with reference to FIGS. 3 and 4 below.

(21) Once a profile has been detected, it is used by the thermal image processing system to generate a mask 113. The thermal image processing system then applies the mask 113 to the thermal image data 110 to remove the background thermal image data 112 and obtain thermal image data of the body part 111.

(22) FIG. 3 is a schematic diagram illustrating aspects of the system of FIG. 2 in more detail.

(23) In one embodiment, the thermal imager may also include a visual imaging sensor 70 that is configured to capture the visual spectrum of the imaged scene to generate visual image data 130a (this may be stored separately to the thermal image data 110 or they may be stored collectively). The visual imaging sensor 70 may capture an RGB image of the scene, for example.

(24) FIG. 4 is a schematic diagram illustrating aspects of the system of FIG. 2 in more detail.

(25) In a preferred embodiment, the thermal imager includes a stereo visual imaging sensor 80 (in one embodiment this is a pair of visual imaging sensors having a predetermined orientation and distance therebetween such that stereo image processing can be performed).

(26) The stereo visual imaging sensor 80 is configured to capture a stereo image of the visual spectrum of the imaged scene to generate visual image data 130b (as before, this may be stored separately to the thermal image data 110 or they may be stored collectively). The visual imaging sensor 80 may capture a pair of RGB images of the scene, for example. In this embodiment, the classifier 120 uses the stereo visual image data 130b to detect the profile of the body part from the visual image data.

(27) The classifier may optionally be configured to determine a spatial orientation of the body part from the image data 130. For example, the classifier may use the stereo visual image data to determine spatial orientation of the body part and the thermal image processing system may then modify spatial orientation of the obtained thermal image data of the body part in dependence on the determined spatial orientation to match a default spatial orientation.

(28) In one embodiment, the thermal image processing system 100 may use the stereo visual image data 130b to generate a 3D model (preferably a 3D topography mesh) of the body part onto which the thermal imaging data is registered. The 3D model may be generated using photometric stereo, structure from motion or other approaches.

(29) Further data may be overlaid onto the 3D model such as interfacial pressure data, thermal measurements from the thermal image data such as Max, Min, Mean, Span, Histogram etc.

(30) FIG. 5 is a schematic diagram illustrating alternate aspects of the system of FIG. 2 in more detail.

(31) In this embodiment, the classifier 120 includes an artificial neural network that has been trained from image data 130 other than the thermal image data including image data of body parts and is configured to detect the profile of the body part. In this embodiment, the image data other than the thermal image data may or may not have been of that particular person's body part and may include thermal image data, visual image data or other image data (or a combination).

(32) The neural network may, for example, be a convolutional neural network. The artificial neural network may be self-organising, feed-forward

(33) It is to be appreciated that certain embodiments of the invention as discussed above may be incorporated as code (e.g., a software algorithm or program) residing in firmware and/or on computer useable medium having control logic for enabling execution on a computer system having a computer processor. Such a computer system typically includes memory storage configured to provide output from execution of the code which configures a processor in accordance with the execution. The code can be arranged as firmware or software, and can be organized as a set of modules such as discrete code modules, function calls, procedure calls or objects in an object-oriented programming environment. If implemented using modules, the code can comprise a single module or a plurality of modules that operate in cooperation with one another.

(34) Optional embodiments of the invention can be understood as including the parts, elements and features referred to or indicated herein, individually or collectively, in any or all combinations of two or more of the parts, elements or features, and wherein specific integers are mentioned herein which have known equivalents in the art to which the invention relates, such known equivalents are deemed to be incorporated herein as if individually set forth.

(35) Although illustrated embodiments of the present invention have been described, it should be understood that various changes, substitutions, and alterations can be made by one of ordinary skill in the art without departing from the present invention which is defined by the recitations in the claims below and equivalents thereof.