Method and device for removing remanence in an infrared image of a static scene

Abstract

The present disclosure relates to a method of removing, by an imaging processing device, remanence artifacts from an image (f.sub.n) of a sequence of images captured by an infrared imaging device, the method comprising: generating a remanence measure for at least some pixels in the image (f.sub.n) based on a difference between the pixels values of the image (f.sub.n) and the pixel values a previous image (f.sub.n1) of the sequence; and removing remanence artifacts from at least some pixels of the image (f.sub.n) based on a remanence estimation for each of the at least some pixels, each remanence estimation being generated based on the remanence measure and on one or more previous remanence estimations of the at least some pixels and on a model of the exponential decay of the remanence.

Claims

1. A method of removing, by an imaging processing device, remanence artifacts from an image (f.sub.n) of a sequence of images captured by an infrared imaging device, the method comprising: generating a remanence measure (*.sub.n,x) for at least some pixels in the image (f.sub.n) based on a difference between the pixels values of the image (f.sub.n) and the pixel values a previous image (f.sub.n1) of the sequence; and removing remanence artifacts from at least some pixels of the image (f.sub.n) based on a remanence estimation (.sub.,n,x) for each of the at least some pixels, each remanence estimation being generated based on the remanence measure (*.sub.n,x) and on one or more previous remanence estimations of the at least some pixels and on a model (W) of the exponential decay of the remanence.

2. The method of claim 1, wherein the remanence estimation is a mean remanence estimation (.sub.,n,x) corresponding to an estimation of an instantaneous mean remanence for each of the at least some pixels.

3. The method of claim 1, wherein generating the remanence measure (*.sub.n,x) for at least some pixels in the image (f.sub.n) involves calculating:
*.sub.n,x=f.sub.n,xf.sub.n1,x[Math 9] where f.sub.n is the image and f.sub.n1 is the previous image.

4. The method of claim 1, further comprising generating the remanence estimation (.sub.,n,x) for each of the at least some pixels by applying Kalman filtering.

5. The method of claim 4, wherein generating the remanence estimation (.sub.,n,x) by applying Kalman filtering comprises: estimating a priori the remanence of each of the at least some pixels of the image (f.sub.n) based on a previous corrected remanence estimation of each of the at least some pixels and on the model (W) of the exponential decay of the remanence; and correcting, for each of the at least some pixels of the image (f.sub.n), the estimated remanence based on the corresponding remanence measure (*.sub.n,x) to obtain an a posteriori estimation.

6. The method of claim 1, further comprising detecting that the sequence of images corresponds to a static scene prior to removing the remanence artifacts from the image (f.sub.n).

7. A non-transitory storage medium storing computer instructions thereon that, when executed by a processor of an image processing device cause the processor to perform the method of claim 1.

8. An imaging processing device comprising: at least one memory storing an image (f.sub.n) of a sequence of images captured by an infrared imaging device and one or more remanence measures (*.sub.n,x) of at least one previous image in the sequence; and one or more processors configured to remove remanence artifacts from the image (f.sub.n) by: generating a remanence estimation (.sub.,n,x) for at least some pixels in the image (f.sub.n) based on a difference between the pixel values of the image (f.sub.n) and the pixel values a previous image (f.sub.n1) of the sequence; and removing remanence artifacts from at least some pixels of the image (f.sub.n) based on an estimation of the remanence generated based on the remanence estimation (.sub.,n,x) and on one or more previous estimations of the remanence of the at least some pixels and on a model (W) of the exponential decay of the remanence.

9. The image processing device of claim 8, wherein the remanence estimation is a mean remanence estimation (.sub.,n,x) corresponding to an estimation of an instantaneous mean remanence for each of the at least some pixels.

10. An infrared imaging device comprising: a microbolometer array; and the image processing device of claim 8.

11. An infrared camera comprising the infrared imaging device of claim 10.

Description

BRIEF DESCRIPTION OF DRAWINGS

(1) The foregoing features and advantages, as well as others, will be described in detail in the following description of specific embodiments given by way of illustration and not limitation with reference to the accompanying drawings, in which:

(2) FIG. 1 schematically illustrates an IR imaging device according to an example embodiment of the present disclosure;

(3) FIG. 2 schematically illustrates an image processing circuit of FIG. 1 in more detail according to an example embodiment;

(4) FIG. 3 represents an image comprising a remanence artifact;

(5) FIG. 4 is a graph representing exponential decay of the remanence of a pixel;

(6) FIG. 5 is a general flow diagram illustrating operations in a method of removing remanence from an image according to an example embodiment of the present disclosure;

(7) FIG. 6 is a graph showing an example of simulated pixel values of a pixel x suffering from remanence over a sequence of 300 images;

(8) FIG. 7 is a graph showing an example of simulated pixel values across a microbolometer array with a remanence area K;

(9) FIG. 8 is a graph showing a further example of simulated pixel values across a microbolometer array;

(10) FIG. 9 is a graph representing a distribution of the difference between remanence estimates and average remanence values;

(11) FIG. 10 is a graph representing values of estimations of the remanence of one pixel along a sequence of n=1000 consecutive images and the estimated remanence values using various methods;

(12) FIG. 11 is a flow diagram illustrating operations in a method of performing Kalman filtering to generate remanence estimations according to an example embodiment of the present disclosure; and

(13) FIG. 12 illustrates the latent structure of a valid time series for application of a Kalman filter.

DETAILED DESCRIPTION

(14) Like features have been designated by like references in the various figures. In particular, the structural and/or functional features that are common among the various embodiments may have the same references and may dispose identical structural, dimensional and material properties.

(15) Unless indicated otherwise, when reference is made to two elements connected together, this signifies a direct connection without any intermediate elements other than conductors, and when reference is made to two elements linked or coupled together, this signifies that these two elements can be connected or they can be linked or coupled via one or more other elements.

(16) In the following disclosure, unless indicated otherwise, when reference is made to absolute positional qualifiers, such as the terms front, back, top, bottom, left, right, etc., or to relative positional qualifiers, such as the terms above, below, higher, lower, etc., or to qualifiers of orientation, such as horizontal, vertical, etc., reference is made to the orientation shown in the figures, or to a microbolometer as orientated during normal use.

(17) Unless specified otherwise, the expressions around, approximately, substantially and in the order of signify within 10%, and preferably within 5%.

(18) FIG. 1 illustrates an IR imaging device 100 comprising a pixel array 102 sensitive to IR light. The IR imaging device 100 for example forms part of an IR camera. In some embodiments, the pixel array 102 is sensitive to long-wave IR light, such as light with a wavelength in the range 7 to 16 m or higher.

(19) In the example of FIG. 1, the pixel array 102 comprises 144 microbolometer pixels 104 arranged in 12 rows and 12 columns. In alternative embodiments, the pixel array 102 could comprise any number of rows and columns of pixels. Typically, the pixel array 102 for example comprises 640 by 480, or 1024 by 768 microbolometer pixels.

(20) In the example of FIG. 1, each column of pixels of the array 102 is associated with a corresponding reference structure 106. Though not functionally a picture element, this structure will be referred to herein as a reference pixel by structural analogy with the imaging (or active) microbolometer pixels 104. Furthermore, an output block (OUTPUT) 108 is coupled to each column of the pixel array 102 and to each of the reference pixels 106, and provides a raw image RAW comprising the signals or readings captured by the pixel array 102 in conjunction with the reference pixels 106.

(21) An example of a pixel array of the bolometer type is for example discussed in more detail in the U.S. Pat. No. 7,700,919, assigned to the present applicant, the contents of which is hereby incorporated by reference to the extent permitted by the law.

(22) A control circuit (CTRL) 110 for example provides control signals to the pixel array 102, to the reference pixels 106, and to the output block 108.

(23) The raw image RAW is for example provided to an image processing circuit (IMAGE PROCESSING) 112, which for example applies 2-point image correction to the pixels of the image, and also remanence correction, to produce a corrected image S.

(24) During a read operation of the pixel array 102, the rows of pixels are for example read out one at a time.

(25) FIG. 2 illustrates the image processing circuit 112 of FIG. 1 in more detail according to an example embodiment.

(26) The functions of the image processing circuit 112 are for example implemented in software, and the image processing circuit 112 comprises a processing device (PROCESSING DEVICE) 202 having one or more processors under the control of instructions stored in an instruction memory (INSTR MEMORY) 204. In alternative embodiments, the functions of the image processing circuit 112 could be implemented at least partially by dedicated hardware. In such a case, the processing device 202 for example comprises an ASIC (application specific integrated circuit) or FPGA (field programmable gate array), and the instruction memory 204 may be omitted.

(27) The processing device 202 receives the raw input image RAW, and generates the corrected image S, which is for example provided to a display (not illustrated) of the imaging device.

(28) The processing device 202 is also coupled to a data memory (MEMORY) 206 for example storing offset values (OFFSET) 208 and gain values (GAIN) 210 used to implement the 2-point image correction.

(29) In some embodiments, shutterless image correction is performed. For example, the offset values are in the form of a vector V.sub.COL representing a structural column spread, and a matrix OFF.sub.DISP representing a 2D non-column structural dispersion introduced by the pixel array 102. The column spread for example principally results from the use of the reference pixel 106 in each column, whilst the row of column reference pixels is generally not perfectly uniform. The 2D non-column dispersion for example principally results from local physical and/or structural differences between the active bolometers of the pixel array resulting for example from technological process dispersion.

(30) An example of a shutterless image correction technique and, in particular, of the generation of the vector V.sub.COL and of the matrix OFF.sub.DISP, and of the correction of pixel values based on this vector and matrix, is described in more detail in the US patent application U.S. Ser. No. 14/695,539 filed on Apr. 24, 2015 assigned to the present applicant, the contents of which is hereby incorporated by reference to the extent permitted by the law.

(31) In alternative embodiments, different types of 2-point image correction could be implemented, including solutions that involve the use of a shutter.

(32) The memory 206 also for example stores one or more remanence estimates (REMANENCE ESTIMATE(S)) 212 calculated for pixels of one or more previously captured images, as will be described in more detail below. In some embodiments, the memory 206 also stores a model W (MODEL W) representing remanence decay, as will be described in more detail below.

(33) FIG. 3 represents an image 300 in which a remanence artifact 302 is present. For example, this image was captured by a microbolometer array, and while the array was active, a glare, and in particular a flux of relatively intense heat, was received by the array. The resistance of the thermal layer of each pixel of the microbolometer array should vary as a function of temperature, the higher the temperature, the lower the resistance. In the image 300, for readability, the gray levels have been inverted: high temperatures are represented by darker pixels. The flux of relatively intense heat causes a significant, persistent, decrease in the resistance of the concerned pixels, leading to the dark trail 302 in the image 300 corresponding to the path of the flux across the microbolometer array. When resulting from the heat of the sun, such an artifact is often referred to as sunburn.

(34) FIG. 4 is a graph representing an example of the remanence .sub.x of a pixel x of a microbolometer array such as the array 102 of FIG. 1, as a function of time, according to an example embodiment. The remanence .sub.x has an exponential decay that can be modelled by the following equation:

(35) $\begin{array}{cc}{}_x\left(t-t_0;,,\right)=e^{-{\left(\frac{t-t_0}{}\right)}^{}}{1}_{t>t_0}& \left[\mathrm{Math}2\right]\end{array}$
where: t.sub.0 is the time when the pixel returned to its normal temperature following the instant of the glare hitting the pixel; x=(row, column) is the coordinate vector of the pixel; , and are parameters; and:

(36) $\begin{array}{cc}{1}_{t>t_0}=\{\begin{array}{cc}1,& \mathrm{if}t>t_0\\ 0,& \mathrm{else}\end{array}& \left[\mathrm{Math}3\right]\end{array}$
is the characteristic function. In the example of FIG. 4, =10, =1, and =0.5.

(37) Remanence is an additive perturbation. Thus the pixel values f.sub.x(t) of an image affected by remanence can be modelled as:
f.sub.x(t)=.sub.x(tt.sub.0;,,)+G.sub..sub.x(tt.sub.0)S.sub.x(t)[Math 4]
where S.sub.x(t) is the image without the glare, G.sub..sub.x is the gain change matrix due to the glare and f.sub.x(t) for example corresponds to 2-point image correction as described above. A method for performing correction based on a gain change matrix is described in more detail in French patent FR2987210, the contents of which is hereby incorporated by reference to the extent permitted by the law. However, the change in sensitivity of the pixels represented by the term G.sub..sub.x(tt.sub.0) S.sub.x(t) can be considered small with respect to ox, and in the following, its contribution will be ignored by considering the term G.sub. as equal to 1.

(38) When an IR imaging device, such as the device 100 of FIG. 1, is used to capture an IR image, there are several different use cases than can be distinguished.

(39) A dynamic situation (hereafter Case 1) can be considered in which the image in the field of view of the pixel array can be considered to be varying, because the IR camera is moving and/or because the scene is changing. For this type of dynamic situation, the above model is appropriate as both the remanence .sub.x(t) and the image S.sub.x(t) are varying in time:
f.sub.x(t)=.sub.x(t)+S.sub.x(t)[Math 5]

(40) Another situation (hereafter Case 2) corresponds to a case in which the image in the field of view of the pixel array is substantially constant. Thus, only the remanence is varying with time. In such a case, the pixel values f.sub.x(t) of an image affected by remanence can be modelled as:
f.sub.x(t)=.sub.x(t)+S.sub.x[Math 6]
where the image S.sub.x without glare can be assumed to be constant.

(41) In both Case 1 and Case 2, a stream of images is for example captured and stored to form a video file and/or relayed in near real-time to an image display of the IR camera.

(42) A third situation (hereafter Case 3) corresponds to a case in which both the image in the field of view of the IR camera and the remanence are varying sufficiently slowly that they can both be considered as constant over time, in other words the time decay is so slow that it cannot be measured within the time frame of assessment. The pixel values f.sub.x of the image in this case can be modelled as:
f.sub.x=.sub.x+S.sub.x[Math 7]

(43) Methods for at least partially removing artifacts caused by remanence for Case 1 and Case 2 will be now be described in more detail. Case 3, which is equivalent to image retouching of a single image with no additional temporal information concerning the lost pixel information, will not be addressed in the present disclosure.

(44) Below, the time is sampled by denoting t=n and t+t=n+1. Hence the value of a matrix M at time t and pixel x will be denoted M.sub.n,x instead of M(t,x).

(45) General Scheme

(46) FIG. 5 is a general flow diagram illustrating operations in a method 500 of correcting remanence in a sequence of images according to an example embodiment of the present disclosure. This method is for example implemented by the image processing circuit 112 of FIGS. 1 and 2. Alternatively, the method could be implemented by another type of processing device that may or may not form part of an IR camera.

(47) Image Capture

(48) In an operation 501, the method starts when an image f.sub.n is captured. This image for example corresponds to a raw image RAW.sub.n to which image correction has been applied, such as 2-point image correction described above.

(49) Change Detection

(50) After operation 501, an operation 502 is for example implemented in which the presence of a changing scene is detected based on the image f.sub.n and on a previously captured image f.sub.n1. The operation 502 is followed by a remanence estimation method 503 involving, for example, estimating a current remanence mean .sub.,n,x at each pixel x of the current image n. The detection of a changing scene in operation 502 is used to select between alternative methods 504 and 505 of performing the remanence estimation operation 503. In particular, if a changing scene is detected in operation 502, in the subsequent remanence estimation method 503, the method of Case 1 is applied corresponding to the method 504 in FIG. 5. If, on the contrary, a static scene is detected, in the subsequent remanence estimation method 503, the method of Case 2, corresponding to the method 505 in FIG. 5, is applied.

(51) For example, the dynamic of the scene can be detected in operation 502 by comparing a temporal change with a threshold that is set to be above the temporal noise level, for example at two or three times the temporal noise level. Such an evaluation could be performed globally for the image, such as based on average temporal changes, in which case all the pixels of the image are for example processed according to the same method (the method 504 of Case 1, or the method 505 of Case 2). Alternatively, a local evaluation could be performed based on the temporal change at individual pixels or individual sub-regions of the image, and a selection between applying the method 504 of Case 1 or the method 505 of Case 2 could be made pixel by pixel or sub-region by sub-region.

(52) The method 504 is described in more detail below in a subsection entitled Case 1, and the method 505 is described in more detail below in a subsection entitled Case 2.

(53) While FIG. 5 illustrates a method involving the execution of both methods 504 and 505, the selection between these methods being made for each frame based on the scene change detection, in alternative embodiments, only the method 504 or only the method 505 could be implemented.

(54) Remanence Correction

(55) After the operation 503, an operation 506 is for example implemented, involving correcting the image based on the estimated remanence. For example, the image is corrected by applying the following equation for each pixel x of the image f.sub.n to generate the image S.sub.n:
S.sub.n,x=f.sub.n,x.sub.,n,x[Math 8]

(56) After operation 506, n is incremented and the operations 501 to 506 are for example repeated following capture of the subsequent image n.

(57) Case 1

(58) Remanence Zone Detection

(59) In case 1, the method 503 involves applying the method 504 of FIG. 5 involving operations 510 to 513.

(60) In the operation 510, one or more remanence zones are updated.

(61) For example, a mask is generated for each image f.sub.n indicating the zones of remanence. These zones can be larger than the actual zones in which the pixels are impacted by the remanence. However, in some embodiments, the remanence zones correspond to at most 75% of the pixels of the image. The mask can for example be generated by detecting zones in which the pixel values are significantly higher than in surrounding areas, by tracking for example the sun's trajectory, although other techniques, including techniques based on edge detection, could also be used.

(62) Inpainting

(63) Then, in the operation 511, an inpainting is applied to those remanence zones in the image. Inpainting is an image reconstruction technique that is known in the art and involves correcting pixel values within the targeted zone based on image information from outside the targeted zone. As an example, a method of inpainting is described in more detail in the 2014 publication by C. Guillemot and O. Le Meur entitled Image inpainting: Overview and recent advances, IEEE signal processing magazine, 31(1), 127-144, the contents of which is hereby incorporated by reference to the extent permitted by the law.

(64) Remanence Measurement

(65) In the operation 512, a noisy measure *.sub.n,x of the remanence at pixel x in the image f.sub.n is processed for each pixel based on the inpainted image. For example, this noisy measure is calculated by performing the following subtraction pixel by pixel:
*.sub.n,x=f.sub.n,xS*.sub.x,n[Math 9]
where S*.sub.n,x is the nth inpainted image at pixel x.
Remanence Estimation

(66) In the operation 513, the remanence is estimated by applying a remanence estimation operation 530. For example, the remanence estimation comprises an estimation of an instantaneous mean remanence .sub.,n,x for each pixel x. This for example involves, for each pixel x, generating a current estimation of the remanence .sub.n,x as a weighted sum between the current remanence measure *.sub.n,x and the previous estimation of the remanence .sub.n1,x. In some embodiments, the operation 530 involves generating the estimation .sub.n,x simply as an average of the measured remanence {*.sub.k,n}.sub.k=nM:n in the previous m images. In alternative embodiments, the operation 530 involves generating the estimation .sub.n,x based on a Kalman filter that takes into account a model of temporal decay of the remanence. For example, the exponential model W may be used. Using a Kalman filter, the estimators of .sub.n,x are for example its mean .sub.,n,x and its variance .sup.2.sub.,n,x that fully characterize it.

(67) The method 504 permits the remanence to be removed based on an observation by the present inventors that, when the image in the field of view of the pixel array is changing, the temporal variation of the pixel values can be considered as noise having a normal distribution centered on .sub.,n,x.

(68) FIG. 6 is a graph showing an example of simulated pixel values (PIXEL VALUE) of a pixel x suffering from remanence over a sequence of 300 images n captured at an image capture rate of, for example, 25 frames per second. The dots in FIG. 6 represent the 300 pixel values, and a continuous curve 602 represents an estimated remanence mean over time. A dashed curve 604 in FIG. 6 represents the true remanence .sub.x, which has an exponential decay. By obtaining a reasonably accurate estimation of this true remanence .sub.x for a given image, it can be subtracted from the pixel value in order to provide the underlying pixel information representing the image scene.

(69) Indeed, considering that the image in the field of view of the pixel array is varying over time, the scene can be considered as stationary noise defined as follows:
S.sub.n,xN(.sub.S,.sub.S.sup.2)[Math 10]
where signifies is distributed according to and N(,.sup.2) is the normal distribution having average value and variance .sup.2, which are here constant over time and space.

(70) The remanence is however non-stationary noise as it has an average value that decays over time:
.sub.n,xN(.sub.,n,x,.sub..sup.2)[Math 11]

(71) The observed image at pixel x of an image f.sub.n is thus the sum of two independent random variables:
f.sub.n,x=S.sub.n,x+.sub.n,xN(.sub.S+.sub.,n,x,.sub.S.sup.2+.sub..sup.2)[Math 12]

(72) FIG. 7 is a graph illustrating simulated pixel values (PIXEL VALUE) of the pixels (PIXEL x) from 0 to P across the microbolometer array, in which it is assumed that pixels in a zone K are in the path of a sunburn.

(73) FIG. 7 represents, by a continuous curve 702, an example of the observed image f.sub.n,x, which is the sum of the average of the image scene .sub.S,n represented by a dashed curve 704 and the average remanence .sub.,n,x represented by a continuous curve 706. Therefore, by estimating the average of the image scene .sub.S,n,x, it is possible to obtain .sub.,n,x by calculating:
.sub.,n,x=f.sub.n,x.sub.S,n,x[Math 13]

(74) In the operation 513 of the method 504 of FIG. 5, the present inventors propose to obtain the estimation of the average of the image scene .sub.S,n using inpainting, which will now be described in more detail with reference to FIG. 8.

(75) FIG. 8 is a graph illustrating pixel values (PIXEL VALUE) of the pixels (PIXEL x) from 0 to P across the microbolometer array, in which it is assumed, similarly to the example of FIG. 7, that pixels in a zone K are in the path of a sunburn.

(76) A continuous curve 802 in FIG. 8 represents a line of the observed image f.sub.n, and a dashed curve 804 represents this line inpainted, denoted S*.sub.n, which can be expressed as follows:
S*.sub.n=Inpaint(f.sub.K,n|f.sub.K,n)[Math 14]
where K_bar is the zone outside the sunburn zone K. In other words, pixel information from the zone outside of the sunburn zone is used to inpaint the zone K.

(77) The curves 806 in FIG. 8 represent successive samples of S. They demonstrate that the inpainting curve 804 is not biased:
E[S*]=.sub.S[Math 15]
where E[ ] is the expected value.

(78) A dashed curve 808 represents the remanence .sub.n, and a continuous curve 810 represents the result of the subtraction f.sub.nS*.sub.n.

(79) In can be assumed that the inpainted portion of the image provides an unbiased estimation of the average of the image scene .sub.S:
S*.sub.nN(.sub.S,.sub.S.sup.2)[Math 16]
and thus an unbiased measure of .sub.,n,x:
*.sub.n,x=f.sub.n,xS*.sub.n,xN(.sub.,n,x,.sub.S.sup.2+.sub..sup.2)[Math 17]

(80) FIG. 9 is a graph representing the distribution of the magnitude (MAGNITUDE) of the difference between the measurement *.sub.n,x of the remanence and the average remanence .sub.,n,x, which corresponds to the normal distribution. Indeed, at every time instant, the measure of *.sub.n corresponding to:
*.sub.n>f.sub.nS*.sub.n[Math 18]
provides an unbiased noisy measurement of the expected value E[*.sub.n,x]=.sub.,n,x of the remanence.

(81) As this value varies with time, the present inventors propose to estimate this value for each new image f.sub.n based on a weighted sum between the current measurement *.sub.n and the previous estimate .sub.,n1, as will now be described in more detail with reference to FIG. 10.

(82) FIG. 10 is a graph representing values z.sub.n of the estimations of the remanence of one pixel x along a sequence of n=1000 consecutive frames. Each continuous curve corresponds to a different method for estimating, in the operation 530 of FIG. 5, the value The dots correspond to the raw remanence *.sub.n,x measurements. A line 1002 represents the average of these raw remanence measurements *.sub.n,x, which can be expressed as:

(83) $\begin{array}{cc}{}_{,n}=\frac{1}{n}\underset{k=1}{\overset{n}{.Math.}}{}_k^{*}& \left[\mathrm{Math}19\right]\end{array}$

(84) A dashed curve 1004 in FIG. 10 represents the true decay of the remanence of the pixel x.

(85) A continuous curve 1006 in FIG. 10 represents a moving average based on a window of 10 images, which can be expressed as:

(86) $\begin{array}{cc}{}_{,n}=\frac{1}{10}\underset{k=n-10}{\overset{n}{.Math.}}{}_k^{*}& \left[\mathrm{Math}20\right]\end{array}$

(87) The curve 1006 provides fast convergence, but is relatively noisy.

(88) A curve 1008 in FIG. 10 represents a moving average based on a window of 100 images, which can be expressed as:

(89) $\begin{array}{cc}{}_{,n}=\frac{1}{100}\underset{k=n-100}{\overset{n}{.Math.}}{}_k^{*}& \left[\mathrm{Math}21\right]\end{array}$

(90) The curve 1008 still provides relatively fast convergence, and is less noisy that the curve 1006. However, this curve is biased, and such a bias risks being visible in the final corrected image.

(91) In some embodiments, in order to obtain a reasonable compromise between speed of convergence and noise, a moving average could be used based on a sliding window of m images, where m is in the range 20 to 150 and for example in the range 20 to 80. In some embodiments, the number m of images is variable with time, for example increasing in time as the image sequence progresses. In some embodiments, there may be drawbacks with this approach based on a sliding window, if for example the result is still biased to some extent, and if there is a need to calibrate the size of the sliding window.

(92) A curve 1010 in FIG. 10 corresponds to the application of a Kalman filter, which has a relatively slow convergence in comparison with the moving average, but converges with relatively high precision to the true decay of the remanence represented by the curve 1004. The Kalman filter takes into account measurements of the remanence for a plurality of images in the sequence, and also the model W of the exponential decay of the remanence. The use of a Kalman filter in the operation 530 will now be described in more detail with reference to FIGS. 11 and 12.

(93) Kalman Filtering

(94) FIG. 11 is a flow diagram illustrating operations in a method of estimating the average remanence value .sub.,n,x using a Kalman filter in the operation 513 of FIG. 5. This method is for example implemented by the image processing circuit 112 of FIGS. 1 and 2. Alternatively, the method could be implemented by another type of processing device that may or may not form part of an IR camera.

(95) Kalman filters are well suited for estimating a latent (hidden) state vector .sub.n at each step n of a linear-Gaussian dynamical time series using a noisy (observed) measure .sub.n* and knowing some hyper-parameters. Such time series are a kind of Markov chain and can be fully described by the following simple sequence:
.sub.n=A.sub.n1+v,
*.sub.n=H.sub.n+,
.sub.0=.sub.,0+,[Math 22]
where A is the transition matrix, H is the measurement matrix, and the parameters v, u and are some zero-mean normal additive noises of covariance matrices , .sub.,0 and R respectively.

(96) FIG. 12 illustrates an example of such time series, where the hidden state vector (of dimension 1 in this example) are the circles and the observed measures are the dots. The inputs from a time (tt) are the previous corrected estimates of the remanence (tt), and the transition matrix A linking the previous estimate at time (tt) to the new estimate at time t. Because everything is Gaussian in this time series, the state vector is fully characterized by its mean vector .sub. and covariance matrix .sub., which are the unknown values to be determined.

(97) At each step n, it is shown [Murphy, Machine Learning, a probabilistic perspective, 2012 p. 638] that knowing the (time-constant) hyper-parameters , R, U, A, H and .sub.0, the latent state .sub.n may be estimated using the following equations:
.sub.,n,x.sup.=A.sub.n1,x,
.sub.,n,x.sup.=A.sub.,n1,xA.sup.T+,
.sub.,n,x=.sub.,n,x.sup.+K.sub.n,x(*.sub.n,xH.sub.,n,x.sup.),
.sub.,n,x=(IK.sub.n,xH).sub.,n,x.sup.,[Math 23]
where:
K.sub.n,x=.sub.,n,x.sup.H.sup.T(H.sub.,x,n.sup.H.sup.T+R).sup.1[Math 24]

(98) is the Kalman gain and I is the identity matrix.

(99) In other words, at each step n, a new a posteriori estimation (.sub.,n,x, .sub.,n,x) is estimated from a measurement .sub.n* and an a priori estimation (.sub.,n,x.sup., .sup..sub., n,x).

(100) Application to Remanence Filtering

(101) Covariance Matrices

(102) In the present disclosure, the hidden variable is the remanence at each pixel and is a scalar. Hence the covariance matrices , R, .sub.,0, .sub. are only variances denoted respectively , r, .sub.,0.sup.2, .sub..sup.2.

(103) Transition Matrix A

(104) In the present disclosure, the hidden state is scalar and the transition matrix A is also a scalar noted a and is directly related to the model W of the remanence. As described in relation with FIG. 4 above:

(105) $\begin{array}{cc}W=\left(t\right)=e^{-{\left(\frac{t-t_0}{}\right)}^{}}{1}_{t>t_0}& \left[\mathrm{Math}25\right]\\ \frac{\left(t\right)}{t}=-\frac{}{}{\left(\frac{t-t_0}{}\right)}^{-1}\left(t\right)& \left[\mathrm{Math}26\right]\end{array}$

(106) In the present example, the parameter is assumed to equal 1, leading to a model for (t+t) as follows:

(107) $\begin{array}{cc}\left(t+t\right)=a\left(t\right),\mathrm{with}a=1-\frac{t}{}& \left[\mathrm{Math}27\right]\end{array}$

(108) The transition matrix a is for example common for all pixels of the array. Alternatively, a different transition matrix a.sub.x could be stored for each pixel.

(109) Measurement Matrix H

(110) As it is described above the measure .sub.n* is also a real scalar value which is directly related to the hidden state using the measurement matrix H:
*.sub.n,x=H.sub.n,x+, with H=1[Math 28]
Initialisation

(111) In an operation 1101 of the method 530 of FIG. 11, the Kalman parameters are initialized by initializing the hidden state of the Kalman filter. For example, an initial average .sub.,0 and variance .sub.,0.sup.2 values may be initialized for all pixels x as:
.sub.,0,x=0,x[Math 29]
and
.sub.,0,x.sup.2=999,x[Math 30]
A Priori Estimation

(112) In an operation 1102, a new a priori estimate is generated for the average remanence based on the noisy measure of remanence, and on the Kalman parameters and the Kalman hyper-parameters. For example, the new a priori estimate is generated for the average remanence at a time n+1:
.sub.,n,x.sup.=a.sub.,n1,x
.sub.,n,x.sup.2=a.sup.2.sub.,n1,x.sup.2+[Math 31]
where is the variance of the model, and is for example user-defined.
A Posteriori Estimation

(113) In an operation 1103, the prior new mean and variance values of the remanence are corrected based on the Kalman parameters, the Kalman hyper-parameters, and the current measured remanence value .sub.n,x*, determined based on the inpainted image:
.sub.,n,x=(1K.sub.n,x).sub.,n,x.sup.+K.sub.n,x*.sub.n,x
.sub.,n,x.sup.2=(1K.sub.n,x).sub.,n,x.sup.2[Math 32]
where:
K.sub.n,x=.sub.,n,x.sup.2(.sub.,n,x.sup.2+r).sup.1[Math 33]
is the Kalman gain.

(114) In other words, an a posteriori estimate of the remanence is generated as the balance between the prior estimate and the measure, and the balance is tuned by the Kalman gain K.

(115) Sequence

(116) With reference again to FIG. 11, after operation 1103, the Kalman parameters are for example updated. As represented in FIG. 5, the method 530 may then be repeated for the next image n+1, involving repeating the operations 1102 and 1103 based on the updated Kalman parameters. Indeed, the parameter initialization operation 1101 is for example performed only on a first iteration of the method of FIG. 5.

(117) Case 2

(118) In case 2, the operation 503 of FIG. 5 involves the method 505 corresponding to a method of correcting remanence in a sequence of images according to a further example embodiment of the present disclosure. This method is for example implemented by the image processing circuit 112 of FIGS. 1 and 2. Alternatively, the method could be implemented by another type of processing device that may or may not form part of an IR camera.

(119) The method 505 involves operations 520 and 521.

(120) In the operation 520, a measure of the remanence in the image f.sub.n is calculated for each pixel based on a previous image f.sub.n1. For example, a measure *.sub.n,x of the remanence is calculated for each pixel x of the image by performing the following subtraction:
*.sub.n,x=f.sub.n,xf.sub.n1,x[Math 34]

(121) In the operation 521, the remanence .sub.n,x of each pixel is estimated based on the remanence measurement and on the exponential decay model W. For example, the remanence estimation comprises an estimation of an instantaneous mean remanence .sub.,n,x for each pixel x. In some embodiments, the remanence mean .sub.,n,x and its variance .sup.2.sub.,n,x are estimated using a Kalman filter as described above in relation with FIG. 11. However, for the case of a static scene, the measurement * is very noisy and therefore, for Case 2, the transition matrix a should generally be more accurate than in Case 1. Therefore, in some embodiments, a local transition matrix a.sub.x=1t/.sub.x of exponential decay of the remanence is stored in the memory 206 for each pixel x. These transition matrices are for example estimated during a preprocessing operation, involving, for example, regressing the exponential decay on two or more of the first frames just following the occurrence of the remanence for estimating .sub.x. Moreover, in case 2, the transition matrix is:

(122) $\begin{array}{cc}{H}_x=1-\exp \left(\frac{t}{{}_x}\right)& \left[\mathrm{Math}35\right]\end{array}$

(123) An advantage of the embodiments described herein is that remanence can be corrected in sequences of images for the case of a changing scene (Case 1) or a static scene (Case 2), in a relatively simple and effective manner.

(124) Various embodiments and variants have been described. Those skilled in the art will understand that certain features of these embodiments can be combined and other variants will readily occur to those skilled in the art. For example, it will be apparent to those skilled in the art that the choice between providing a common exponential decay model W for all pixels, or different models for each pixel x, will depend on the required precision and on the characteristics of the IR pixel array.