Method and electronic device for noise mitigation

Abstract

The present invention generally relates to a method for removing disturbances in an image captured by a fingerprint sensor, and specifically to reduction of noise in an acquired fingerprint image by incorporating the sensing principle into the applied method for noise reduction, wherein the noise reduced image is used for determining a representation of a fingerprint pattern. Advantages with the invention include enhanced determination of fingerprint patterns from fingerprint images captured using a fingerprint sensor. The invention also relates to a corresponding electronic device and to a computer program product.

Claims

1. A method of determining a representation of a fingerprint pattern of a finger captured using a fingerprint sensor comprising a plurality of sensing elements, the method comprising: selecting a sampling matrix, wherein the sampling matrix represents a sampling pattern corresponding to a plurality of sensing elements to be sampled at a same time for acquiring a fingerprint image using the fingerprint sensor; acquiring the fingerprint image using the fingerprint sensor according to the sampling matrix; applying a filter to the acquired fingerprint image to obtain a filtered fingerprint image, wherein setup of the filter depends on a spatial distribution of the sampling pattern of the selected sampling matrix, and the filter is provided for noise reduction within the acquired fingerprint image; and determining the representation of said fingerprint pattern based on said filtered fingerprint image, wherein the fingerprint sensor is a two-dimensional fingerprint sensor and the sampling matrix corresponds to a selected portion of the plurality of sensing elements.

2. The method according to claim 1, wherein the selected portion of the plurality of sensing elements are selected as an adjacently arranged group of sensing elements.

3. The method according to claim 1, wherein the selected portion of the plurality of sensing elements are spatially separated over the fingerprint sensor.

4. The method according to claim 1, wherein the method further comprises: determining a log-likelihood ratio for noise being present within the fingerprint image, and comparing the log-likelihood ratio with a predetermined threshold, wherein the linear filter is applied to the acquired fingerprint image only if the log-likelihood ratio is above the predetermined threshold.

5. The method according to claim 4, wherein determining a log-likelihood ratio comprises evaluating a likelihood function for an assumption that noise is not present, and determining the likelihood function for an assumption that noise is present in accordance with the sampling matrix.

6. The method according to claim 1, further comprising applying a non-linear mapping function to the acquired fingerprint image.

7. The method according to claim 6, wherein the non-linear mapping function is a logarithmic function.

8. The method according to claim 1, wherein the linear filter is a Wiener filter.

9. The method according to claim 1, wherein the method further comprises: determining a likelihood for noise being present within the fingerprint image, and comparing the determined likelihood with a predetermined threshold, wherein the linear filter is applied to the acquired fingerprint image only if the likelihood is above the predetermined threshold.

10. A portable electronic device, comprising: a fingerprint sensor having an array of pixels, and one or more processors electrically connected to the fingerprint sensor, wherein the one or more processors are configured for: selecting a sampling matrix, wherein the sampling matrix represents a sampling pattern corresponding to a plurality of sensing elements to be sampled at a same time for acquiring a fingerprint image using the fingerprint sensor; applying a linear filter to the acquired fingerprint image to obtain a filtered fingerprint image, wherein setup of the linear filter depends on a spatial distribution of the sampling pattern of the selected sampling matrix and the linear filter is provided for noise reduction within the acquired fingerprint image; and determining a representation of a fingerprint pattern based on said filtered fingerprint image, wherein the fingerprint sensor is a two-dimensional fingerprint sensor and the sampling matrix corresponds to a selected portion of the plurality of sensing elements.

11. The portable electronic device according to claim 10, wherein the fingerprint sensor employs capacitive sensing technology.

12. The portable electronic device according to claim 10, wherein the fingerprint sensor comprises at least 160160 pixels.

13. The portable electronic device according to claim 10, wherein the portable electronic device is a mobile phone.

14. The portable electronic device according to claim 10, wherein the fingerprint sensor comprises at least 192192 pixels.

15. The portable electronic device according to claim 10, wherein the fingerprint sensor comprises at least 20880 pixels.

16. A computer program product comprising a non-transitory computer readable medium having stored thereon a computer program for one or more processors adapted for controlling a portable electronic device, the portable electronic device comprising a fingerprint sensor and one or more processors, wherein the fingerprint sensor is a two-dimensional fingerprint sensor and the sampling matrix corresponds to a selected portion of the plurality of sensing elements, and wherein the computer program product comprises: code for selecting a sampling matrix, wherein the sampling matrix represents a sampling pattern corresponding to a plurality of sensing elements to be sampled at a same time for acquiring a fingerprint image using the fingerprint sensor; code for acquiring the fingerprint image using the fingerprint sensor and according to the sampling matrix; code for applying a linear filter to the acquired fingerprint image to obtain a filtered fingerprint image, wherein setup of the linear filter depends on a spatial distribution of the sampling pattern of the selected sampling matrix and the linear filter is provided for noise reduction within the acquired fingerprint image; and code for determining a representation of a fingerprint pattern based on said filtered fingerprint image.

17. The method according to claim 1, wherein selecting the sampling matrix comprises selecting the sampling matrix from a plurality of possible sampling matrix options.

18. The portable electronic device according to claim 10, wherein selecting the sampling matrix comprises selecting the sampling matrix from a plurality of possible sampling matrix options.

19. The computer program product according to claim 16, wherein the code for selecting the sampling matrix comprises code for selecting the sampling matrix from a plurality of possible sampling matrix options.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

(1) The various aspects of the invention, including its particular features and advantages, will be readily understood from the following detailed description and the accompanying drawings, in which:

(2) FIG. 1 schematically illustrates an application for a fingerprint sensing system according to an example embodiment of the present invention;

(3) FIG. 2 is a representative illustration of common-mode noise from a switched power supply;

(4) FIG. 3A schematically shows a first embodiment of the fingerprint sensing system according to the present invention;

(5) FIG. 3B schematically shows a second embodiment of the fingerprint sensing system according to the present invention;

(6) FIG. 4 illustrates a noisy fingerprint image;

(7) FIG. 5A-5E illustrates different sampling configurations;

(8) FIGS. 6A-6B is a flowchart disclosing the exemplary steps of the invention according to a currently preferred embodiment of the invention; and

(9) FIGS. 7A-7E provides a functional illustration of the flowchart shown in FIGS. 6A-6B.

DETAILED DESCRIPTION

(10) The present invention will now be described more fully hereinafter with reference to the accompanying drawings, in which currently preferred embodiments of the invention are shown. This invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided for thoroughness and completeness, and fully convey the scope of the invention to the skilled person. Like reference characters refer to like elements throughout.

(11) Turning now to the drawings and to FIG. 1 in particular, there is schematically illustrated an example application for a fingerprint sensing system according to embodiments of the present invention, in the form of a mobile phone 1 with an integrated fingerprint sensing system 2. The fingerprint sensing system 2 may, for example, be used for unlocking the mobile phone 1 and/or for authorizing transactions carried out using the mobile phone etc.

(12) As is shown in FIG. 1, the mobile phone 1 is being charged using a charger 3 connected to an AC power socket 4. The charging of the phone may introduce common-mode noise that makes the mobile phone 1 follow the electric potential of the common-mode noisethe mobile phone 1 will bounce up and down electrically.

(13) The common-mode noise will, from the perspective of the mobile phone 1, cause the user to appear electrically noisy. This may influence uses of the mobile phone 1 that rely upon an evaluation of the potential difference between the mobile phone 1 and the user. Such uses include, for example, use of a capacitive touchscreen and use of the fingerprint sensing system 2.

(14) Preferably and as is apparent for the skilled person, the mobile phone 1 shown in FIG. 1 may further comprises a first antenna for WLAN/Wi-Fi communication, a second antenna for telecommunication communication, a microphone, a speaker, and a phone control unit. Further hardware elements are of course possibly comprised with the mobile phone. It should furthermore be noted that the invention may be applicable in relation to any other type of portable electronic device, such as a laptop, a remote control, a tablet computer, or any other type of present or future similarly configured device.

(15) FIG. 2 is a representative illustration of the common-mode noise from the charger 3 in FIG. 1. As can be seen in FIG. 2, the common-mode noise may have a low frequency component (50/60 Hz depending on the AC power frequency) with relatively high amplitude, and a high frequency switch mode component (the enlarged part of FIG. 2) with a lower amplitude. In particular the high frequency component of the common-mode noise can cause problems for a fingerprint sensing system. In the following, it will be described how various embodiments of the present invention at least partly mitigate such problems.

(16) First, two embodiments of the fingerprint system according to the present inventionone touch sensor based system and one swipe sensor based systemwill be described with reference to FIGS. 3A-B.

(17) FIG. 3A schematically shows a touch sensor based fingerprint sensing system, in the form of packaged touch sensor component 9 preferably comprising a two-dimensional sensor array 10 and for example a conductive bezel or frame 11 for providing an excitation signal to the finger of the user. Although not explicitly shown in FIG. 3A, the sensor component 9 also comprises a power supply interface and a communication interface.

(18) The sensor array 10 comprises a large number of sensing elements, 12 (only one of the sensing elements has been indicated with a reference numeral to avoid cluttering the drawing), each being controllable to sense a distance between a sensing structure (top plate) comprised in the sensing element 12 and the surface of a finger contacting the top surface of the sensor array 10. In the enlarged portion of the sensor array 10 in FIG. 3A, a first group 13 of sensing elements are marked S for sensing, where the first group of sensing elements are all sensed together at one time.

(19) FIG. 3B schematically shows a swipe sensor based fingerprint sensing system, in the form of packaged swipe sensor component 19 comprising a sensor array 20 and conductive strips 21a and 21b for providing an excitation signal to the finger of the user. Although not explicitly shown in FIG. 3B, the sensor component 19 also comprises a power supply interface and a communication interface.

(20) The sensor array 20 comprises one or several lines of sensing elements, 12 (only one of the sensing elements has been indicated with a reference numeral to avoid cluttering the drawing), each being controllable to sense a distance between a sensing structure (top plate) comprised in the sensing element 12 and the surface of a finger contacting the top surface of the sensor array 20. In the enlarged portion of the sensor component 19 in FIG. 3B, a group 23 of sensing elements are marked S for sensing, in a similar manner as in regards to FIG. 3A indicating a group based sensing strategy.

(21) The fingerprint sensor components 9, 19 in FIGS. 3A - B may advantageously be manufactured using CMOS technology, but other techniques and processes may also be feasible. For instance, an insulating substrate may be used and/or thin-film technology may be utilized for some or all process steps of the manufacturing process.

(22) Turning now to FIG. 4 which illustrates a noisy fingerprint image 400 captured using a fingerprint sensor applying a capacitive measurement principle. As mentioned above, one type of noise that can occur is that of CMN, which is noise injected onto both the desired signal and the reference signal against which the desired signal is measured. If an ideal differential measurement of the desired signal is carried out against the reference signal, the CMN will therefore perfectly cancel out and hence not degrade the measurement of the desired signal. In practice it can however be very challenging to achieve good rejection of CMN due to e.g. imperfect cancellation and parasitic capacitances to other signals not impacted by the same CMN. One example is that of capacitances to earth ground, which does not follow the CMN signal and therefore directly exposes the absolute CMN level.

(23) As discussed above, this is especially a problem in systems utilizing a capacitive measurement principle, e.g. for fingerprint sensors or capacitive touch screens. Furthermore, the signal level of CMN coming from e.g. certain switch-mode power supplies in chargers may be very large, e.g. 40V peak-to-peak, as compared to the drive voltage supplied to the bezel in e.g. a fingerprint sensing system. As a result, the large signal level of the CMN can therefore be a dominating interference in the resulting fingerprint image.

(24) Due to the way that the CMN affects the measurements, the noise can typically be modeled as being multiplicative, i.e. the amount of CMN scales with the value of the fingerprint. It should be emphasized that the method according to the present invention also support other type's noise couplings, e.g. additive noise, and that a non-linear mapping function may be applied to the input signal, e.g. a logarithm function to make the multiplicative effects additive.

(25) When compared to the existing solutions, a major advantage of the method according to the present invention is that it guides the configuration of the sensing principle utilized in order to improve the CMN mitigation and then afterwards is capable of incorporating this knowledge into the CMN detection, estimation and rejection.

(26) In a possible implementation of the invention, a three step process is provided, including 1) setting up the sampling to improve mitigation of CMN, 2) detecting if CMN is present, and if so, 3) provides a means for rejecting it by filtering.

(27) Since the detection relies on similar techniques as the filtering, the filtering will be first described followed by the noise detection. The filtering typically consists of either a 1- or 2-dimensional Linear Minimum Mean Square Error (LMMSE) estimate of the CMN followed, either explicitly or implicitly, by cancellation of the CMN estimate from the input.

(28) The CMN estimate has been derived such that it takes the characteristic of the sampling process and the noise into account while it at the same time exploits the structure of signal of interest. This has the advantage that the impact of the sampling process can be assessed and optimized in order to provide the best rejection of the noise. In the example of a fingerprint sensor system, the signal of interest would be the image of the fingerprint. The same structures used for LMMSE filtering are reused in the Generalized Likelihood Ratio Testing (GLRT) described subsequently.

(29) Since the CMN may be assumed to be constant over the spatial sampling dimension, i.e. between pixels in the image, the sampling procedure is an important factor in order to separate the desired (fingerprint) signal from that of the CMN. The sampling procedure both spatially and temporally is therefore key to achieving good mitigation of the noise. Examples of such spatio-temporal sampling configurations, which may be combined as desired, are, with further reference to FIGS. 5A-5E: Individual sampling where all samples are collected independently, see FIG. 5A, Block sampling where a localized spatial area is sampled at the same time, see FIG. 5B, Overlapped sampling where overlapping blocks of spatial samples are sampled one block at a time, see FIG. 5C, Parallel sampling where non-overlapping spatial blocks are sampled independently. The blocks could for example be distributed spatially, see FIG. 5D, and Time-varying spatial sampling procedures, see FIG. 5E.

(30) The detailed mathematical description providing the link between the sampling procedure used and the filtering follows below. Throughout this description bold lowercase letters (e.g. x) denote column vectors, while bold uppercase letters denote matrices (e.g. X). The matrix transpose is denoted ().sup.T, and X.sub.i,j:k is a vector containing elements [(i,j), (i,j+1), . . . , (i, k)] in X. The identity matrix is denoted I and 0 or 1 are vectors (or matrices) where all the elements have the value 0 or 1. The matrix trace is indicated by tr{} and denotes the real valued domain.

(31) In determining the sampling to improve mitigation of CMN, let {tilde over (Y)} .sup.MN denote the image captured by sensing system where M is the number of rows and N is the number of columns in the image. The mean value of the image

(32) $\begin{array}{cc}{}_y=\frac{1}{\mathrm{MN}}\underset{i=1}{\overset{\mathrm{MN}}{.Math.}}{\tilde{y}}_i& \left(\mathrm{Equation}1\right)\end{array}$
can be subtracted from the captured image to get an image Y.sub.i,j={tilde over (Y)}.sub.i,j.sub.y with zero mean. Since such a mapping is always feasible, Y will be the image of interest in the derivation below. The vectorized version of the matrix Y is given as yvec(Y)=[Y.sub.1,1:N, Y.sub.2,1:N, . . . , Y.sub.M,1:N].sup.T which implies that the transpose of all row-vectors in Y are stacked on top of each other row-by-row. This implies that y .sup.MN and the system can be described by the following linear model

(33) $\begin{array}{cc}y=\mathrm{Sc}+f\mathrm{where}S{}^{\mathrm{MN}\frac{\mathrm{MN}}{B}}& \left(\mathrm{Equation}2\right)\end{array}$
is a spatial sampling matrix describing how the CMN vector,

(34) $c{}^{\frac{\mathrm{MN}}{B}},$
impacts the image Y where B is the spatial sampling block-size. The spatial sampling block-size hence specifies how many desired (fingerprint) samples are acquired at the same time and thus sharing the same realization of the CMN. The vector f .sup.MN denotes the desired (fingerprint) signal as well as any additional noise present. It is well known that the LMMSE estimate of the linear model in equation 2 can be computed as
.sub.LMMSE=(S.sup.T.sub.f.sup.1S+.sub.c.sup.1).sup.1S.sup.T.sub.f.sup.1y=.sub.cS.sup.T(S.sub.cS.sup.T+.sub.f).sup.1y(Equation 3)
where .sub.f denotes the spatial covariance matrix of the fingerprint and .sub.c is the covariance matrix of the CMN samples, both assumed to be zero-mean. If multiple samples in time are performed, the model can directly be extended to also cover this aspect by setting up S to be the spatio-temporal sampling matrix and covariance matrices .sub.f and .sub.c to be the spatio-temporal covariance matrices of the fingerprint and CMN respectively and thereby producing a total number of samples exceeding MN. Such spatio-temporal filtering may be beneficial if the fingerprint changes only slightly between images.

(35) Also of interest in determining a good sampling procedure is the posterior covariance matrix P.sub. of the estimate .sub.LMMSE given as
P.sub.=(S.sup.T.sub.f.sup.1S+.sub.c.sup.1).sup.1=(I.sub.cS.sup.T(S.sub.cS.sup.T+.sub.f).sup.1S).sub.c(Equation 4)

(36) Ideally there is no error in the CMN estimate, i.e. P.sub.=0, but this is in general not possible. The sampling procedure that determine S is therefore very important for the ability to reject the CMN as well as possible and the design/choice of how to sample is therefore a very important aspect of this. Under the ideal assumption that .sub.c and .sub.f are perfectly known a-priori, choosing the sampling such as to minimize the posterior error given by tr{P.sub.} is a reasonable choice, but other optimization targets may also be used. As the sampling block-size and associated constraints of the HW also play a role in this optimization, this is a non-trivial design problem where an optimization measure such as tr{P.sub.} may be used for guidance.

(37) As an example of this guidance, the special case of sampling a full image at the same time, i.e. S=1 is a vector of all ones, while assuming that .sub.f does not span this sampling space directly yields P.sub.=0 since 1.sup.T(11.sup.T+.sub.f).sup.11=1. This special case showing perfect estimation, and hence subsequent cancellation, of the CMN is achievable by having a specific sampling design which is optimized to exploit the assumed covariance structure of the fingerprint and may be highly challenging to implement in practice. However, the general principle is to optimize the sampling procedure in order to provide the best CMN estimate for subsequent cancellation by exploiting prior covariance knowledge of the fingerprint and the CMN.

(38) The general LMMSE filter described here can be used for estimating the CMN, but the complexity of this implementation will typically be large, and one possible approximation to this full solution can be obtained by utilizing a Kronecker structure over e.g. the horizontal and/or vertical dimensions. If the desired signal and the CMN can be assumed to only correlate locally, a (block) sliding-window approximation to the LMMSE filter may also be used either alone or together with the Kronecker structure to ease implementation.

(39) Let the CMN vector be given by c(0, P.sub.cI), where P.sub.c denotes the power of the CMN driving signal and denotes the normal distribution. This implies that the LMMSE solution in equation 3 is
.sub.LMMSE=(S.sup.TP.sub.f.sup.1(.sub.f,y.sup.1.sub.f,x.sup.1)S+P.sub.c.sup.1I).sup.1S.sup.TP.sub.f.sup.1(.sub.f,y.sup.1.sub.f,x.sup.1)y(Equation 5)
where it has also been assumed that the fingerprint vector is given by f(0, P.sub.f(.sub.f,y.sub.f,x)). In equation 5, P.sub.f represents the power of the signal from the fingerprint while .sub.f,y denotes the covariance in the vertical direction of the fingerprint and .sub.f,x is the covariance in the horizontal direction. If the CMN impact on the image Y can be expressed by S=S.sub.yS.sub.x, equation 5 becomes
.sub.LMMSE=((S.sub.y.sup.TS.sub.x.sup.T)P.sub.f.sup.1(.sub.f,y.sup.1.sub.f,x.sup.1)(S.sub.yS.sub.x)+P.sub.c.sup.1I).sup.1(S.sub.y.sup.TS.sub.x.sup.T)P.sub.f.sup.1(.sub.f,y.sup.1.sub.f,x.sup.1)y(Equation 6)
which implies that the overall CMN will be constrained to have noise components in the vertical and horizontal directions represented by S.sub.y and S.sub.x respectively.

(40) The sensor system can be designed such that the image Y is constructed by sampling several pixels simultaneously and this can be incorporated in the structure of S.

(41) Examples of different capturing methods are listed here Horizontal sampling where B=L.sub.x pixels in the rows of Y have been captured at the same time leads to S=I.sub.y(I.sub.x1). Vertical sampling where B=L.sub.y pixels in the columns of Y have been captured at the same time leads to S=(I.sub.y1)I.sub.x. Localized 2D sampling where a local block of B=L.sub.yL.sub.x of pixels in the image have been captured at the same time leads to S=(I.sub.y1)(I.sub.x1). General 2D sampling where a number of B of pixels in the image have been captured at the same time as indicated by the general matrix S.
where the size of vector 1 is given by the number of pixels that are captured at the same time. Inserting the special cases of S shown above in equation 6 yields solutions exploiting the sampling structure. Based on the type of CMN expected and the HW constraints regarding the achievable sampling structure, a suitable sampling structure and the associated S matrix may then be chosen based on an optimization criteria such as e.g. given by equation 4. As an example indicating how the sampling structure can be exploited when setting up the filter, the case of horizontal sampling gives

(42) $\begin{array}{cc}{\hat{c}}_{\mathrm{LMMSE}}={\left(\left(I_y.Math.\left(I_x.Math.1^T\right)\right)\left(\underset{f,y}{\overset{-1}{.Math.}}.Math.\underset{f,x}{\overset{-1}{.Math.}}\right)\left(I_y.Math.\left(I_x.Math.1\right)\right)+\frac{P_f}{P_c}\left(I_y.Math.I_x\right)\right)}^{-1}\left(I_y.Math.\left(I_x.Math.1^T\right)\right)\left(\underset{f,y}{\overset{-1}{.Math.}}.Math.\underset{f,x}{\overset{-1}{.Math.}}\right)y& \left(\mathrm{Equation}7\right)\end{array}$
This expression can be rewritten to

(43) $\begin{array}{cc}{\hat{c}}_{\mathrm{LMMSE}}={\left(\underset{f,y}{\overset{-1}{.Math.}}.Math.\left(I_x.Math.1^T\right)\underset{f,x}{\overset{-1}{.Math.}}\left(I_x.Math.1\right)+\frac{P_f}{P_c}\left(I_y.Math.I_x\right)\right)}^{-1}\left(\underset{f,y}{\overset{-1}{.Math.}}.Math.\left(I_x.Math.1^T\right)\underset{f,x}{\overset{-1}{.Math.}}\right)y& \left(\mathrm{Equation}8\right)\end{array}$

(44) By defining

(45) ${\tilde{S}}_x\stackrel{}{=}\underset{f,x}{\overset{-\frac{1}{2}}{.Math.}}{S}_x,$
which for the horizontal sampling implies that

(46) ${\tilde{S}}_x=\underset{f,x}{\overset{-\frac{1}{2}}{.Math.}}\left(I_x.Math.1\right)\mathrm{and}\tilde{y}\stackrel{}{=}\left(I_y.Math.\underset{f,x}{\overset{-\frac{1}{2}}{.Math.}}\right)y,$
the expression in equation 8 it can be simplified to

(47) $\begin{array}{cc}{\hat{c}}_{\mathrm{LMMSE}}={\left(\underset{f,y}{\overset{-1}{.Math.}}.Math.\left({\tilde{S}}_x^T{\tilde{S}}_x\right)+\frac{P_f}{P_c}I\right)}^{-1}\left(\underset{f,y}{\overset{-1}{.Math.}}.Math.{\tilde{S}}_x^T\right)\tilde{y}& \left(\mathrm{Equation}9\right)\end{array}$

(48) By performing a singular value decomposition (SVD) of {tilde over (S)}.sub.x=U.sub.xD.sub.xV.sub.x.sup.T and an eigenvalue decomposition of the covariance matrix .sub.f,y.sup.1=V.sub.f,yD.sub.f,y.sup.1V.sub.f,y.sup.T, the expression in equation 9 becomes

(49) $\begin{array}{cc}\begin{array}{c}{\hat{c}}_{\mathrm{LMMSE}}={\left(\left(V_{f,y}{D}_{f,y}^{-1}{V}_{f,y}^T\right).Math.\left(V_x{D}_x^T{D}_x{V}_x^T\right)+\frac{P_f}{P_c}I\right)}^{-1}\\ \left(\left(V_{f,y}{D}_{f,y}^{-1}{V}_{f,y}^T\right).Math.\left(V_x{D}_x^T{U}_x^T\right)\right)\tilde{y}\\ =\left(V_{f,y}.Math.V_x\right){\left(D_{f,y}^{-1}.Math.D_x^T{D}_x+\frac{P_f}{P_c}I\right)}^{-1}\\ \left(D_{f,y}^{-1}.Math.D_x^T\right){\left(V_{f,y}.Math.U_x\right)}^T\tilde{y}\end{array}& \left(\mathrm{Equation}10\right)\end{array}$

(50) Due to the Kronecker structure present in equation 10, this can be implemented efficiently by utilizing the separation between the horizontal and vertical spatial dimensions. The special case of one-dimensional processing in either the horizontal or vertical direction are special cases of the expressions above where S.sub.x and .sub.f,x or S.sub.y and .sub.f,y are diagonal matrices respectively.

(51) In the general LMMSE expression of equation 3, the covariance matrix .sub.c represents the prior information about the common-mode noise. In the Kronecker structured expressions above, so far .sub.c has been considered to be a scaled identity matrix as this simplifies the expression and leads to a lower complexity in implementation. However, a-priori knowledge concerning the CMN may be available and this can then be incorporated into .sub.c for improved estimation accuracy. However when the Kronecker structure is utilized for an efficient implementation, this Kronecker structure will only be possible to directly exploit if the log-likelihood term S.sup.T.sub.f.sup.1S and prior .sub.c share a common basis. This may be hard to achieve and can make it difficult to efficiently incorporate prior information about the CMN.

(52) A possible solution to this problem is to approximate the prior using the basis from the log-likelihood term or vice versa. An example of this is to constrain the prior to be of the form .sub.c=(V.sub.yV.sub.x)(D.sub.yD.sub.x)(V.sub.yV.sub.x).sup.T where V.sub.x and V.sub.y are the horizontal and vertical bases respectively and D.sub.x and D.sub.y are the associated spectra approximating the a-priori information for the CMN. The bases used could then be extracted from the log-likelihood term S.sup.T.sub.f.sup.1S or a common basis could be enforced for both the log-likelihood and prior terms, e.g. the fixed DCT basis.

(53) Turning now to the concept of the implementation of a Generalized Likelihood Ratio Test (GLRT), which can be conducted by examining if the multivariate distribution of the data is more likely to contain CMN or not. This can be achieved by evaluating the likelihood of the hypothesis of CMN versus the counter-hypothesis of no CMN. Given that the hypothesis can be modeled as zero-mean multivariate normal distributions, this results in H.sub.cmn:y(0, P.sub.f+c.sub.f+c) where P.sub.f+c.sub.f+c is the total signal covariance matrix of the fingerprint and CMN with P.sub.f+c being a generally unknown power level. Similarly for the counter-hypothesis of no CMN, this results in H.sub.no-cmn:y(0P.sub.f.sub.f) where P.sub.f.sub.f is the total signal covariance matrix of the fingerprint with P.sub.f being a generally unknown power level. Given the multivariate Gaussian likelihood functions, the GLRT can be written as

(54) 0$\begin{array}{cc}\begin{array}{c}\mathrm{LLR}=\ln \left(\frac{{\max }_{P_{f+c}}{.Math.2P_{f+c}\underset{f+c}{.Math.}.Math.}^{-\frac{1}{2}}\exp \left(-\frac{1}{2P_{f+c}}{y}^T\underset{f+c}{\overset{-1}{.Math.}}y\right)}{{\max }_{P_f}{.Math.2P_f\underset{f}{.Math.}.Math.}^{-\frac{1}{2}}\exp \left(-\frac{1}{2P_f}{y}^T\underset{f}{\overset{-1}{.Math.}}y\right)}\right)\\ =\ln \left(\frac{{.Math.\frac{2}{\mathrm{MN}}{y}^T\underset{f+c}{\overset{-1}{.Math.}}y\underset{f+c}{.Math.}.Math.}^{-\frac{1}{2}}\exp \left(-\frac{\mathrm{MN}}{2}\right)}{{.Math.\frac{2}{\mathrm{MN}}{y}^T\underset{f}{\overset{-1}{.Math.}}y\underset{f}{.Math.}.Math.}^{-\frac{1}{2}}\exp \left(-\frac{\mathrm{MN}}{2}\right)}\right)\\ =-\frac{1}{2}\ln \left(\frac{.Math.y^T\underset{f+c}{\overset{-1}{.Math.}}y\underset{f+c}{.Math.}.Math.}{.Math.y^T\underset{f}{\overset{-1}{.Math.}}y\underset{f}{.Math.}.Math.}\right)\\ =-\frac{\mathrm{MN}}{2}\ln \left(\frac{y^T\underset{f+c}{\overset{-1}{.Math.}}y}{y^T\underset{f}{\overset{-1}{.Math.}}y}\right)-\frac{1}{2}\ln \left(\frac{.Math.\underset{f+c}{.Math.}.Math.}{.Math.\underset{f}{.Math.}.Math.}\right)\end{array}& \left(\mathrm{Equation}11\right)\end{array}$
where CMN being present will be declared if the log-likelihood ratio score LLR exceeds a given threshold.

(55) By assuming that the fingerprint and CMN are additive and independent leading to .sub.f+c=.sub.f+S.sub.cS.sup.T, the implementation complexity of equation 11 can be reduced by using the matrix inversion lemma .sub.f+c.sup.1=C.sub.fC.sub.f.sup.TC.sub.fC.sub.f+cC.sub.f+c.sup.TC.sub.f.sup.T where the Cholesky factors are defined as
C.sub.fC.sub.f.sup.T.sub.f.sup.1
C.sub.f+cC.sub.f+c.sup.TC.sub.f.sup.TS(.sub.c.sup.1+S.sup.T.sub.f.sup.1S).sup.1S.sup.TC.sub.f(Equation 12 )

(56) This allows rewriting of equation 11 as:

(57) $\left(\mathrm{Equation}13\right)$$\begin{array}{c}\mathrm{LLR}=-\frac{\mathrm{MN}}{2}\ln \left(\frac{y^T\left(C_f{C}_f^T-C_f{C}_{f+c}{C}_{f+c}^T{C}_f^T\right)y}{y^T{C}_f{C}_f^{T}y}\right)-\frac{1}{2}\ln \left(\frac{.Math.\underset{f+c}{.Math.}.Math.}{.Math.\underset{f}{.Math.}.Math.}\right)\\ =-\frac{\mathrm{MN}}{2}\ln \left(1-\frac{{.Math.C_{f+c}^T{C}_f^{T}y.Math.}^2}{{.Math.C_f^{T}y.Math.}^2}\right)-\frac{1}{2}\ln \left(\frac{.Math.\underset{f+c}{.Math.}.Math.}{.Math.\underset{f}{.Math.}.Math.}\right)\end{array}$

(58) An alternative approach for computing an LLR for CMN detection is to directly evaluate the log-likelihood ratio using normalized covariance matrices as

(59) $\begin{array}{cc}\begin{array}{c}LLR=-\frac{1}{2}{y}^T{Q}_{f+c}^{-1}y-\frac{1}{2}{y}^T{Q}_f^{-1}y-\frac{1}{2}\ln \left(\frac{.Math.Q_{f+c}.Math.}{.Math.Q_f.Math.}\right)\\ =-\frac{1}{2}{y}^T\underset{\underset{F}{}}{\left(Q_{f+c}^{-1}-Q_f^{-1}\right)}y-\frac{1}{2}\ln \left(\frac{.Math.Q_{f+c}.Math.}{.Math.Q_f.Math.}\right)\end{array}& \left(\mathrm{Equation}14\right)\\ \mathrm{where}{Q}_{f+c}\stackrel{}{=}\frac{\mathrm{MN}}{\mathrm{tr}\left\{\underset{f+c}{.Math.}\right\}}\underset{f+c}{.Math.}\mathrm{and}{Q}_f\stackrel{}{=}\frac{\mathrm{MN}}{\mathrm{tr}\left\{\underset{f}{.Math.}\right\}}\underset{f}{.Math.}.& \end{array}$
The matrices needed to compute the chosen LLR may be either precomputed offline and put into a filter bank or may be computed online in response to updated estimates of .sub.f+c and .sub.f. If desired, a low-rank approximation of the matrix F in the alternative approach may be utilized to ease implementation.

(60) In one embodiment of the invention where the sampling is done in either the horizontal (or vertical) direction, one option is to perform a 1D LMMSE estimate and GLRT of the CMN by processing each row (or column) independently. The advantage of such a process is that this will lower the computational complexity of the LMMSE estimation.

(61) For horizontal sampling each row of the image, Y, is picked out, which implies that y=Y.sub.i,1:N for i=[1, . . . , M], which implies that the vector y is now only N-dimensional.

(62) Likewise, for vertical sampling each column of the image is picked out, which implies that y=Y.sub.1:M,j where j=[1, . . . , N], which implies that the vector y is now only M-dimensional.

(63) In such a 1D setup the Generalized Likelihood Ratio Testing (GLRT) can be performed row-by-row (or column-by-column) which implies that M log-likelihood values for the horizontal sampling (or N values for vertical sampling) can be computed for the entire image.

(64) It may also be feasible to compute a log-likelihood value (i.e. perform GLRT) for each row (or column) and these can then be used for CMN detection. In one embodiment of the invention the GLRT outputs one log-likelihood value per row (or column) and these values can be filtered/averaged over several rows (or columns) before they are compared against a threshold value which is used for CMN detection. Furthermore, the filtering of log-likelihood values may also be done by performing a sliding window averaging and the output of this can be compared against a threshold value.

(65) Turning now to FIGS. 6A-B and 7A-E in conjunction, where FIG. 6 is a flow chart showing the basic steps according to the invention for determining a fingerprint pattern using fingerprint sensor. FIGS. 7A-7E provides a corresponding functional illustration of some of the method steps. The fingerprint sensor provided in relation to FIG. 7A is a two-dimensional sensor exemplified as having 88 pixels. It should be noted that the use of 88 pixels is only used for providing a simplified explanation of the inventive method. Thus, any number of pixels may be used. As discussed above in relation to FIG. 3A, such a fingerprint sensor comprises a plurality of sensing element, each sensing element having an output for providing a pixel signal indicative of the capacitive coupling between the corresponding sensing element and a finger. Furthermore, other types of fingerprint sensors may however be used, including for example one-dimensional sensors and sensors employing different sensing techniques.

(66) Following the provision of the fingerprint sensor, fingerprint (.sub.f)and noise (CMN) (.sub.c) covariance matrices are selected. Based on the selected covariance matrices, a sampling procedure is determined and specified by a sampling matrix (S) as shown in FIG. 7B. The size of the sampling matrix (S) is defined by the size of the fingerprint sensor and the block size for sampling of a selected few of the sensing elements. In the illustration provided, the sampling size is selected to be four (i.e. sampling of four pixels at a time). According to the discussion provided in relation to Equation 2, the sampling matrix (S) is defined as

(67) $S{}^{\mathrm{MN}\frac{\mathrm{MN}}{B}},$
where M indicates the number of rows of the fingerprint sensor, N indicates the number of columns of the fingerprint sensor and B is the number of pixels sampled at one time. Based on this definition and the exemplifying size of the fingerprint sensor/number of pixels sampled at one time, a sampling matrix (S) is defined as comprising 64 rows and 16 columns.

(68) Accordingly, each of the columns of the sampling matrix shown in FIG. 7B will represent each sensor sampling. In FIG. 7B the sampling pattern is selected to correspond to the sampling strategy shown in FIG. 5B, i.e. a squared block based pattern of 22 pixels, sliding from the top left of the sensor to the bottom right of the sensor. The ones shown in FIG. 7B indicate the pixels to be sampled for each sequential sample; zeroes are shown for pixels not being sampled. As indicated in relation to FIGS. 5A-5E, other sampling strategies may be selected, preferably selected in such a way that the amount of noise within the acquired fingerprint image is minimized.

(69) Following setting up the sampling matrix/sampling strategy, the fingerprint sensor is sampled accordingly and a fingerprint image is acquired. As shown in FIG. 7C, the fingerprint image is a noisy image, where the noise pattern is dependent on the block based sequential sampling strategy discussed above.

(70) In accordance to the invention, the noise detection scheme as discussed above in relation to equation 11 and based on the selected sampling matrix is applied for determining the likelihood of noise being present in the acquired fingerprint image. The likelihood is determined by setting up two competing models, the first model being based on the assumption that noise is present, and the other model being based on the assumption that noise is not present. In accordance to the invention, at least the model used where noise is assumed to be present takes into account the selected sampling matrix. It may also be possible to take into account the selected sampling matrix in relation to the model used where noise is assumed not to be present.

(71) A log-likelihood ratio is determined and compared to a predetermined threshold. In case it is determined that the likelihood of noise (in comparison to the threshold) is too high, the process continues to a step of estimation of the noise, such as CMN, using a LMMSE filter as is discussed above in relation to equations 5-10.

(72) Also for setting up the (linear) filter to be used for noise mitigation if noise has been determined to be present will take into account the selected sampling matrix. The acquired, noisy, fingerprint image is subsequently filtered using the sampling matrix dependent filter, typically being a Wiener filter, and a clean image comprising less noise is formed, as is shown in FIG. 7D. As can be seen from FIG. 7D, the block artifacts have been removed. Once a less noisy fingerprint image has been formed, it may be possible to better determine a fingerprint pattern as is shown in FIG. 7E, including finding ridges forming e.g. arches, loops, etc., in any manner known to the skilled addressee. Further post processing for determination of a fingerprint pattern is also possible and within the scope of the invention.

(73) The control functionality of the present disclosure may be implemented using existing computer processors, or by a special purpose computer processor for an appropriate system, incorporated for this or another purpose, or by a hardwired system. Embodiments within the scope of the present disclosure include program products comprising machine-readable media for carrying or having machine-executable instructions or data structures stored thereon. Such machine-readable media can be any available media that can be accessed by a general purpose or special purpose computer or other machine with a processor. By way of example, such machine-readable media can comprise RAM, ROM, EPROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to carry or store desired program code in the form of machine-executable instructions or data structures and which can be accessed by a general purpose or special purpose computer or other machine with a processor. When information is transferred or provided over a network or another communications connection (either hardwired, wireless, or a combination of hardwired or wireless) to a machine, the machine properly views the connection as a machine-readable medium. Thus, any such connection is properly termed a machine-readable medium. Combinations of the above are also included within the scope of machine-readable media. Machine-executable instructions include, for example, instructions and data which cause a general purpose computer, special purpose computer, or special purpose processing machines to perform a certain function or group of functions.

(74) Although the figures may show a sequence the order of the steps may differ from what is depicted. Also two or more steps may be performed concurrently or with partial concurrence. Such variation will depend on the software and hardware systems chosen and on designer choice. All such variations are within the scope of the disclosure. Likewise, software implementations could be accomplished with standard programming techniques with rule based logic and other logic to accomplish the various connection steps, processing steps, comparison steps and decision steps. Additionally, even though the invention has been described with reference to specific exemplifying embodiments thereof, many different alterations, modifications and the like will become apparent for those skilled in the art.

(75) In addition, variations to the disclosed embodiments can be understood and effected by the skilled addressee in practicing the claimed invention, from a study of the drawings, the disclosure, and the appended claims. Furthermore, in the claims, the word comprising does not exclude other elements or steps, and the indefinite article a or an does not exclude a plurality.