VARIABLE PIXEL BINNING IN AN OPTICAL BIOMETRIC IMAGING DEVICE

Abstract

The present invention relates to a method of configuring an optical biometric imaging device comprising a photodetector pixel array, the method comprising: acquiring a biometric image of a biometric object using a first binning setting for binning of detection signals from subarrays of the photodetector pixels into a single pixel value;determining a measure indicative of the feature resolution of the biometric image; and determining, when the measure indicates that the feature resolution is below a feature resolution threshold, a second binning setting adapted to provide a higher feature resolution compared to the feature resolution of the biometric image.

Claims

1. A method of configuring an optical biometric imaging device comprising a photodetector pixel array, the method comprising: acquiring a biometric image of a biometric object using a first binning setting for binning of detection signals from subarrays of the photodetector pixels into a single pixel value; determining a measure indicative of the feature resolution of the biometric image; determining, when the measure indicates that the feature resolution is below a feature resolution threshold, a second binning setting adapted to provide a higher feature resolution compared to the feature resolution of the biometric image; and acquiring a second biometric image of the biometric object using the second binning setting.

2. The method according to claim 1, wherein the measure indicative of the feature resolution is a projected pixel size.

3. The method according to claim 1, wherein with the second binning setting, detection signals from fewer pixels are combined compared to with the first binning setting.

4. The method according to claim 1, comprising: determining a second measure indicative of the feature resolution of the second biometric image; determining, when the second measure indicates that the feature resolution is below the feature resolution threshold, a further binning setting adapted to provide a higher feature resolution compared to the feature resolution of the second biometric image.

5. The method according to claim 4, comprising: acquiring further images with further binning settings for which further measures indicative of the feature resolution are determined, until the measure indicates a feature resolution which exceeds the feature resolution threshold, selecting the binning setting for which the respective measure indicates a feature resolution that exceeds the feature resolution threshold.

6. The method according to claim 1, wherein subsequent binning settings reduces the number of rows of pixels in each bin by one, and the number of columns of pixels by one.

7. The method according to claim 1, wherein subsequent binning settings reduces the number of rows of pixels in each bin by more than one, and the number of columns of pixels by more than one.

8. The method according to claim 1, wherein the subarray of photodetector pixels is square, rectangular, or circular subarray.

9. The method according to claim 1, further comprising: determining, when the measure indicates that the feature resolution exceeds a second feature resolution threshold, a third binning setting adapted to provide a lower feature resolution compared to the feature resolution of the biometric image.

10. The method according to claim 9, wherein the feature resolution threshold is a first feature resolution threshold, wherein the second feature resolution threshold is indicative of a higher feature resolution than the first feature resolution.

11. An optical biometric imaging device configured to be arranged under an at least partially transparent display panel and configured to capture an image of an object located on an opposite side of the transparent display panel, the biometric imaging device comprising: an image sensor comprising a photodetector pixel array; an array of light defining elements, wherein each light defining element is configured to define a subarray of pixels which receives light from the corresponding light defining element, a control unit connected to the image sensor and configured to: control the image sensor to acquire a first biometric image of a biometric object using a first binning setting for binning of detection signals from subarrays of the photodetector pixels into a single pixel value; determine a measure indicative of the feature resolution of the first biometric image, determine, when the measure indicates that the feature resolution is below a feature resolution threshold, a second binning setting adapted to provide a higher feature resolution compared to the feature resolution of the first biometric image; and control the image sensor to acquire a second biometric image of the biometric object using the second binning setting.

12. The optical biometric imaging device according to claim 11, wherein the light defining elements are microlenses.

13. The optical biometric imaging device according to claim 12, wherein each microlenses are configured to redirect light onto a subarray of pixels in the photodetector pixel array.

14. The optical biometric imaging device according to claim 11, configured as an optical fingerprint imaging device.

15. An electronic device comprising: a transparent display panel; the optical biometric imaging device according to claim 11, and processing circuitry configured to: receive a signal from the optical biometric imaging device indicative of a fingerprint of a finger touching the transparent display panel, perform a fingerprint authentication procedure based on the detected fingerprint.

16. The electronic device according to claim 15, wherein the electronic device is a mobile device.

17. A computer program product comprising a non-transitory computer readable medium having stored thereon computer program means for controlling an optical biometric imaging device, the optical biometric imaging device comprising an image sensor and a control unit, wherein the computer program product comprises: code for determining a measure indicative of the feature resolution of a biometric image acquired with a first binning setting; and code for determining, when the measure indicates that the feature resolution is below a feature resolution threshold, a second binning setting adapted to provide a higher feature resolution compared to the feature resolution of the biometric image; and code for controlling the image sensor to acquire a second biometric image of the biometric object using the second binning setting.

18. A method of configuring an optical biometric imaging device comprising a photodetector pixel array, the method comprising: acquiring a biometric image of a biometric object using a first binning setting for binning of detection signals from subarrays of the photodetector pixels into a single pixel value; determining a measure indicative of the feature resolution of the biometric image; and determining, when the measure indicates that the feature resolution exceeds a second feature resolution threshold, a third binning setting adapted to provide a lower feature resolution compared to the feature resolution of the biometric image; acquiring a biometric image of the biometric object using the third binning setting.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0050] These and other aspects of the present invention will now be described in more detail, with reference to the appended drawings showing an example embodiment of the invention, wherein:

[0051] FIG. 1 schematically illustrates an example of an electronic device according to embodiments of the invention;

[0052] FIG. 2 is a schematic box diagram of an electronic device according to embodiments of the invention;

[0053] FIG. 3a schematically illustrates a biometric imaging device according to an embodiment of the invention;

[0054] FIG. 3b schematically illustrates a further configuration of the biometric imaging device illustrated in FIG. 3a;

[0055] FIG. 4a-c conceptually illustrates the concept of binning of pixels;

[0056] FIG. 5a-c conceptually illustrates another example of binning of pixels;

[0057] FIG. 6a-e conceptually illustrate the effect of binning;

[0058] FIG. 7 is a flow-chart of method steps according to embodiments of the invention;

[0059] FIG. 8 is a flow-chart of method steps according to embodiments of the invention;

[0060] FIG. 9 is a flow-chart of method steps according to embodiments of the invention; and

[0061] FIG. 10 is a flow-chart of method steps according to embodiments of the invention.

DETAILED DESCRIPTION OF EXAMPLE EMBODIMENTS

[0062] In the present detailed description, various embodiments of the biometric imaging device according to the present invention are mainly described with reference to a biometric imaging device arranged under a display panel. However, it should be noted that the described biometric imaging device also may be used in other biometric imaging applications such as in a biometric imaging devices located under a cover glass or the like.

[0063] Turning now to the drawings and in particular to FIG. 1, there is schematically illustrated an example of an electronic device configured to apply the concept according to the present disclosure, in the form of a mobile device 101 with an integrated in-display optical biometric imaging device 100 and a display panel 104 with a touch screen interface 106. The optical biometric imaging device 100 may, for example, be used for unlocking the mobile device 100 and/or for authorizing transactions carried out using the mobile device 100, etc.

[0064] The optical biometric imaging device 100 is here shown to be smaller than the display panel 104, but still relatively large, e.g. a large area implementation. In other advantageous implementations the biometric imaging device 100 may be the same size as the display panel 104, i.e. a full display solution. Thus, in such case the user may place his/her finger anywhere on the display panel for biometric authentication. The biometric imaging device 100 may in other possible implementations be smaller than the depicted biometric imaging device, such as providing a hot-zone implementation.

[0065] Preferably and as is apparent for the skilled person, the mobile device 101 shown in FIG. 1 may further comprise a first antenna for WLAN/Wi-Fi communication, a second antenna for telecommunication communication, a microphone, a speaker, and a phone control unit. Further hardware elements are of course possibly comprised with the mobile device.

[0066] It should furthermore be noted that the invention may be applicable in relation to any other type of electronic devices comprising transparent display panels, such as a laptop, a tablet computer, etc.

[0067] FIG. 2 is a schematic box diagram of an electronic device 200 according to embodiments of the invention. The electronic device 200 comprises a transparent display panel 204 and an optical biometric imaging device 100 conceptually illustrated to be arranged under the transparent display panel 204. Furthermore, the electronic device 200 comprises processing circuitry such as control unit 202. The control unit 202 may be stand-alone control unit of the electronic device 202, e.g. a device controller. Alternatively, the control unit 202 may be comprised in the optical biometric imaging device 100.

[0068] The control unit 202 is configured to receive a signal indicative of a detected object from the biometric imaging device 100. The received signal may comprise image data.

[0069] Based on the received signal the control unit 202 is configured to detect a fingerprint, and based on the detected fingerprint the control unit 202 is configured to perform a fingerprint authentication procedure. Such fingerprint authentication procedures are considered per se known to the skilled person and will not be described further herein.

[0070] FIG. 3a schematically illustrates an optical biometric imaging device 100 according to an embodiment of the invention. The optical biometric imaging device 100 is here arranged under an at least partially transparent display panel 302. However, the biometric imaging device 100 may be arranged under any cover structure which is sufficiently transparent, as long as the image sensor receives a sufficient amount of light to capture an image of a biometric object in contact with the outer surface of the cover structure, such as a fingerprint or a palmprint. In the following, a biometric imaging device 100 configured to capture an image of a finger 303 in contact with an outer surface 305 of the display panel 102 is described.

[0071] The biometric imaging device 100 comprises an image sensor 308 comprising a photodetector pixel array 309, where each pixel 310 is an individually controllable photodetector configured to detect an amount of incoming light and to generate an electric signal indicative of the light received by the detector. The image sensor 308 may be any suitable type of image sensor, such as a CMOS or CCD sensor connected to associated control circuitry. The operation and control of such an image sensor can be assumed to be known and will not be discussed herein.

[0072] The biometric imaging device 100 further optionally comprises a transparent substrate 312 arranged to cover the image sensor 308, and optionally an opaque layer 314 covering an upper surface of the transparent substrate 312. The optional opaque layer 314 further comprises a plurality of separate openings 316 (only one is shown) arranged at a distance from each other.

[0073] In this example embodiment the biometric imaging device comprises an array of microlenses 318 (only one is shown) each arranged in a respective opening 316 of the optional opaque layer 314 and in the same plane as the opaque layer 314. It is however conceivable that the opaque layer 314 and the microlenses 318 are arranged in separate layers. Moreover, the microlens 318 has the same size and shape as the opening 316 to prevent any stray light which has not passed through the microlens 318 from reaching the image sensor 308. It is also conceivable that the microlenses 318 are larger that the openings 316.

[0074] Each microlens 318 is configured to redirect light through the transparent substrate 312 and onto a subarray 320 of pixels in the photodetector pixel array 309. Thus, the microlens is configured to define a subarray 320, (320′ in FIG. 3b) of pixels which receives light from the corresponding the microlens. The subarrays 320, 320′ are here defined as arrays of pixels which receive light from only one microlens 318. It should further be noted that the microlenses 318 and display pixels are not drawn to scale. The microlens 318 is shown to receive light reflected by the finger 303 which has propagated through the display panel 302 before reaching the microlens 318 and the light received by the microlens 318 is focused onto the image sensor 308. Different sub-arrays 320, 320′ of pixels receiving light from the individual microlenses 318 are preferably non-overlapping with adjacent subarrays receiving light from neighboring microlenses.

[0075] The biometric imaging device 100 further comprises an intermediate layer 322 located between the opaque layer 314 and the display panel 302. The intermediate layer 322 may for example comprise an adhesive layer used to attach the display panel 302 to the imaging device 100, and a refractive index of the intermediate layer 322 is different from a refractive index of the microlens 318. The intermediate layer comprises an air gap between the transparent display panel and the microlenses 318.

[0076] Furthermore, the intermediate layer 322 may also comprise an antireflection coating, an optical filter (e.g. infrared filter) and/or a polarizing filter, which are not illustrated separately herein. It is in general preferable that the refractive index of the microlens 318 is as high as possible and that it is different from the refractive index of any adjacent material above or below the microlens 318.

[0077] In FIG. 3a the width D of a portion of the object 303 is indicated. The distance odi indicates the distance from the light defining elements, here illustrated as the microlens 318, to the object 303. The distance id is the distance between image sensor pixel array 309 and the microlens 318. The demagnification provided by the biometric imaging device 100 provides that the width D is projected onto a width ii on the pixel array 109, where ii is given by

[00001]$i_1=D*\frac{\mathrm{id}}{{\mathrm{od}}_1}.$

[0078] In order to be able to resolve biometric features it is desirable that the pixel size, including any binning, is smaller than ii given that D represents the smallest desirable biometric feature to be resolvable. In other words, the projected pixel size which may be given by the demagnification

[00002]$\frac{{\mathrm{od}}_1}{\mathrm{id}}$

multiplied by the present pixel size including any binning should be smaller than the smallest desirable biometric feature to be resolvable, preferably less than a desired fraction of the smallest desirable biometric feature to be resolved, e.g. 25%. The projected pixel size may be considered the pixel size in the object plane, i.e. at the object 303. Thus, given the demagnification, it is possible to project the present pixel size including any binning to the object plane. This projected pixel in the object plane should be smaller than the smallest desirable biometric feature to be resolvable. It is possible to determine the pixel size in the object plane from an acquired image.

[0079] FIG. 3b illustrates a configuration of the biometric imaging device 100 in which the distance odi shown in FIG. 3a has now changed to a larger distance ode by the addition of e.g. a protection glass 324. The finger 303 is here touching the surface 305′ of the protection glass 324. In this case, the demagnification of the biometric imaging device 100 provides that the width D is projected onto a width i.sub.2 on the pixel array 309, where i.sub.2 is given by

[00003]$i_2=D*\frac{\mathrm{id}}{{\mathrm{od}}_2}.$

[0080] This may lead to that the projected pixel size is too large. In other words, the demagnification

[00004]$\frac{{\mathrm{od}}_2}{\mathrm{id}}$

multiplied by the present pixel size including any binning is no longer smaller than the desired fraction of the smallest desirable biometric feature to be resolved. This results in a reduced image resolution and reduction in biometric performance. By changing the binning setting, i.e. reducing the number of pixels used in the binning, the present pixel size including the updated binning can be configured so that the projected pixel size is smaller than the desired fraction of the smallest desirable biometric feature to be resolvable.

[0081] FIGS. 4a-c conceptually illustrates binning by combining a set of pixels into a larger pixel. Each of FIGS. 4a-c illustrates a pixel array 309 with individual pixels 310.

[0082] FIG. 4a conceptually illustrates binning of 4×4 pixels 310. This means that the detected signals from the pixels in the 4×4 subarray are combined into a single signal, thus effectively producing a larger pixel comprising the 4×4 pixels. In other words, by binning of pixels 16 signals have been reduced to one, which means less data has to be transferred from the image sensor, but with the cost of reduced image resolution.

[0083] If it turns out that the projected pixel size, i.e. the projection of the binned pixels 402 is too large to be able to resolve the desired features, due to changed demagnification, the present disclosure provides to change the binning.

[0084] One example of another binning setting is shown in FIG. 4b where the binning setting is to use 3×3 pixels 404 instead of the 4×4 shown in FIG. 4a. In this way the projected pixel size is reduced compared to using the binning shown in FIG. 4a. If the reduction is not sufficient, a further binning setting may be applied to use 2×2 pixels 406 as shown in FIG. 4c.

[0085] The total number of pixels is larger in a practical implementation, but the number of shown pixels in the arrays 309 is reduced for clarity in the drawings.

[0086] FIGS. 5a-c conceptually illustrates another example of binning by combining a set of pixels into a larger pixel. Each of FIGS. 5a-c illustrates a pixel array 309 with individual pixels 310.

[0087] FIG. 5a conceptually illustrates binning of an asymmetric subarray 408 of 4×3 pixels 310. The detected signals from the pixels in the 4×3 subarray are combined into a single signal, thus effectively producing a larger pixel comprising the 4×3 pixels.

[0088] As in the case described with reference to FIGS. 4a-c, if the projected pixel size is too large to be able to resolve the desired features, the binning may be altered.

[0089] FIG. 5b illustrates one example of altered binning where fewer pixels are used, in this case 4×2 pixels are binned in the subarray 410. If the reduction in the number of pixels used for binning is not sufficient, a further binning setting may be applied to use 3×2 pixels in the binning subarray 412 as shown in FIG. 5c.

[0090] FIG. 6a-e conceptually illustrate the effect of binning. FIG. 6a illustrates the actual light distribution impinging on the image sensor. It is this light distribution that is sampled with the photodetector pixel array using a given binning setting and with the sampling distance given by the pitch of the sensor pixels.

[0091] FIG. 6b illustrates the image as sampled when the demagnification is relatively low and with no binning. In other words, the sampling is performed with a 1×1 pixel binning setting. The features from the original light distribution, shown in FIG. 6a, are clearly resolved. Further, FIG. 6c illustrates the image as sampled when the demagnification is relatively low, but with a 2×2 pixel binning setting. The image resolution in FIG. 6c is reduced compared to FIG. 6b, but the features can still be resolved. Thus, using a 2×2 binning with low demagnification (e.g. as shown in FIG. 3a) would reduce the amount of data transferred from the image sensor but still provide sufficient resolution.

[0092] FIG. 6d illustrates the image as sampled when the demagnification is relatively high and with a 2×2 pixel binning setting. With this 2×2 pixel binning setting at the relatively high demagnification, the features from the original light distribution shown in FIG. 6a are barely resolved. In other words, the projected pixel size is too large when using a 2×2 pixel binning setting at this relatively high demagnification.

[0093] FIG. 6e illustrates the image as sampled when the demagnification is relatively high as in FIG. 6d, but the binning has been reduced by one step to a 1×1 pixel binning setting. With the 1×1 pixel binning setting at the relatively high demagnification, the features from the original light distribution shown in FIG. 6a are again resolvable.

[0094] FIG. 7 is a flow-chart of method steps according to embodiments of the invention. In step 102, acquire a biometric image of a biometric object using a first binning setting for binning of detection signals from subarrays of the photodetector pixels into a single pixel value. Subsequently S104, determining a measure indicative of the feature resolution of the biometric image. When the measure indicates that the feature resolution is below a feature resolution threshold, determine S106 a second binning setting adapted to provide a higher feature resolution compared to the feature resolution of the biometric image.

[0095] FIG. 8 is a flow-chart of method steps according to embodiments of the invention. From a step S106 has a binning setting been determined. Next, acquiring S204 a further image, e.g. a second image, with the determined binning setting for which a further measure indicative of the feature resolution is determined. If the measure indicates that the feature resolution exceeds the feature resolution threshold, selecting S206 the binning setting for which the measure indicates a feature resolution below the feature resolution threshold. However, if the measure indicates that the feature resolution is below the feature resolution threshold, a further binning setting is determined, in step S106, adapted to provide a higher feature resolution compared to the feature resolution of the previously acquired biometric image.

[0096] In other words, FIG. 8 illustrates to acquiring S204 further images with further binning settings for which further measures are determined, until the feature resolution is determined to exceed the feature resolution threshold. When the feature resolution corresponding to a certain binning setting is determined to exceed the feature resolution threshold, selecting S206 that binning setting for which the respective measure indicates a feature resolution that exceeds the feature resolution threshold. The selected binning setting may then be used for acquiring further biometric images used for biometric authentication.

[0097] As is understood the binning may also be used to reduce the feature resolution, i.e. changing the binning from what is shown in FIG. 5c (FIG. 4c) to what is shown in FIG. 5b (FIG. 4b), and possibly to what is shown in FIG. 5a (FIG. 4a). This may be advantageous if the projected pixel size is unnecessarily small and the feature resolution is limited by other factors in the biometric imaging device, of if the demagnification factor has decreased, for example due to the distance between the display and the microlens has decreased.

[0098] FIG. 9 illustrates a further flow-chart incorporating method steps according to the above. If the feature resolution is determined to be below a first resolution threshold, the step according to the above step S106 is followed. However, if the measure indicates that the feature resolution exceeds a second feature resolution threshold, a third binning setting adapted to provide a lower feature resolution compared to the feature resolution of the biometric image is determined in step S108. If the measure indicates that the feature resolution does not exceed the second feature resolution threshold, the method may return to step S102.

[0099] The flow chart in FIG. 10 also illustrates the method steps in accordance with the aspect of reducing the feature resolution. A biometric image of a biometric object is acquired in step S302 using a first binning setting for binning of detection signals from subarrays of the photodetector pixels into a single pixel value. Next, determining S304 a measure indicative of the feature resolution of the biometric image. When the measure indicates that the feature resolution exceeds a second feature resolution threshold determining S306 a third binning setting adapted to provide a lower feature resolution compared to the feature resolution of the biometric image.

[0100] In some possible implementations the light defining elements may be comprised of an array of coded aperture elements formed in a transmission mask, an array of pin holes, etc. Thus, the specific configuration of the light defining elements is such that each of them defines a subarray of pixels which receive light from a light defining element and the specific implementation is not limiting to the functionality of the presently disclosed technology including to adaptively change the binning setting.

[0101] A control unit may include a microprocessor, microcontroller, programmable digital signal processor or another programmable device. The control unit may also, or instead, include an application specific integrated circuit, a programmable gate array or programmable array logic, a programmable logic device, or a digital signal processor. Where the control unit includes a programmable device such as the microprocessor, microcontroller or programmable digital signal processor mentioned above, the processor may further include computer executable code that controls operation of the programmable device. It should be understood that all or some parts of the functionality provided by means of the control unit (or generally discussed as “processing circuitry”) may be at least partly integrated with the biometric imaging device.

[0102] Even though the invention has been described with reference to specific exemplifying embodiments thereof, many different alterations, modifications and the like will become apparent for those skilled in the art. Also, it should be noted that parts of the imaging device and method for manufacturing the imaging device may be omitted, interchanged or arranged in various ways, the imaging device yet being able to perform the functionality of the present invention. The microlenses are herein illustrated as plano-convex lenses having the flat surface orientated towards the transparent substrate. It is also possible to use other lens configurations and shapes. A plano-convex lens may for example be arranged with the flat surface towards the display panel, and in one embodiment the lens may be attached to a bottom surface of the display panel even though the imaging performance may be degraded compared to the reverse orientation of the microlens. It is also possible to use other types of lenses such as convex lenses, bi-convex lenses, convex-concave, etc., or multiple lens elements. An advantage of using a plano-convex lens is the ease of manufacturing and assembly provided by a lens having a flat surface.

[0103] Additionally, variations to the disclosed embodiments can be understood and effected by the skilled person in practicing the claimed invention, from a study of the drawings, the disclosure, and the appended claims. In the claims, the word “comprising” does not exclude other elements or steps, and the indefinite article “a” or “an” does not exclude a plurality. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.