METHOD AND SYSTEM FOR CONTROLLING AN ELECTRONIC DEVICE

Abstract

The present invention generally relates to a method for transitioning a device controller comprised with an electronic device from an at least partly inactive mode to an at least partly active mode, the electronic device further comprising a pre-processing module and a fingerprint sensor configured to acquire image data. The invention also relates to a corresponding electronic device and to a computer program product.

Claims

1. A method for transitioning a device controller comprised with an electronic device from an at least partly inactive mode to an at least partly active mode, the electronic device further comprising a pre-processing module and a fingerprint sensor configured to acquire image data, said method comprising the steps of: determining the presence of an object at a vicinity of the fingerprint sensor; acquiring, using the fingerprint sensor, image data representative of the object; pre-processing the acquired image data, using the pre-processing module, to determine features indicative of a fingerprint, wherein the device controller is in the at least partly inactive mode; matching, using the pre-processing module, the determined features with at least a set of stored fingerprint features of a finger of a user of the electronic device; generating an instruction to transition the device controller to the at least partly active mode if a result of the matching indicates that the acquired image data corresponds to the at least one finger of the user of the electronic device, providing the acquired image data to the device controller being transitioned to the at least partly active mode; and performing a fingerprint authentication procedure, using the device controller, based on the image data and at least a fingerprint template.

2. The method according to claim 1, wherein the step of matching comprises determining a matching score between the image data and the at least a set of stored fingerprint features, and determining that the image data corresponds to stored fingerprint features of the at least one finger of the user of the electronic device if the matching score exceeds a threshold.

3. The method according to claim 1, wherein a false accept rate of the matching, using the pre-processing module, is substantially higher than a false accept rate of the fingerprint authentication procedure.

4. The method according to claim 1, further comprising the steps of: providing, to the device controller, information relating to the matching performed at the pre-processing module, wherein the fingerprint authentication procedure is further based on the information relating to the matching at the pre-processing module.

5. The method according to claim 1, further comprising the steps of: unlocking the electronic device if the fingerprint authentication procedure results in a decision that the image data matches the at least one fingerprint template.

6. The method according to claim 1, wherein the at least one set of stored fingerprint features of one finger of the user comprises predetermined fingerprint ridge flow characteristics.

7. The method according to claim 6, wherein the fingerprint ridge flow characteristics comprises a set of global ridge flow patterns.

8. The method according to claim 7, wherein the set of global ridge flow patterns comprises at least one of information relating to an arch, a tented arch, a right loop, a left loop, and a whorl.

9. The method according to claim 6, wherein the fingerprint ridge flow characteristics comprises a set of local ridge flow descriptors.

10. The method according to claim 9, wherein the local ridge flow descriptors comprises at least one of local ridge orientation, or ridge curvature, or ridge density.

11. The method according to claim 6, further comprising the step of: updating the predetermined fingerprint ridge flow characteristics based on the acquired image data.

12. The method according to claim 1, wherein the pre-processing module is comprised with control circuitry provided with the fingerprint sensor.

13. The method according to claim 1, wherein the pre-processing module is a component of the device controller.

14. The method according to claim 1, wherein the at least partly inactive mode is a low power mode and the at least partly active mode is a normal operational mode for the device controller.

15. An electronic device, comprising: a device controller, the device controller configured to be arranged in an at least partly inactive mode or an at least partly active mode; a pre-processing module; and a fingerprint sensor configured to acquire image data, wherein the electronic device is arranged to: determine the presence of an object at a vicinity of the fingerprint sensor; acquire, using the fingerprint sensor, image data representative of the object; pre-process the acquired image data, using the pre-processing module, to determine features indicative of a fingerprint, wherein the device controller is in the at least partly inactive mode; match, using the pre-processing module, the determined features with at least a set of stored fingerprint features of a finger of a user of the electronic device; generate an instruction to transition the device controller from the at least partly inactive mode to the at least partly active mode if a result of the matching indicates that the acquired image data corresponds to the at least one finger of the user of the electronic device provide the acquired image data to the device controller being transitioned to the at least partly active mode; and perform a fingerprint authentication procedure, using the device controller, based on the image data and at least a fingerprint template.

16. The electronic device according to claim 15, wherein the pre-processing module is comprised with control circuitry provided with the fingerprint sensor.

17. The electronic device according to claim 15, wherein the pre-processing module is a component of the device controller.

18. The electronic device according to claim 15, wherein the fingerprint sensor is a capacitive fingerprint sensor.

19. The electronic device according to claim 15, wherein the electronic device is a mobile phone.

20. A computer program product comprising a non-transitory computer readable medium having stored thereon computer program means for controlling an electronic device, the electronic device comprising a device controller configured to be arranged in an at least partly inactive mode or an at least partly active mode, a pre-processing module, and a fingerprint sensor configured to acquire image data, wherein the computer program product comprises: code for determining the presence of an object at a vicinity of the fingerprint sensor; code for acquiring, using the fingerprint sensor, image data representative of the object; code for pre-processing the acquired image data, using the pre-processing module, to determine features indicative of a fingerprint, wherein the device controller is in the at least partly inactive mode; code for matching, using the pre-processing module, the determined features with at least a set of stored fingerprint features of a finger of a user of the electronic device; code for generating an instruction to transition the device controller to an at least partly active mode if a result of the matching indicates that the acquired image data corresponds to the at least one finger of the user of the electronic device code for providing the acquired image data to the device controller being transitioned to the at least partly active mode; and code for performing a fingerprint authentication procedure, using the device controller, based on the image data and at least a fingerprint template.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The various aspects of the invention, including its particular features and advantages, will be readily understood from the following detailed description and the accompanying drawings, in which:

[0027] FIG. 1 schematically exemplify an electronic device according to the present invention, in the form of a mobile phone comprising an integrated fingerprint sensor;

[0028] FIG. 2 schematically shows the fingerprint sensor array comprised in the electronic device in FIG. 1;

[0029] FIGS. 3A-3F schematically illustrates the method steps for transitioning a device controller of an electronic device from a partly in-active mode to an active mode, according to an embodiment of the invention;

[0030] FIGS. 4A-4F show exemplary fingerprint ridge flow characteristics in the form of global ridge flow patterns;

[0031] FIG. 5 conceptually illustrates an electronic device according to a currently preferred embodiment of the invention, and

[0032] FIG. 6 is a flow-chart of method steps according to an embodiment of the invention.

DETAILED DESCRIPTION

[0033] The present invention will now be described more fully hereinafter with reference to the accompanying drawings, in which currently preferred embodiments of the invention are shown. This invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided for thoroughness and completeness, and fully convey the scope of the invention to the skilled person. Like reference characters refer to like elements throughout.

[0034] Turning now to the drawings and to FIG. 1 in particular, there is schematically illustrated the electronic device according to the present invention, in the form of a mobile phone 100 with an integrated fingerprint sensor 102 and a display unit 104 with a touch screen interface 106. In this embodiment the fingerprint sensor 102 and the display unit 104 are together arranged at the front side of the mobile phone 100. The fingerprint sensor 102 may, for example, be used for unlocking the mobile phone 100 and/or for authorizing transactions carried out using the mobile phone 100, etc. The fingerprint sensor 102 may of course also be placed on the backside of the mobile phone 100.

[0035] Preferably and as is apparent for the skilled person, the mobile phone 100 shown in FIG. 1 further comprises a first antenna for WLAN/Wi-Fi communication, a second antenna for telecommunication communication, a microphone, a speaker, and a phone control unit. Further hardware elements are of course possibly comprised with the mobile phone. It should furthermore be noted that the invention may be applicable in relation to any other type of portable electronic device, such as a laptop, a remote control, a tablet computer, or any other type of present or future similarly configured device.

[0036] With reference to FIG. 2, there is conceptually illustrated a somewhat enlarged view of the fingerprint sensor 102. In the case of employing a capacitive sensing technology, the fingerprint sensor 102 is configured to comprise a large plurality of sensing elements, preferably arranged as a two-dimensional array. The two-dimensional array may have sizes depending on the planned implementation and in an embodiment 160×160 pixels are used. Other sizes are of course possible and within the scope of the invention, including two-dimensional array with less pixels as compared to the above example. A single sensing element (also denoted as a pixel) is in FIG. 2 indicated by reference numeral 202.

[0037] In FIGS. 3A-3F in conjunction with FIG. 6 there is conceptually shown method steps (S600-S612 in FIG. 6) according to an embodiment of the invention. In FIG. 3A there is schematically shown an electronic device 300, a device controller 302 comprised in the electronic device 300, the electronic device 300 further comprising a pre-processing module 304 and a fingerprint sensor 102. The device controller 300 may be the mobile phone 100 illustrated in FIG. 1. The fingerprint sensor 102 is configured to acquire image data of an object 301. In a first step (S600) illustrated in FIG. 3B, the presence of an object in the vicinity of the fingerprint sensor 102 is determined. This may be done by the fingerprint sensor 102, or by any other type of suitable sensor, e.g. ultrasonic, optical, etc. The fingerprint sensor 102 may determine the presence of the object 301 by determining that there is an input on one of the sensing elements 202 described with reference to FIG. 2. The device controller 302 is in a partly in-active mode, e.g. an idle mode or a “low power” mode.

[0038] Subsequently (S602), the fingerprint sensor 102 acquires image data representative of the object 301. Thus, at this stage, the fingerprint sensor 102 acquires image data of the object still being unidentified to the electronic device 300. The device controller is still in the partly in-active mode. Next, as schematically illustrated in FIG. 3C, the pre-processing module 304 receives the image data from the fingerprint sensor 102 so that the image data may be pre-processed (S604) in order to perform a pre-qualification of the image data. The pre-processing module 304 is configured to determine features in the image data, the features may be indicative of a fingerprint. The device controller 302 is still in the partly in-active mode. Note that the pre-processing module may be comprised with control circuitry (not shown) which may be provided with the fingerprint sensor 102, thus the pre-processing module 304 may be part of the fingerprint sensor 102. Alternatively, the pre-processing module 304 may be a component of the device controller 302, thus the pre-processing module 304 may be integrated with the device controller 302.

[0039] Subsequently (S606), as is schematically illustrated in FIG. 3D, the pre-processing module 304 performs a matching procedure in which the determined features in the image data are matched with stored fingerprint features 306 of a finger of a user of the electronic device 300. This matching step is not performed as a full fingerprint authentication procedure, but instead a pre-qualifying matching to determine if the features in the image data determined by the pre-processing module has a chance of matching the stored fingerprint features in a subsequent authentication step. Thus, the stored fingerprint features do not have to be full fingerprint templates, but at least part of a fingerprint template is accessible to the pre-processing module 304. For example, the pre-processing module may have access to data storage 310 of at least part of a fingerprint template whereas the device controller 302 may have access to a data storage 312 of full fingerprint templates. The data storage 312 is preferably a secure data storage 312. The storages 310 and 312 may also be combined in a single storage to which the pre-processing module 304 and the device controller 302 may access different parts of the combined storage.

[0040] Next, as illustrated in FIG. 3E, if the result of the matching is that the image data 305 indicative of a feature of the object corresponds to the stored fingerprint features 306, the pre-processing module 304 generates (S608) an instruction 308 to transition the device controller 302 to the at least partly active mode, e.g. a normal mode of operation for the device controller 302. Thus, the device controller 302 may be in the partly in-active mode during the pre-processing steps performed by the pre-processing module 304. However, if it is determined that the image data 305 indicative of a feature of the object does not correspond to the stored fingerprint features 306, there is no need to transition the device controller to the at least partly active mode, thus no such instruction is generated by the pre-processing module 304, and the device controller 302 remains in the partly in-active mode, thus there is no need to “wake up” the host (device controller 302) in this case.

[0041] In addition to providing the instruction 308 to the device controller 302, the pre-processing module may also provide (S610) to the device controller 302 information obtained from the matching of the features in the image data 305 with the stored fingerprint features 306. For example, if an instruction for transitioning the device controller 302 to the partly active mode is provided, information relating to matching features may also be provided to the device controller 302. The device controller 302 may continue to perform (S612) a full fingerprint authentication (schematically shown in FIG. 3F), and in such case the device controller 302 already has some matching information which may speed up the fingerprint authentication procedure. If the fingerprint authentication procedure results in a decision that the image data 305 matches the at least one fingerprint template 306, the device controller may decide to unlock the electronic device 300. However, if the image data 305 does not match the fingerprint template 306, the device controller does not unlock the electronic device 300. The locking function may for example be screen lock for the mobile phone 100.

[0042] The image data 305 may be provided to the device controller 302 from the pre-processing module 304 or from the fingerprint sensor 102. The matching performed by the pre-processing module 302 may be performed using the process, e.g. software, as the fingerprint authentication, or the matching performed by the pre-processing module 302 may be performed in a process parallel with the fingerprint authentication. In other words, the matching and the fingerprint authentication may be integrated with each other or performed separate. Furthermore, a false accept rate of the matching, using the pre-processing module, is preferably substantially higher than a false accept rate of the fingerprint authentication procedure. Thus, the matching using the pre-processing module is performed a different levels of security. For example, the features used in the matching with the pre-processing module are less complex and thereby requires less processing power than a full authentication. The full authentication may be performed subsequently with the device controller if it is transitioned to the partly active mode. As an example, the false accept rate (FAR) of the matching, using the pre-processing module may be e.g. in the range of 1/200 to 1/10, for example 1/100 or 1/50. A typical FAR in the authentication procedure is in the range of 1/200 000 to 1/10 000, for example 1/100 000 or 1/50 000.

[0043] The matching performed by the pre-processing module 304 may be performed based on fingerprint ridge flow characteristics. Exemplary fingerprint ridge flow characteristics in the form of global ridge flow patterns are conceptually illustrated in FIGS. 4A-4F. For example, the fingerprint ridge flow characteristics may comprises a set of global ridge flow patterns shown in FIGS. 4A-4F, such as relating to an arch (FIG. 4A), a tented arch (FIG. 4B), a right loop (FIG. 4C), a left loop (FIG. 4D), and a whorl (FIG. 4E). The global ridge flow patterns may for example be taken from fingerprint classifications such as the Henry scheme. Global singularities such as core and delta point can also be detected, for instance by computing the Poincare index of the local orientation field. Alternatively or additionally, the fingerprint ridge flow characteristics may preferably comprise a set of local ridge flow descriptors (e.g. based on characteristics of the ridges indicated in FIG. 4F where a single ridge is indicated by the numeral 402) of the fingerprint such as e.g. local ridge orientation, or ridge curvature, or ridge density. These may be applied in both gradient based methods as well as methods applied in the Fourier domain.

[0044] The matching step performed by the pre-processing module 304 may comprise determining a matching score between the image data 305 and the stored fingerprint features 306. If the matching score exceeds a threshold, it may be determined that the image data corresponds to stored fingerprint features of the at least one finger of the user of the electronic device 300. The matching score may be determined by comparing more or less coarse versions of the local ridge flow descriptors or for example more general features derived from the local ridge flow descriptors such as histograms, mean, and/or variance.

[0045] In addition, in a further exemplary embodiment of the invention, the described method may implement an adaptive mode in which the predetermined fingerprint ridge flow characteristics is updated based on the acquired image data. For example, features from image data 305 which was determined to not be a fingerprint may be used to update thresholds or models. Similarly, if the features are determined to originate from a fingerprint, the corresponding models (e.g. ridge flow characteristics) may be updated accordingly.

[0046] FIG. 5 conceptually illustrates an electronic device 300 according to an embodiment of the invention. The electronic device 300 may for example be a mobile phone as described with reference to FIG. 2. The electronic device 300 comprises a device controller 302, a pre-processing module 304, and a fingerprint sensor 102, and in addition a data storage 502 for storing fingerprint templates. The data storage 502 may alternatively be integrated with the device controller but is shown separate from the device controller 302 in FIG. 5. The fingerprint sensor may also include its own processing circuitry. As schematically illustrated in FIG. 5, the fingerprint sensor 102 may send data (e.g. image data) to the pre-processing module 304. The pre-processing module 304 has the capability to communicate back to the fingerprint sensor 102. Further, the pre-processing module 304 may retrieve stored fingerprint features from the data storage 502.

[0047] The pre-processing module 302 may also store data related the acquired image data on the data storage 502, for example in the case of updating the fingerprint ridge flow characteristics in an adaptive mode. The pre-processing module 304 may further communicate with the device controller 302 in order to wake up the device controller 302, i.e. transition the device controller 302 from the partly in-active mode to the partly active mode. The device controller 302 may also receive information from the matching step from the pre-processing module 302. Further, the device controller also has access to fingerprint templates stored on the data storage 502. The fingerprint sensor 102 may also communicate with the device controller 302. For example, the image data may be sent from the fingerprint sensor 102 to the device controller 302, although alternatively, it is also possible for the pre-processing module 304 to send the image data to the device controller 302. The device controller is in the partly in-active mode until it receives an instruction generated by the pre-processing module to transition to the partly active mode.

[0048] The device controller 302 and the pre-processing module 304 may each include a microprocessor, microcontroller, programmable digital signal processor or another programmable device. The device controller 302 and the pre-processing module 304 may also, or instead, each include an application specific integrated circuit, a programmable gate array or programmable array logic, a programmable logic device, or a digital signal processor. Where the device controller 302 or the pre-processing module 304 includes a programmable device such as the microprocessor, microcontroller or programmable digital signal processor mentioned above, the processor may further include computer executable code that controls operation of the programmable device. It should be understood that all or some parts of the functionality provided by means of the device controller 302 and the pre-processing module 304 (or generally discussed as “processing circuitry”) may be at least partly integrated with the fingerprint sensor 102.

[0049] The control functionality of the present disclosure may be implemented using existing computer processors, or by a special purpose computer processor for an appropriate system, incorporated for this or another purpose, or by a hardwire system. Embodiments within the scope of the present disclosure include program products comprising machine-readable medium for carrying or having machine-executable instructions or data structures stored thereon. Such machine-readable media can be any available media that can be accessed by a general purpose or special purpose computer or other machine with a processor. By way of example, such machine-readable media can comprise RAM, ROM, EPROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to carry or store desired program code in the form of machine-executable instructions or data structures and which can be accessed by a general purpose or special purpose computer or other machine with a processor. When information is transferred or provided over a network or another communications connection (either hardwired, wireless, or a combination of hardwired or wireless) to a machine, the machine properly views the connection as a machine-readable medium. Thus, any such connection is properly termed a machine-readable medium. Combinations of the above are also included within the scope of machine-readable media. Machine-executable instructions include, for example, instructions and data which cause a general purpose computer, special purpose computer, or special purpose processing machines to perform a certain function or group of functions.

[0050] Although the figures may show a sequence the order of the steps may differ from what is depicted. Also two or more steps may be performed concurrently or with partial concurrence. Such variation will depend on the software and hardware systems chosen and on designer choice. All such variations are within the scope of the disclosure. Likewise, software implementations could be accomplished with standard programming techniques with rule based logic and other logic to accomplish the various connection steps, processing steps, comparison steps and decision steps. Additionally, even though the invention has been described with reference to specific exemplifying embodiments thereof, many different alterations, modifications and the like will become apparent for those skilled in the art.

[0051] In addition, variations to the disclosed embodiments can be understood and effected by the skilled addressee in practicing the claimed invention, from a study of the drawings, the disclosure, and the appended claims. Furthermore, in the claims, the word “comprising” does not exclude other elements or steps, and the indefinite article “a” or “an” does not exclude a plurality.