METHOD AND SYSTEM FOR READING AND VALIDATING IDENTITY DOCUMENTS

Abstract

Method and system for reading and validating identity documents that involves acquiring an image of a first and/or a second side of an identity document using a camera of a portable device; recognizing whether MRZ characters exist or are readable in the acquired image; if said MRZ characters are readable or do exist reading them obtaining a pre-identified document, or otherwise, detecting a series of local points of interests and their positions on the image calculating descriptors or vectors; and identifying the type or model of said identity document, starting by correcting perspective distortions caused by a bad relative position of the identity document with respect to the camera for obtaining a corrected and substantially rectangular image of the first and/or second side of the document at a predetermined scale which is used to perform, automatically, said identification of the identity document type or model and to automatically read and identify text and non-text information included in said corrected and substantially rectangular image.

Claims

1. A method for reading and validating identity documents, comprising: a) acquiring an image of at least one of a first and a second side of an identity document, only for a visible light spectrum, using a camera of a portable device; b) receiving, by an electronic system connected with said camera of said portable device, said acquired image, said electronic system automatically recognizing whether at least characters of a machine-readable zone, or MRZ characters, of the identity document are readable or exist in said acquired image by detecting candidate lines to be lines corresponding to MRZ characters by: lowering resolution of the acquired image and using a crests detector on the acquired image, at said low resolution, said crests detector looking for ridges and valleys on the acquired image and looking for the candidate lines over said ridges and valleys using a line detection algorithm; and performing a morphological treatment including filtering the candidate lines by selecting a zone where a candidate line is found and by verifying whether said candidate line corresponds to MRZ characters considering a format and a relative position of the candidate lines; c) in dependence upon existence or reading conditions of the MRZ characters: c1) if said MRZ characters are readable or do exist in the acquired image, the MRZ characters being read by reading each candidate line by maximizing contrast, segmenting regions of the MRZ characters, looking for bounding boxes where the MRZ characters are and reading the MRZ characters one by one, normalizing the boxes, obtaining a pre-identified document; or c2) if said MRZ characters are not readable or simply do not exist in the acquired image, detecting in the acquired image, a series of local points of interest and their positions on the acquired image, and calculating for each detected point of interest one or more descriptors or vectors of local characteristics substantially invariant to changes in scale, orientation, light and affine transformations in local environments; d1) comparing the MRZ characters of step c1) with those MRZ characters of at least one candidate identity document type or model stored in a database, and determining a perspective distortion that the MRZ characters experience, or d2) comparing the calculated descriptors or vectors of step c2) with those of reference descriptors of at least one image of several candidate identity document types or models stored in a database, and performing a matching with one of said candidate documents by dense matching of said local characteristics and determining a perspective distortion that said descriptors of the acquired image experience; e) correcting said perspective distortion caused by a bad relative position of the identity document with respect to the camera, including distance and orientation, for obtaining, in said portable device, a corrected and substantially rectangular image of the at least one first and second side of the identity document at a predetermined scale so as to perform an identification of the identity document type or model and to read and identify text and non-text information included in said corrected and substantially rectangular image; and f) reading and validating the document.

2. The method according to claim 1, further comprising: carrying out, prior to said step e), correction of perspective distortion with respect to an image shown on a display of the portable device prior to performing said acquiring of step a) by attempting to adjust the relative position of the identity document with respect to the camera, including distance and orientation.

3. The method according to claim 2, further comprising carrying out said correction by performing the following steps: showing on a display of said portable device a plurality of visual guides associated with respective ID formats of identity documents, manually attempting to adjust on said display the image of the identity document to be acquired in relation to one of said visual guides by the user moving said portable device or the identity document; and carrying out said step a) once said image to be acquired is at least partially adjusted on said display with said visual guides.

4. The method according to claim 3, wherein said visual guides are respective rectangles, each of them having dimensions corresponding to a certain ID format, including formats ID-1, ID-2 and ID-3 according to regulation ICAO-9303, said adjustment comprising framing the image to be acquired from the first or second side of the identity document in one of said rectangles on said display.

5. The method according to claim 1, further comprising: carrying out said correction of the perspective distortion with respect to the image acquired in said step a), correcting the geometry of the image by an automatic adjustment of positions of its respective points on the image, which positions are derived from the relative positions of the identity document with respect to the camera, including distance and orientation, at a moment in which its image was acquired.

6. The method according to claim 5, wherein when said image acquired in said step a) is an image of a first side including said MRZ characters, the method comprises carrying out said correction of the perspective distortion after at least part of said step c1) by performing the following steps: analyzing at least part of the MRZ characters read in step c1), and determining the position thereof on the acquired image as a result of said analysis; comparing the determined positions of the MRZ characters with those of the MRZ characters of at least one candidate identity document type or model, and determining the perspective distortion that the MRZ characters experience; creating a perspective distortions correction function including correction parameters estimated from the determined perspective distortion of the MRZ characters; and applying said perspective distortions correction function to the acquired image to obtain as a result said corrected and substantially rectangular image of the first side of the identity document at a predetermined scale.

7. The method according to claim 5, when said image acquired in said step a) is an image of a first or a second side not including said MRZ characters, the method comprises carrying out said correction of the perspective distortion after said step a), by performing the following steps: detecting in said acquired image a series of local points of interest and their positions on the acquired image, and calculating for each detected point of interest one or more descriptors or vectors of local characteristics substantially invariant to changes in scale, orientation, light and affine transformations in local environments; comparing at least the positions of said descriptors on the acquired image with those of reference descriptors of at least one image of at least one candidate identity document type or model, and determining the perspective distortion that said descriptors of the acquired image experience; creating a perspective distortions correction function including correction parameters estimated from the determined perspective distortion of the descriptors; and applying said perspective distortions correction function to the acquired image to obtain as a result said corrected and substantially rectangular image of the first or the second side of the identity document at a predetermined scale enabling said identification of the identity document type or model.

8. The method according to claim 7, further comprising: comparing said descriptors with reference descriptors of dictionaries or of images of one or more candidate identity document types or models to find coincidences, not only positional ones, which allow subsequent validation from making a pre-identification of at least the identity document type or model.

9. The method according to claim 7, further comprising: after said identifying of the type or model of said identity document, applying, on said corrected and substantially rectangular image obtained, a series of filters based on patterns or masks associated with different zones of said corrected and substantially rectangular image and in local descriptors, to identify a series of at least one of global and local characteristics, or points of interest, which allow an improvement in the identification of the identity document.

10. The method according to claim 9, further comprising: Improving correction of said possible perspective distortions caused by a bad relative position of the identity document with respect to the camera, the improving correction arising from using said improvement in the identification of the identity document.

11. The method according to claim 7, further comprising: identifying non-text graphic information in said corrected and substantially rectangular acquired or generated image.

12. The method according to claim 7, wherein when said type or model identification has already been performed for said first side, the method comprises, with respect to said second side, correcting possible perspective distortions caused by a bad relative position of the identity document with respect to the camera, including distance and orientation, for the purpose of obtaining in said portable device a corrected and substantially rectangular image of said second side of the identity document at a predetermined scale which allows automatically performing said reading and identification of text and non-text information.

13. The method according to claim 7, further comprising applying a series of validation tests to the information read or identified, including authentication tests.

14. A system for reading and validating identity documents, comprising: an image acquisition unit for acquiring an image of at least one of a first and a second side of an identity document for a visible light spectrum; and an electronic system connected with said image acquisition unit for receiving said acquired image, and for automatically recognizing whether at least characters of a machine-readable zone, or MRZ characters of the identity document are readable or exist in said acquired image by detecting candidate lines to be lines corresponding to MRZ characters by lowering resolution of the acquired image and using a crests detector on the acquired image, at said low resolution, said crests detector looking for ridges and valleys on the acquired image and looking for the candidate lines over said ridges and valleys using a line detection algorithm, and by performing a morphological treatment including filtering the candidate lines by selecting a zone where a candidate line is found and by verifying whether said candidate line corresponds to MRZ characters considering a format and a relative position of the candidate lines; wherein said electronic system is intended for identifying the type or model of said identity document from information included in the received image, said system having: a portable device (1) including said image acquisition unit, which is a camera, and at least one display (2) connected with said electronic system for showing at least the images focused on by the camera and the acquired image; and said electronic system being arranged at least in part in said portable device (1), and uses an algorithm for the correction, or enabling the correction, of perspective distortions caused by a bad relative position of the identity document with respect to the camera, including distance and orientation, for the purpose of obtaining in said portable device (1) a corrected and substantially rectangular image of said first or second side of the identity document at a predetermined scale which is used by said electronic system to perform said identification of the identity document type or model and to read and identify at least one of a text and non-text information included in said corrected and substantially rectangular image, wherein if said MRZ characters are readable or do exist in the acquired image, the MRZ characters being read by reading each detected candidate line by maximizing contrast, segmenting regions of the MRZ characters, looking for bounding boxes where the MRZ characters are and reading the MRZ characters one by one, normalizing the boxes.

15. The system according to claim 14, wherein the electronic system is entirely arranged in the portable device (1).

16. The system according to claim 14, wherein the remote computing unit communicated with the portable device (1) via cable or wirelessly.

17. The system according to claim 14, wherein the remote computing unit is a secure authentication entity or server.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0094] The previous and other advantages and features will be better understood from the following detailed description of some embodiments in relation to the attached drawings, which must be interpreted in an illustrative and non-limiting manner, in which:

[0095] FIG. 1 is a plan view of a mobile device of the system proposed by the second aspect of the invention, in the display of which three visual guides are shown in the form of respective rectangles;

[0096] FIGS. 2a and 2b are respective sides of an identity document with different zones of interest indicated therein by means of rectangles formed by dotted lines; and

[0097] FIG. 3 is a flow chart showing an embodiment of the method proposed by the first aspect of the invention.

[0098] FIG. 4 is another flow chart showing the steps of the proposed method for reading and validating identity documents.

[0099] FIG. 5 is a flow chart detailing the steps performed by the electronic system for recognizing if MRZ characters of the identity document are readable or exist in the acquired image.

[0100] FIG. 6 is a flow chart detailing the steps followed by the crests detector used by the electronic system.

[0101] FIG. 7 is a flow chart showing the steps executed by the proposed method to read the MRZ characters.

DETAILED DESCRIPTION OF SOME EMBODIMENTS

[0102] FIG. 1 shows the portable device 1 of the system proposed by the second aspect of the invention, in the display 2 of which visual guides are shown in the form of respective rectangles G1, G2, G3, each of them with dimensions corresponding to a certain ID format, including formats ID-1, ID-2 and ID-3 according to regulation ICAO-9303 (ICAO: International Civil Aviation Organization).

[0103] By means of said rectangles G1, G2, G3 shown in said display 2, the user can perform the previous manual aid for correction of perspective distortions, framing the document seen on the display 2 when it is focused on with the camera (not shown) in one of the rectangles G1, G2, G3 arranged for such purpose, and taking the photograph in the moment it is best framed, thus assuring that the acquired image corresponds to a corrected and substantially rectangular image and at a predetermined scale, represented for example in pixels/cm, which the software responsible for processing it needs to know to identify the document type or model.

[0104] FIGS. 2a and 2b show both sides of an identity document, the side of FIG. 2b being the one previously referred to as first side including a machine-readable zone, or MRZ, indicated as Z1, in this case formed by three lines of MRZ characters, which have been represented by small rectangles in the same manner that the remaining text information included both on the first side depicted in FIG. 2b and on the second side shown FIG. 2a has been depicted.

[0105] It can be observed in said FIGS. 2a and 2b that there are different text and non-text zones of interest to be read and validated, some of which have been indicated with references Z1, Z2 and Z3, for example, in relation to FIG. 2a, zone Z2 corresponding to a zone including VIZ characters, included on one side of the document not including MRZ characters, which are on the side shown in FIG. 2b.

[0106] FIG. 3 shows a flow chart relating to an embodiment of the method proposed by the first aspect of the invention.

[0107] The steps indicated in the different boxes of the diagram, starting with the initial box I to the end box F, are described below.

[0108] A1: This box corresponds to the previously described step a) for the acquisition of an image as well as optionally for the detection of the conditions in which said acquisition has occurred, said detection for example carried out by means of an accelerometer installed in the portable device the output signals of which allow improving the correction of perspective distortions, or for example carried out by means of a GPS locator for determining the coordinates of the mobile device for possible subsequent uses.

[0109] A2: In this step the MRZ characters in the acquired image are detected and read.

[0110] A3: The question indicated by this conditional or decision symbol box poses two possible options: the MRZ characters have been detected and read or they have not.

[0111] A4: Passing through this box is mainly due to the fact that the side of the document the image of which has been acquired in Al does not contain MRZ characters, either because it is a document type that does not contain them anywhere, or because it contains them on the other side. The actions to be performed consist of the previously described detection of local points of interest and corresponding calculation of local descriptors. In this case, a series of comparisons are made, by means of using filters suitable for such purpose, with reference descriptors of dictionaries or of images of one or more candidate identity document models, to find coincidences, not only positional ones, which allow performing a pre-identification of at least the identity document model, to be subsequently validated.

[0112] A5: If the MRZ characters have been read, the correction of perspective distortions is performed in this step according to the first variant of an embodiment described in a previous section, i.e., from the position of the MRZ characters on the image.

[0113] A6: In this step, the identification of the document from the detection and identification of other parts of the acquired image, as previously described, is refined.

[0114] A7: This step consists of performing the previously described correction of perspective distortions based on using as a reference the positions of the local descriptors on the image, improving the correction performed in A5 or, if coming from box A4, enabling the identification of the identity type or model, which validates the pre-identification made in A4.

[0115] A8: The VIZ characters are read in this step at least once the document model has already been identified.

[0116] A9: This box consists of performing the validation of the document by means of applying a series of validation tests (checking the control digits of the MRZ, the consistency of dates, the image patterns, etc.) to the read or identified information, including authentication tests.

[0117] A10: The user is shown the results of the reading and of the validation, for example through the display 2 of the portable device 1, in this step.

[0118] A11: After the mentioned presentation of results, said results are processed, said processing, represented by the present box, consisting of, for example, storing the results in the portable device 1 or in a server, or in automatically sending them to an official authority.

[0119] With reference now to FIG. 4, therein it is illustrated another embodiment of the proposed method. The method, to read and validate an identity document, first, step 401, comprises acquiring an image of the identity document using the camera of the portable device, only for a visible light spectrum. Following, step 402, the electronic system receives the acquired image and further performs, at step 402, a procedure for recognizing whether MRZ characters of the identity document are readable or exist in the acquired image. Two situations can arise here, it can happen that the MRZ are readable or do exist or otherwise that the MRZ are not readable or do not exist. According to the first situation, the MRZ characters at step 404 are read, obtaining a pre-identified document, and later compared, step 406, with MRZ characters of a candidate identity document or model stored in a database, determining a perspective distortion that the MRZ experience. According to the second situation, step 405, a series of local points of interests and their positions on the image are detected and one or more descriptors or vectors are calculated. Then, at step 407, the one or more calculated descriptors or vectors are compared with descriptors of candidate identity documents or models stored in a database, determining a perspective distortion that the descriptors experience. Finally, regardless of the situation occurred, perspective distortion caused by a bad relative position of the identity document with respect to the camera, including distance and orientation, are corrected, step 408, and the document is read and validated.

[0120] FIG. 5 illustrates in more detail previous step 403, i.e. how the electronic system recognizes if MRZ characters exist or are readable in the acquired image. According to this embodiment, first the resolution of the image is lowered (step 501) and then a crests detector is used on the acquired image (see FIG. 6). Finally, at step 503, a morphological treatment including the filtering of the detected candidate lines is performed. This is preferably done by verifying whether said candidate line corresponds to MRZ characters considering the format and the relative position of the candidate lines.

[0121] FIG. 6 illustrates an embodiment of the crests detector. According to this embodiment, the crest detector looks for ridges and valleys on the acquired image (step 601) and then looks for the candidate lines over said ridges and valleys using a line detection algorithm (step 602). As stated before, the crests detector taught by López et. al. can be used. Other any crests detector known in the field can be likewise used. Likewise, any of the line detection algorithms previously enumerated can be used.

[0122] With reference to FIG. 7, therein it is detailed the previous step 404, i.e. how the MRZ characters are read. According to this embodiment, each candidate line is read by the electronic system implementing an algorithm (any of the above mentioned algorithms can be used), step 701, that maximizes contrast (step 702), segments the regions of the MRZ characters (step 703), looks for the bounding boxes where the MRZ are (step 704) and finally reads the MRZ one by one, normalizing boxes (step 705).

[0123] A person skilled in the art could introduce changes and modifications in the described embodiments without departing from the scope of the invention as it is defined in the following claims.