System for monitoring a person

Abstract

A system for monitoring a person, for example a traveler, who is moving through a facility or a building and carries a biometric travel document and various electronic devices, for example mobile telephones, laptops, tablet computers, e-readers, notebooks, wireless headphones, smartwatches, or fitness trackers. An electronic controller communicates with an input device for inputting contents of the biometric travel document and for forwarding the contents for further processing. An electronically operated database generates a feature space in which the device characteristics are entered by the transceivers as a digital imprint of the traveler and which identifies the traveler with a particular probability resulting from the ratio of a weighted sum of the individual entries in the feature space to the maximum possible sum.

Claims

1. A system for monitoring a person, for example a traveler, who is moving through a facility or a building and carries with him a biometric travel document and also various electronic devices, for example mobile telephones, laptops, tablet computers, e-readers, notebooks, wireless headphones, smartwatches, fitness trackers, wherein the system has an electronic controller which is intended and adapted to communicate with: an input device for inputting information of the biometric travel document and forwarding the information for further processing; access barriers to predetermined areas of the facility or building in order to release or block those predetermined areas for the traveler in dependence on checks of the traveler; transceivers for detecting the electronic devices carried by the traveler and at least partially wirelessly analyzing them in order to acquire from each of the devices device characteristics which are characteristic of the particular traveler to a specific degree at least while he is in the building; and an electronically operated database which is adapted and programmed to generate for each traveler a feature space in which, in addition to the information of the biometric travel document, the device characteristics are entered by the transceivers as a digital imprint of the traveler, and which identifies the traveler with a specific probability given by a weighted sum of individual entries of the feature space, and, when in the further process the electronic devices carried by the traveler are newly detected by the transceivers, to compare newly recorded data recorded in the database with the previous entries in the feature space and, if the newly recorded data have a better probability with which the traveler is to be identified, to replace the corresponding previous data in the feature space by the newly recorded data, wherein preprocessing of the data of the traveler is carried out in order to segment and prepare the data for further analyses, and in order to evaluate the individual features of the feature space by calculating the quality of the individual features, and wherein the calculation of the quality of the individual features is configured to be used in the weighted sum of individual entries of the feature space which gives the specific probability of the identification of the traveler.

2. The system for monitoring a person as claimed in claim 1, which aggregates (part) identifications of the traveler from his non-biometric or biometric features with the digital imprint of the traveler to form an overall identification, in that the information of the biometric travel document is also entered in the feature space by the input device.

3. The system for monitoring a person as claimed in claim 2, in which specific transceivers for particular types of devices record the data characteristics of the devices carried by the traveler, specific sensors for biometric or non-biometric features record the biometric or non-biometric features of the traveler, and/or the input device records the biometric travel document and/or a flight ticket.

4. The system for monitoring a person as claimed in claim 1 wherein the transceivers are arranged at different locations within the facility or the building close to one of the predetermined areas and record at least parts of the digital imprint of the traveler, signal those parts to the database in order to initiate a comparison of those parts with the feature space of the traveler, and in order, in dependence on the result of the comparison, to effect the release or blocking of the access barriers to the predetermined area of the facility or of the building for that traveler.

5. The system for monitoring a person as claimed in claim 2 wherein an initialization of the feature space with the biometric data and non-biometric data of the traveler takes place in a monitored environment.

6. The system for monitoring a person as claimed in claim 1 wherein after aggregation of the evaluations and calculation of an overall quality of the individual features, a signal is given to a monitoring authority in relation to the individual traveler in order to initiate further actions.

7. A method for monitoring a person, for example a traveler, who is moving through a facility or a building and carries with him a biometric travel document and also various electronic devices, for example mobile telephones, laptops, tablet computers, e-readers, notebooks, wireless headphones, smartwatches, fitness trackers, in which an electronic controller communicates with: an input device in which information of the biometric travel document is inputted and forwarded for further processing; access barriers to predetermined areas of the facility or building in order to release or block those predetermined areas for the traveler in dependence on checks of the traveler; transceivers for detecting the electronic devices carried by the traveler and at least partially wirelessly analyzing them in order to acquire from each of the devices device characteristics which are characteristic of the particular traveler to a specific degree at least while he is in the building; and an electronically operated database which is adapted and programmed to generate for each traveler a feature space in which, in addition to the information of the biometric travel document, the device characteristics are entered by the transceivers as a digital imprint of the traveler, and which identifies the traveler with a specific probability given by a weighted sum of individual entries of the feature space, and, when in the further process the electronic devices carried by the traveler are newly detected by the transceivers, to compare newly recorded data recorded in the database with the previous entries in the feature space and, if the newly recorded data have a better probability with which the traveler is to be identified, to replace the corresponding previous data in the feature space by the newly recorded data, wherein preprocessing of the data of the traveler is carried out in order to segment and prepare the data for further analyses, and in order to evaluate the individual features of the feature space by calculating the quality of the individual features, and wherein the calculation of the quality of the individual features is configured to be used in the weighted sum of individual entries of the feature space which gives the specific probability of the identification of the traveler.

8. The method for monitoring a person as claimed in claim 7, which aggregates (part) identifications of the traveler from his non-biometric or biometric features with the digital imprint of the traveler to form an overall identification, in that the information of the biometric travel document is also entered in the feature space.

9. The method for monitoring a person as claimed in claim 7, in which the transceivers are arranged at different locations within the facility or the building close to one of the predetermined areas and record at least parts of the digital imprint of the traveler, signal those parts to the database in order to initiate a comparison of those parts with the feature space of the traveler, and in order, in dependence on the result of the comparison, to effect the release or blocking of the access barriers to the predetermined area of the facility or of the building for that traveler.

10. The method for monitoring a person as claimed in claim 8 wherein an initialization of the feature space with the biometric data and non-biometric data takes place in a monitored environment, and/or in which, after aggregation of the evaluations and calculation of an overall quality of the individual features, a signal is given to a monitoring authority in relation to the individual traveler in order to initiate further actions.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

(1) Further features, properties, advantages and possible modifications will become clear to a person skilled in the art from the following description, in which reference is made to the accompanying drawings.

(2) FIG. 1 shows, in a schematic plan view, a building or a facility in the form of, by way of example, an airport in which the solution presented herein is implemented.

DETAILED DESCRIPTION OF THE EMBODIMENTS

(3) The solution presented herein will be explained in greater detail hereinbelow by means of an exemplary embodiment shown only schematically in the drawing. The arrangement shown in FIG. 1 is a schematic plan view of a building in which, by way of example, the solution presented herein is implemented.

(4) The starting point is that a traveler, who is able to move through the building along the routes shown by broken lines, carries with him a biometric travel document RD in the form of a biometric passport, which contains as information I the surname, forename/s, sex, date of birth, place of birth, nationality, height, photograph, issuing authority, date of issue, date of expiry, ID number and type of ID, data chip with facial image and fingerprints, signature, machine-readable zone, and also various electronic devices G1 . . . Gn, for example mobile telephone, laptop, tablet computer, e-reader, wireless headphones, smartwatch, fitness tracker, etc. The building is divided into predetermined areas which the traveler may not enter or leave at all or may enter or leave only after his identity and/or his authorization have been checked. These predetermined areas are delimited from one another by the double dot-dash lines. Access to the predetermined area of the building is released or blocked for the traveler by access gates ZS, in dependence on checks of the traveler.

(5) The access gate may be a baffle gate which travelers pass through one at a time without having to present their travel documents for checking.

(6) Instead, the system presented herein has an electronic controller ECU, which is adapted to communicate with various transceivers SE. Each of these transceivers SE serves to establish contact wirelessly with one or more of the electronic devices G1 . . . Gn carried by the traveler when the traveler is in the immediate vicinity of these transceivers SE. The transceivers SE can thus detect electronic devices G1 . . . Gn carried by the traveler and analyze them at least partially wirelessly in order to acquire device characteristics GK of the devices G1 . . . Gn. These device characteristics GK are characteristic of the respective traveler to a specific degree at least while he is in the building. These device characteristics GK can be, for example, in the case of Bluetooth devices their individual and unique 48-bit long MAC address or the browser signature in a tablet computer, e-reader or other personal digital assistant, or the serial number IMSI stored in the SIM card of the smartphone. The transceivers SE are each able and adapted to record one or more device characteristics GK. For this purpose they each have a local controller which is capable, for example, of managing communication with a Bluetooth device of the traveler, or of communicating with a PDA of the traveler in order to read an IMSI of a telephone SIM card contained therein, etc.

(7) At an entrance gate or a desk for checking in the traveler there is provided an input device T, in which on the one hand the flight ticket and on the other hand also the biometric travel document RD of the passenger are read and the contents of which are forwarded to the electronic controller ECU for further processing. Moreover, by means of the various transceivers SE, device characteristics GK of the electronic devices G1 . . . Gn of the traveler are recorded and, optionally, photographs or measurements (e.g. stride length, etc.) of the traveler are recorded by means of sensors and likewise forwarded to the electronic controller ECU. This electronic controller ECU processes the data obtained and enters them in an electronically operated database DB. In the database DB, a feature space MR is generated for each traveler, in which the device characteristics GK are entered by the transceivers SE as the digital imprint of the traveler. By means of this digital imprint, that is to say the totality of the currently available device characteristics GK for a particular traveler, there is calculated in the database DB for the traveler a specific probability p with which the traveler is to be identified. This probability p can be calculated, for example, from the ratio of a weighted sum of the individual entries of the feature space MR to the maximum possible sum. Other calculations of the weightings are possible/conceivable.

(8) When the traveler passes the various transceivers SE in the building, the device characteristics GK of his electronic devices G1 . . . Gn are recorded, photographs, stride length, etc. of the traveler are optionally generated and likewise signaled to the electronic controller ECU to the controller ECU of the database DB. In one variant this takes place with, and in another variant without, the associated (individual) or overall weightings of the recorded data. These data are then compared with the previous entries in the feature space of the traveler. If the more recent recorded data have a better probability p with which the traveler is to be identified, these more recent recorded data can also replace the corresponding previous data in the feature space of the traveler.

(9) Thus, as soon as the traveler enters the building, an initialization of the feature space generated for him with biometric data and non-biometric data takes place in a monitored environment.

(10) For the inputting and for the later comparison of the data, the data of the traveler are subjected to preprocessing in order to segment and prepare them for further analysis. In addition, the individual features of the feature space are evaluated by calculating the quality of the individual features.

(11) In one configuration of the system, (part) identifications of the traveler from his non-biometric and/or biometric features are aggregated with the digital imprint of the traveler in the feature space MR to form an overall identification. For this purpose, all or parts of the information I of the biometric travel document RD are also inputted by the input device T into the feature space MR.

(12) The sensors and transceivers SE are arranged at one location or different locations inside the building close to the predetermined areas and record at least non-biometric and parts of the digital imprint of the traveler when the traveler enters the vicinity of one of those predetermined areas. These parts of the digital imprint of the traveler are signaled to the database DB and correspondingly weighted in order to initiate a comparison of those parts with the corresponding entries in the feature space MR of the traveler. Depending on the result of the comparison, the access gates ZS to the predetermined area of the building are released or blocked for that traveler.

(13) As soon as the traveler reaches one of the access barriers ZS to a predetermined area of the building, for example the check-in area, device characteristics GK of the electronic devices G1 . . . Gn and also, optionally, photographs of the traveler are recorded by means of the various transceivers SE located there. Depending on the check of the entries in the feature space of the person with the device characteristics GK obtained by the various transceivers SE in the region of the access barrier ZS in question, that predetermined area is released or blocked for the traveler.

(14) The input device T comprises, in addition to a display unit, for example, a barcode scanner or a document reader with which, in addition to a 2D barcode of a boarding card or of a ticket printed at home, the chip of a personal identification document, that is to say, for example, of a passport, can be read in respect of the biometric image or the RFID data. This document reader can also read out and recognize the machine-readable data of the personal identification document.

(15) At the input device T, the traveler is required to prove his identity; the creation of the feature space MR of the traveler in the database DB is then initiated by the input device T. A fingerprint reader for one or more fingerprints can also be provided in order to record the fingerprints of the traveler. Finally, the travel document (passport or the like) is also read and entered with the data of the flight ticket in the feature space MR of the traveler in the database DB.

(16) The traveler is then given a signal to proceed on his way through the building, that is to say, for example, that his check-in process is complete and he should proceed to the departure gate.

(17) In the region of hand luggage control, it is then no longer necessary for the traveler to show his boarding card; device characteristics GK of the electronic devices G1 . . . Gn and optionally photographs of the traveler are recorded by means of the various transceivers SE located there and compared with the data of the feature space of the traveler. The photographs can thereby be taken without the traveler being required to look into a camera in order to prepare a facial image. Instead, required features are recorded and compared with the features in the feature space without the cooperation of the traveler. The previously generated digital imprint is compared with the digital imprint stored in the database DB (by a transceiver SE, via an input device T, etc.) in a pattern recognition. In the case of a positive outcome of the comparison, the traveler can pass through the area and hand luggage control. In the case of a negative outcome of the comparison, security personnel correspondingly informed by the database DB can clarify the situation with the traveler. This can all take place without the traveler having had to stop for identification.

(18) The weighting of the individual features and of the overall image is to have the effect that strong ID features are more heavily included in the overall result a three-valued logic is possible (Yes, it is certain “1”; No, it is not certain “0”; the feature could not be recorded (not known)) values other than “not known” are real numbers/R; 0<=x<=1 weaker ID features can be aggregated with certainty

(19) Specialist knowledge for identification is thus stored in a quantifiable manner and is thus usable. Other than in the case of neural networks, the decision paths are mathematically replicable. The solution presented here is based on the feature space, from which a feature vector is derived. As in a neural network or in fuzzy logic, this is used as the input value and parameterized via weightings in order to calculate a value for the probability of a person belonging to an identity.

(20) TABLE-US-00001 Feature Result Evaluation Interpretation Face 60% of face 1.0 Extremely Face can be concealed, if it recognized? recognized important is recognized, then it is the main ID factor Hair color 100% match 0.8 Important As a supplement to the face. matches? In case of doubt, these features are added. Hair style 95% match 0.8 Important As a supplement to the face. matches? In case of doubt, these features are added. Clothing 80% match 0.5 Less As a supplement to the face. matches? important In case of doubt, these features are added. Make of mobile 100% match 0.5 Less As a plausibility check - if telephone important the make does not match then it is probably not him. Mobile device 100% match 0.8 Important If the facial recognition is MAC address uncertain and the other features are not available, the MAC address can be used. Electronic Could not be 0.5 Less As a plausibility test - if the feature (NFC recorded important signature does not match signature) then it is probably not him. eMRTD 100% match 0.25 not Weak feature which is signature important shared by many system participants . . .

(21) The aggregation specification is a multi-step logic with a fallback option, similar to a procedure in the case of a fuzzy logic.

(22) Rule 1: Only known data are processed (three-valued logic: 1 . . . 0, not known).

(23) Rule 2: Weighted probabilities are processed. The weightings are determined empirically in order to give particular weight to specific features (e.g. the face). The value range of the weightings is usually 0<W.sub.i≤1 but could also be W.sub.i>1 in order to give weight to individual values disproportionately. Weights with W.sub.i=0 are theoretically possible but not meaningful, because the corresponding feature is thereby completely equalized.

(24) Rule 3: The result “digital imprint DA is the ratio of the sum of the weighted features W.sub.ix.sub.i to the sum of those weightings W.sub.i whose feature value is not “not known”:

(25) $\mathrm{DA}=\begin{array}{c}\underset{\_}{\mathrm{sum}\mathrm{of}\mathrm{the}\mathrm{weighted}\mathrm{features}{W}_i{x}_i}\\ \mathrm{sum}\mathrm{of}\mathrm{the}\mathrm{weightings}{W}_i\mathrm{whose}\mathrm{feature}\mathrm{value}\mathrm{is}\mathrm{not}“\mathrm{not}\mathrm{known}”\end{array}$$W_i=\mathrm{weighting}\mathrm{of}\mathrm{feature}i;$$x_i=\mathrm{value}\mathrm{of}\mathrm{feature}i.$

(26) Rule 4: A threshold value is to be specified, above which DA can be assumed to identify the traveler. The following guide values apply for the threshold value:

(27) 1.0: all measurable features x.sub.i were determined with 100% certainty. The traveler has been identified without doubt.

(28) 0.75: overall, the x.sub.i give a high probability of identification of the traveler, even if individual features in some circumstances are allocated lower values.

(29) 0.5: the x.sub.i are weak and could have arisen by guesswork/by chance.

(30) 0.25: overall, the x.sub.i give a high probability of non-identification.

(31) 0.0: the person is with 100% certainty not the traveler to be identified.

(32) According to the rules, the following configuration is obtained for the example:

(33) TABLE-US-00002 Feature x.sub.i W.sub.i Facial features 0.6 1 Hair color 1 0.8 Hair style 0.95 0.8 Clothing 0.8 0.5 E-feature 1 - make of telephone 1 0.5 E-feature 2 - MAC address 1 0.8 E-feature 3 - NFC signature 0.5 E-feature 4 - eMRTD signature 1 0.25 Relative sum DA 0.884

(34) Thus, despite poor results of the facial recognition, the person is identified with certainty. Even if the weighting for the face is raised significantly (e.g. to factor 5), a value of DA=0.753 would be calculated for DA as a result of the other features determined with a high degree of certainty (x.sub.i>0.75), which still represents a high probability of identification.

(35) The analysis and processing of the data can be as follows, that is to say, for example:

(36) TABLE-US-00003 Feature Markedness Probability Weighting Result Facial features Clearly visible 0.8 1 0.8 Hair color Mid-brown 1 0.5 0.5 Style Short Not known 0.5 — Clothing Red jacket, 0.9 0.75 0.675 jeans, scarf Electronic imprint “0x3A5B6 . . . 1 1 1 C8D” Overall result: 0.846

(37) As a result of this aggregation of features, an identification can still be made with a certain degree of probability if, in a subsequent recording of the traveler, the face is concealed, that is to say, for example:

(38) TABLE-US-00004 Feature Markedness Probability Weighting Result Facial features Scarcely visible 0.1 1 0.1 Hair color Mid-brown 1 0.5 0.5 Style Short 1 0.5 0.5 Clothing Red jacket, 0.9 0.75 0.675 jeans, scarf Electronic imprint “0x3A5B6 . . . 1 1 1 C8D” Overall result: 0.74

(39) The values for the weighting are here only by way of example. If 0.75 is defined as the threshold value, for example, security personnel as the supervisory body are informed, for example, and a manual verification or identification of the traveler may be carried out.

(40) The above-described variants of the system and its structural and operational aspects serve merely for better understanding of the structure, the mode of functioning and the properties; they do not limit the disclosure, for example, to the exemplary embodiments. The FIG. is schematic, whereby essential properties and effects are in some cases shown on a significantly enlarged scale in order to clarify the functions, active principles, technical configurations and features. Any mode of functioning, any principle, any technical configuration and any feature that is/are disclosed in the figures or in the text can be combined freely and arbitrarily with all the claims, any feature in the text and in the other figures, other modes of functioning, principles, technical configurations and features which are contained in this disclosure or follow therefrom, so that all conceivable combinations of the described system are to be included. Combinations between all the individual implementations in the text, that is to say in every section of the description, in the claims, and also combinations between different variants in the text, in the claims and in the figures, are also included. The claims also do not limit the disclosure and thus the possible combinations of all the indicated features with one another. All the disclosed features are explicitly also disclosed herein individually and in combination with all the other features.