Data fusion based safety surveillance system and method
10509968 ยท 2019-12-17
Assignee
Inventors
- Jiunn-Tsair Chen (Taoyuan, TW)
- Chin-Sheng Kuan (Taoyuan, TW)
- Wei-Jen Chien (Taoyuan, TW)
- Wen-Hao Hsiao (Taoyuan, TW)
Cpc classification
G06V10/811
PHYSICS
G08B13/18
PHYSICS
G06V40/70
PHYSICS
G06V20/52
PHYSICS
International classification
Abstract
A data fusion based safety surveillances system has a first through N.sup.th virtual monitoring systems and a data fusion and decision device. The data fusion and decision device defines an i.sup.th detection model of the i.sup.th virtual monitoring system, and estimates an i.sup.th detection loss probability of the i.sup.th virtual monitoring system according to detection loss sample numbers of batches of i.sup.th monitored data of the i.sup.th virtual monitoring system under conditions corresponding to the locations, batches of context data and existence of intruder, the i.sup.th detection model, the batches of i.sup.th monitored data corresponding to locations of the i.sup.th virtual monitoring system and the batches of context data. The data fusion and decision device determines a fusion parameter set according to the first through N.sup.th detection loss probabilities, and performs data fusion on a first through N.sup.th detection results to generate a decision result.
Claims
1. A data fusion based safety surveillances system comprises: a first through N.sup.th virtual monitoring systems formed by one or more physical monitoring system executing different algorithms, wherein N is greater than or equal to 2; and a data fusion and decision device linking to the first through N.sup.th virtual monitoring systems for defining a first through N.sup.th detection models of the first through N.sup.th virtual monitoring systems, wherein the i.sup.th detection model is used to represent a relationship between batches of i.sup.th monitored data of the i.sup.th virtual monitoring system and a plurality of i.sup.th detection results corresponding to the batches of the i.sup.th monitored data, wherein i is an integer from 1 to N; wherein the fusion and decision device estimates an i.sup.th detection loss probability of the i.sup.th virtual monitoring system according to (1) detection loss sample numbers of the batches of the ith monitored data of the ith virtual monitoring system under conditions, (2) the ith detection model, (3) the batches of the ith monitored data and (4) the batches of the context data, wherein the conditions are corresponding to locations, batches of context data and existence of intruder; the data fusion and decision device determines a fusion parameter set according to the first through N.sup.th detection loss probabilities; and the data fusion and decision device performs data fusion on a first through N.sup.th detection results of the first through N.sup.th virtual monitoring systems to generate a decision result.
2. The data fusion based safety surveillances system of claim 1, wherein the fusion and decision device estimates an i.sup.th false alarm probability of the i.sup.th virtual monitoring system according to false alarm sample numbers of the batches of the i.sup.th monitored data of the i.sup.th virtual monitoring system under conditions corresponding to the locations, the batches of the context data and inexistence of intruder, the i.sup.th detection model, the batches of i.sup.th monitored data corresponding to the locations of the i.sup.th virtual monitoring system and the batches of the context data, and the fusion and decision device determines the fusion parameter set according to the first through N.sup.th detection loss probabilities and the first through N.sup.th false alarm probabilities.
3. The data fusion based safety surveillances system of claim 1, wherein the i.sup.th detection model is defined by using a machine learning algorithm, an artificial intelligence algorithm or other one model solving algorithm according to acquired batches of known monitored data of the i.sup.th virtual monitoring system and known detected results corresponding to the acquired batches of the known monitored data.
4. The data fusion based safety surveillances system of claim 1, wherein the data fusion is performed by a logic operation function, a reliability rule or a context rule, and the logic operation function, the reliability rule or the context rule is determined by the fusion parameter set.
5. The data fusion based safety surveillances system of claim 1, wherein each of the batches of context data is a weather data defining variables of a rainfall, a wind speed, a temperature and a brightness.
6. A data fusion based safety surveillances method comprises: defining a first through N.sup.th detection models of the first through N.sup.th virtual monitoring systems, wherein the i.sup.th detection model is used to represent a relationship between batches of i.sup.th monitored data of the i.sup.th virtual monitoring system and a plurality of i.sup.th detection results corresponding to the batches of the i.sup.th monitored data, wherein i is an integer from 1 to N; estimating an i.sup.th detection loss probability of the i.sup.th virtual monitoring system according to (1) detection loss sample numbers of the batches of the ith monitored data of the ith virtual monitoring system under conditions, (2) the ith detection model, (3) the batches of the ith monitored data and (4) the batches of the context data, wherein the conditions are corresponding to locations, batches of context data and existence of intruder; determining a fusion parameter set according to the first through N.sup.th detection loss probabilities; and performing data fusion on a first through N.sup.th detection results of the first through N.sup.th virtual monitoring systems to generate a decision result.
7. The data fusion based safety surveillances method of claim 6 further comprises: estimating an i.sup.th false alarm probability of the i.sup.th virtual monitoring system according to false alarm sample numbers of the batches of the i.sup.th monitored data of the i.sup.th virtual monitoring system under conditions corresponding to the locations, the batches of the context data and inexistence of intruder, the i.sup.th detection model, the batches of i.sup.th monitored data corresponding to the locations of the i.sup.th virtual monitoring system and the batches of the context data, wherein the fusion parameter set is determined according to the first through N.sup.th detection loss probabilities and the first through N.sup.th false alarm probabilities.
8. The data fusion based safety surveillances method of claim 6, wherein the i.sup.th detection model is defined by using a machine learning algorithm, an artificial intelligence algorithm or other one model solving algorithm according to acquired batches of known monitored data of the i.sup.th virtual monitoring system and known detected results corresponding to the acquired batches of the known monitored data.
9. The data fusion based safety surveillances method of claim 6, wherein the data fusion is performed by a logic operation function, a reliability rule or a context rule, and the logic operation function, the reliability rule or the context rule is determined by the fusion parameter set.
10. The data fusion based safety surveillances method of claim 6, wherein each of the batches of the context data is a weather data defining variables of a rainfall, a wind speed, a temperature and a brightness.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
(1)
(2)
(3)
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
(4) For a fuller understanding of the objects, features and functions of the present disclosure, the present disclosure will be described in detail by way of the following specific embodiments in conjunction with the accompanying drawings.
(5) An embodiment of the present disclosure provides a data fusion based safety surveillances system and method, which eliminates the need for an additional physical monitoring system, fuses the plurality of detection results of multiple virtual monitoring systems formed by one or more existing physical monitoring systems executing different algorithms, and further generates a decision result.
(6) In an embodiment, a training process is firstly performed on each virtual monitoring system to define each detection model. The training process is illustrated as follows. For each virtual monitoring system, a safety surveillance system and method collect the batches of the known monitored data and known detection results of the virtual monitoring system corresponding to the batches of the known monitored data, and defines a detection model related to the batches of the monitored data and the detection results of the virtual monitoring system by using the machine learning algorithm, the artificial intelligence algorithm or other model solving algorithms. Next, for each virtual monitoring system, the safety surveillances system and method estimate a detection lose probability according to detection loss sample numbers of batches of monitored data under conditions of the intruder existence, context probabilities corresponding to locations, batches of i.sup.th monitored data corresponding to the locations and the detection model, and further estimate a false alarm probability according to false alarm sample numbers of batches of monitored data under conditions of intruder inexistence, the context probabilities corresponding to the locations, the batches of monitored data corresponding to the locations and the detection model.
(7) After the training process has been done, the inference process is performed to obtain the decision result. Specifically, the safety surveillances system and method set the fusion parameter set according to the estimated detection loss probability and the false alarm probability of multiple virtual monitoring systems, wherein the fusion parameter set is used to determine the fusion manner of the multiple detection results. Afterwards, the safety surveillances system and method can generate the decision result according to the fusion parameter set and the detection results of multiple virtual monitoring systems (that is, the data fusion of multiple detection results is performed according to the fusion parameter set to generate the decision result).
(8) It should be noted that the damage due to the detection loss may be much larger than the damage due to the false alarm. Therefore, in one embodiment of the present disclosure, the false alarm probability may not be estimated and the safety surveillances system and method may set the fusion parameter set according to the estimated detection loss probability of multiple virtual monitoring systems merely.
(9) Optionally, in one embodiment of the present disclosure, the data fusion manner of multiple detection results can be performed through a logical operation function, and the fusion parameter set can be used to determine the logical operation function of the detection results of multiple virtual monitoring systems. For example, the logic AND operation is performed on partial detection results, the logic OR operation is performed on other partial detection results, and then the logical OR or AND operation is performed on the multiple logic operation results to generate the decision result.
(10) Optionally, in one embodiment of the present disclosure, a data fusion manner of multiple detection results can be performed through a reliability rule, and the fusion parameter set can be used to determine the reliability rule. For example, the decision result is outputted only if the decision result is finally greater than the reliability threshold, or the detection result is used for data fusion only if the reliability of the detection result is greater than the reliability threshold.
(11) Optionally, in one embodiment of the present disclosure, the data fusion manner of multiple detection results can be performed through a context rule, and the fusion parameter set can be used to determine the context rule. For example, each detection result corresponds to context information, such as coordinates, type of object, number of objects and accuracy, and multiple detection results are fused based on the context information according to the context rule.
(12) First, referring to
(13) It should be noted that, in practice, each of the one or more physical monitoring systems has different algorithms, so to form the first through N.sup.th virtual monitoring systems 111-11N, and that is, each of the first through N.sup.th virtual monitoring systems 111-11N is generated by a physical monitoring system based on one of the algorithms. The foregoing physical monitoring system is, for example, a thermal imaging system, an electronic fence system, a laser ranging system or a virtual wall system, but the disclosure is not limited thereto. The variable N can be an integer. For example, there are two physical monitoring systems respectively having one algorithm and two algorithms to be executed, and thus N can be 3. However, the present disclosure is not limited thereto.
(14) The data fusion and decision device 12 includes a plurality of circuits which are configured to make the data fusion and decision device 12 have computability. For example, the data fusion and decision device 12 is a general-purpose computer that has a corresponding software program installed thereon to perform related operations, so as to execute steps required by the data fusion and decision device 12 and to achieve functions specified by the data fusion and decision device 12.
(15) In general, thermal imaging systems are susceptible to temperature effects. Electronic fence systems or laser ranging systems are susceptible to moisture interference due to the characteristics of the band of the detected signal. In addition, the virtual wall system is provided by the visible light camera as the source of the image for identification. Therefore, once the image quality is poor or the natural conditions affect (for example, there is no light at night, the wind speed is too high causing the leaves to shake, or the number of the image pixels (i.e. the image resolution) is not enough), the final detection result of the virtual wall system will be directly or indirectly affected. Therefore, it can be understood that context data is an important factor that affects the detection results of the above-mentioned virtual monitoring systems. From the above example, the context data can be, for example, the weather data including the variables of humidity, wind speed, temperature, brightness and so on. However, the present disclosure does not limit that the context data must be the weather data.
(16) Since the environment may have an impact on the detection result of the virtual monitoring system, context factors still must be taken into consideration when estimating the detecting loss probability and the false alarm probability. Take the context data being the weather data as an example, the humidity can be expressed as a binary value of rainy day or no rain or simply expressed as amounts of the rainfall; the wind speed can be expressed as a ternary value of windless, breeze or strong wind, or simply expressed as degrees of the wind force; the temperature can be expressed as a binary value of low temperature or high temperature, or simply expressed as a temperature value; and the brightness can be expressed as a binary value of daytime or night, or simply expressed as a brightness value.
(17) The data fusion and decision device 12 collects multiple batches of known first through N.sup.th monitored data of the first virtual monitoring system 111, the second virtual monitoring system 112, . . . , the Nth virtual monitoring system 11N and known first through N.sup.th detection results. The foregoing multiple batches of known first monitored data, the second monitored data, and the N.sup.th monitored data are respectively multiple batches of raw data of detection loss and false alarm of the first virtual monitoring system 111, the second virtual monitoring system 112, . . . , the N.sup.th virtual monitoring system 11N, and thus it can facilitate to reduce the subsequent calculation of the detection loss sample numbers and the false alarm sample numbers of the batches of the monitored data under different conditions with and without an intruder. However, the present disclosure is not limited thereto. The foregoing multiple batches of known first monitored data, second monitored data, and N.sup.th monitored data can also be the raw data out of the detection loss and false alarm raw data of the first virtual monitoring system 111, the second virtual monitoring system 112, . . . , the N.sup.th virtual monitoring system 11N, but in this case, the additional calculation of the detection loss sample numbers and the false alarm sample numbers of the batches of the monitored data under different conditions with and without an intruder is required.
(18) Then, the data fusion and decision device 12 can define the i.sup.th detection model with the relationship between the batches of the i.sup.th monitored data and the i.sup.th detection results according to the multiple batches of the known i.sup.th monitored data of the i.sup.th virtual monitoring system and the multiple batches of the known multiple i.sup.th detection results corresponding to the batches of the known i.sup.th monitored data by using a machine learning algorithm, an artificial intelligence algorithm or other model solving algorithms (that is, the present disclosure does not limit the behavior of the algorithms with solution finding model), wherein i is an integer being 1 through N. Afterwards, the data fusion and decision device 12 estimates an i.sup.th detection loss probability of the i.sup.th virtual monitoring system according to the detection loss sample numbers of the batches of the i.sup.th monitored data of the i.sup.th virtual monitoring system under conditions corresponding to the locations, the batches of context data and existence of intruder, batches of context data (used to obtain the context probabilities of multiple locations), the batches of i.sup.th monitored data corresponding to the locations and the i.sup.th detection model of the i.sup.th virtual monitoring system; and the fusion and decision 12 device further estimates an i.sup.th false alarm probability of the i.sup.th virtual monitoring system according to the false alarm sample numbers of the batches of the i.sup.th monitored data of the i.sup.th virtual monitoring system under conditions corresponding to the locations, the batches of context data and inexistence of intruder, the batches of context data (used to obtain the context probabilities of multiple locations), the batches of i.sup.th monitored data corresponding to the locations and the i.sup.th detection model of the i.sup.th virtual monitoring system.
(19) Then, the data fusion and decision device 12 sets a fusion parameter set according to the first through N.sup.th detection loss probabilities of the first through N.sup.th virtual monitoring systems 111-11N and the first through N.sup.th false alarm probability of the first through N.sup.th virtual monitoring systems 111-11N, wherein the fusion parameter set is used for determining a fusion manner of the first through N.sup.th detection results of the first through N.sup.th virtual monitoring systems 111-11N. Then, the data fusion and decision device 12 generates a decision result according to the fusion parameter set and the first through N.sup.th detection results of the first through N.sup.th virtual monitoring systems 111-11N to determine whether an intruder intrudes.
(20) For example, the data fusion manner of the first through N.sup.th detection results of the first through N.sup.th monitoring systems 111-11N can be performed through a logic operation function, and the fusion parameter set can be used to determine the logical operation function of the first through N.sup.th detection results of the first through N.sup.th monitoring systems 111-11N. For example, if N is 3, then the logical AND operation is performed on the first and second detection results, and the logical OR operation is performed on the acquired operation result and the third detection result to acquire the decision result.
(21) Next, the details of estimating the i.sup.th detection loss probability and the i.sup.th false alarm probability of the i.sup.th virtual monitoring system are further described as follows. The i.sup.th detection loss probability of the i.sup.th virtual monitoring system can be expressed as: M.sub.i,l=.sub.k(1R.sub.i,l(y.sup.i,k))p(y.sup.i,k,W.sub.m,L.sub.k|H.sub.1)dy.sup.i,kdW.sub.m (abbreviated as formula 1), wherein variables i, k and l respectively represent a virtual monitoring system index value, a location index value and a parameter set index value, a variable L.sub.k represents the location corresponding to the location index value k, a variable y.sup.i,k represents the i.sup.th monitored data acquired by the i.sup.th virtual monitoring system at the location L.sub.k, a function R.sub.i,l(y.sup.i,k) represents the i.sup.th detection result of the i.sup.th virtual monitoring system based on the i.sup.th monitored data y.sup.i,k when the 1.sup.th parameter set is adopted, the function R.sub.i,l(y.sup.i,k) can be 1 or 0 for indicating whether an intruder is detected, the variable H.sub.1 represents a condition that the intruder intrudes, a variable W.sub.m represents a vector of the context data (as mentioned above, for example, the context data can be the weather data which is defined to include four variables of humidity, wind speed, temperature and brightness), the parameter set index value 1 is determined by the location L.sub.k and context data W.sub.m, and a function p(y.sup.i,k,W.sub.m,L.sub.k|H.sub.1) represents a joint probability of that the i.sup.th virtual monitoring system is located at the location L.sub.k, the context data is W.sub.m and the i.sup.th monitored data is y.sup.i,k under the condition of existence of intruder (intruder existence).
(22) Because the detection results of the thermal imaging system and the virtual wall system may be affected due to the shaking of the leaves of the surrounding trees, the detection results of the electronic fence system may be affected easily due to the sensitivity to the groundwater or puddles, and the virtual monitoring systems generated from the above physical monitoring systems by executing different algorithms may also be very sensitive to the terrain of the slopes. Therefore, the function p(y.sup.i,k,W.sub.m,L.sub.k|H.sub.1) in the above formula 1 can be simplified and rewritten as p(y.sup.i,k,W.sub.m,L.sub.k|H.sub.1)=p(y.sup.i,k|H.sub.1,W.sub.m,L.sub.k)p(W.sub.m,L.sub.k) (abbreviated as formula 2), wherein a function p(y.sup.i,k|H.sub.1,W.sub.m,L.sub.k) represents a probability that the i.sup.th monitored data of the i.sup.th virtual monitoring is y.sup.i,k of condition of the location being at the location L.sub.k, the context data being W.sub.m and existence of intruder, and the function p (W.sub.m,L.sub.k) represents a probability that the location is at the location L.sub.k and the context data is W.sub.m (that is, the context probability at the location L.sub.k).
(23) Since the variable p(W.sub.m,L.sub.k) is independent to the condition of H.sub.1, the formula 2 can be rewritten into M.sub.i,l=.sub.kE.sub.y|W.sub.
(24) Then, using the statistical concept that the sample number approximates the expected value, the formula 3 can be rewritten as M.sub.i,l=.sub.kS.sub.y|W.sub.
(25) Since the i.sup.th detection result R.sub.i,l(y.sup.i,k) cannot be obtained from the i.sup.th monitored data of the i.sup.th virtual monitoring system, the machine learning algorithm, artificial intelligence algorithm or other model solving algorithm can be used to define the detection model of the i.sup.th monitored data y.sup.i,k and the i.sup.th measurement result R.sub.i,l(y.sup.i,k) according to the known i.sup.th monitored data y.sup.i,k and the known corresponding detection result R.sub.i,l(y.sup.i,k), and that is, the model of the function R.sub.i,l(y.sup.i,k) can be known. Then, through the formula 4 and based on the detection model of the i.sup.th virtual monitoring system, the i.sup.th detection loss probability of the i.sup.th virtual monitoring system can be estimated according to the context data W.sub.m (the context probability of location L.sub.k can be known through statistics), the location L.sub.k, the detection loss sample number of the batches of the i.sup.th monitored data y.sup.i,k of the i.sup.th virtual monitoring system under the condition of the location being at the location L.sub.k, the context data being W.sub.m and existence of intruder and the i.sup.th monitored data y.sup.i,k of each location L.sub.k.
(26) Similarly, the false alarm probability of the i.sup.th virtual monitoring system can be expressed as: F.sub.i,l=.sub.kR.sub.i,l(y.sup.i,k)p(y.sup.i,k,W.sub.m,L.sub.k|H.sub.0)dy.sup.i,kdW.sub.m (abbreviated as formula 5), wherein the function p(y.sup.i,k,W.sub.m,L.sub.k|H.sub.0) represents the joint probability that the i.sup.th virtual monitoring system is located at position L.sub.k, the context data is W.sub.m and the i.sup.th monitored data is y.sup.i,k of condition of inexistence of intruder (or intruder inexistence), and the variable H.sub.0 represents the condition that no intruder intrudes. Based on a similar derivation, the formula 5 can therefore be simplified to M.sub.i,l=.sub.kE.sub.y|W.sub.
(27) Then, through formula 6, based on the detection model of the i.sup.th virtual monitoring system, the false alarm probability of the i.sup.th virtual monitoring system can be estimated according to the context data W.sub.m (the context probability of the location L.sub.k can be known through statistics), the false alarm sample number of the batches of the i.sup.th monitored data of the i.sup.th virtual monitoring system under the condition of the location being at the location L.sub.k, the environment context data being W.sub.m and inexistence of intruder and the i.sup.th monitored data y.sup.i,k of each location. After acquiring of the first through N.sup.th detection loss probabilities and the first through N.sup.th false alarm probabilities of the first through N.sup.th virtual monitoring systems 111-11N, the fusion parameter set can be determined according to the estimated first through N.sup.th detection loss probabilities and the first through N.sup.th false alarm probabilities, and then the first through N.sup.th detection results of the first through N.sup.th virtual monitoring systems 111-11N can be fused based on the fusion parameter set to generate the decision result. In this way, it is possible to generate more accurate decision result by using the detection results of the virtual monitoring systems formed by at least one physical monitoring system executing different algorithms, so as to reduce the overall the detection loss probability and the overall false alarm probability of the data fusion based safety surveillance system 1.
(28) Then, referring to
(29) The data fusion and decision device 3 of
(30) First, at step S201, the detection model defining unit 31 acquires the batches of the known first through N.sup.th monitored data and the batches of the known first through N.sup.th detection results of the first through N.sup.th virtual monitoring systems, wherein the batches of the known first through N.sup.th detection results correspond to the batches of the known first through N.sup.th monitored data. Then, at step S202, the detection model defining unit 31 defines the detection model of the first through N.sup.th virtual monitoring systems through a model solving algorithm (for example, through a machine learning algorithm, an artificial intelligence algorithm or other model solving algorithm), wherein the detection model of the i.sup.th virtual monitoring system is defined based on the batches of the known i.sup.th monitored data and the batch of the known corresponding i.sup.th detection results.
(31) Then, at step S203, the probability estimating unit 32 estimates a i.sup.th detection loss probability of the i.sup.th virtual monitoring system according to batches of context data (context probability of each location can be known through statistics), the batches of i.sup.th monitored data corresponding to the locations, the detection loss sample numbers of the batches of i.sup.th monitored data of the i.sup.th virtual monitoring system under conditions corresponding to the locations, the batches of context data and existence of intruder and the detection model of the i.sup.th virtual monitoring system; and the probability estimating unit 32 further estimates a i.sup.th false alarm probability of the i.sup.th virtual monitoring system according to batches of context data (context probability of each location can be known through statistics), the batches of i.sup.th monitored data corresponding to the locations, the false alarm sample numbers of the batches of i.sup.th monitored data of the i.sup.th virtual monitoring system under corresponding to the locations, the batches of context data and inexistence of intruder and the detection model of the plurality of the i.sup.th virtual monitoring system.
(32) Then, at step S204, the fusion parameter setting unit 33 sets the fusion parameter set according to the estimated i.sup.th through N.sup.th detection loss probabilities and the estimated i.sup.th through N.sup.th false alarm probabilities of the i.sup.th through N.sup.th virtual monitoring systems. Then, at step S205, the decision unit 34 performs data fusion of the i.sup.th through N.sup.th detection results of the i.sup.th through N.sup.th virtual monitoring systems according to the fusion parameter set to generate the decision result.
(33) Accordingly, without adding any physical monitoring system, the data fusion based safety surveillances system and method provided by the embodiments of the present disclosure uses the detection results of the virtual monitoring systems formed by at least one existing physical monitoring system executing different algorithms to perform the data fusion of the detection results, so as to generate a decision result, and to reduce the overall detection loss and false alarm probabilities. In other words, the data fusion based safety surveillances system and method provide a low-cost technical solution to reduce the detection loss and false alarm probabilities of the safety surveillances system.
(34) The present disclosure has been disclosed in the above paragraphs with reference to the preferred embodiments. However, it should be understood by those skilled in the art that the foregoing embodiments are merely used to describe the present disclosure, and should not be construed as limiting the scope of the present. It should be noted that variations and substitutions equivalent to those of the foregoing embodiments should be construed to be included within the scope of the present disclosure. Therefore, the protection scope of the present disclosure should be based on the scope of the claims.