SMART SURVEILLANCE SYSTEM AND SMART SURVEILLANCE CONTROL METHOD USING UAV

Abstract

Provided is a smart surveillance system that includes one unmanned aerial vehicle (UAV), a plurality of Internet of Things (IoT) terminals distributed in a surveillance area, and a plurality of base stations distributed in the surveillance area, wherein the UAV selects any IoT terminal from among the plurality of IoT terminals using deep learning auction training, receives surveillance data from the selected IoT terminal, and transmits the surveillance data to a data center through the Internet and any one base station among the plurality of base stations.

Claims

1. A smart surveillance system comprising: one unmanned aerial vehicle (UAV); a plurality of Internet of Things (IoT) terminals distributed in a surveillance area; and a plurality of base stations distributed in the surveillance area, wherein the UAV selects any IoT terminal from among the plurality of IoT terminals using deep learning auction training, receives surveillance data from the selected IoT terminal, and transmits the surveillance data to a data center through the Internet and any one base station among the plurality of base stations.

2. The smart surveillance system of claim 1, wherein the UAV includes: a controller provided with a deep learning auction training network; a transceiver connected to the controller; and a storage unit connected to the controller, wherein the controller selects any IoT terminal from among the plurality of IoT terminals using the deep learning auction training network.

3. The smart surveillance system of claim 2, wherein the deep learning auction training network includes a plurality of monotonic networks, one allocation network connected to the plurality of monotonic networks, and one payment network connected to the plurality of monotonic networks.

4. A smart surveillance control method comprising: an operation (A) of starting, by a controller of an unmanned aerial vehicle (UAV), smart surveillance control while wirelessly transmitting a surveillance data request signal to a plurality of Internet of Things (IoT) terminals distributed in a surveillance area; an operation (B) of receiving, by the controller, a quotation weight (b.sub.i) from each of the plurality of IoT terminals; an operation (C) of inputting, by the controller, the quotation weight (b.sub.i) to a deep learning auction training network provided therein and performing deep learning auction training; an operation (D) of setting, by the controller, a movement route to an IoT terminal having a final weight p detected by performing the deep learning auction training; and an operation (E) of receiving, by the controller, surveillance data from the IoT terminal having the final weight p as surveillance information and transmitting the surveillance information to a data center through base stations and the Internet.

5. The smart surveillance control method of claim 4, wherein, in the operation (B), the quotation weight (b.sub.i) is calculated using a data redundancy (s.sub.i) and a displacement (d.sub.i) between the UAV and the IoT terminal.

6. The smart surveillance control method of claim 4, wherein the deep learning auction training network includes a plurality of monotonic networks, one allocation network connected to the plurality of monotonic networks, and one payment network connected to the plurality of monotonic networks.

7. The smart surveillance control method of claim 6, wherein the operation (C) further includes: an operation (C-1) of inputting, by the controller, the quotation weight (b.sub.i) to each of the plurality of monotonic networks and converting the input quotation weight (b.sub.i) into a plurality of virtual evaluation weights (b.sub.i); an operation (C-2) of inputting, by the controller, the plurality of virtual evaluation weights (b.sub.i) to the allocation network and determining an IoT terminal corresponding to a highest virtual evaluation weight (b.sub.i); and an operation (C-3) of inputting, by the controller, the plurality of virtual evaluation weights (b.sub.i) to the payment network and determining a final weight (p) indicated by the IoT terminal having the highest virtual evaluation weight (b.sub.i).

8. The smart surveillance control method of claim 4, wherein the controller repeatedly performs the operations (B) to (E) in order to select the next IoT terminal that will receive surveillance data and collect the surveillance data.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The above and other objects, features and advantages of the present invention will become more apparent to those of ordinary skill in the art by describing exemplary embodiments thereof in detail with reference to the accompanying drawings, in which:

[0022] FIG. 1 is an exemplary diagram illustrating the operation of a smart surveillance system according to an embodiment of the present invention;

[0023] FIG. 2 is a configuration diagram of an unmanned aerial vehicle (UAV) constituting the smart surveillance system according to the embodiment of the present invention;

[0024] FIG. 3 is a flowchart for describing a smart surveillance control method according to another embodiment of the present invention;

[0025] FIG. 4 is a structural diagram of a deep learning auction training network according to the smart surveillance control method according to another embodiment of the present invention;

[0026] FIG. 5 is an exemplary diagram illustrating a route setting process according to the smart surveillance control method according to another embodiment of the present invention; and

[0027] FIG. 6 is a graph showing the efficiency of the smart surveillance control method according to another embodiment of the present invention.

DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENTS

[0028] Purposes, specific advantages, and novel features of the present invention will be clear from exemplary embodiments and the following detailed descriptions in connection with the accompanying drawings. In this specification, when reference numerals are assigned to components of each drawing, it should be noted that, when the same components are illustrated in different drawings, the same numerals are assigned to the same components whenever possible. It will be understood that, although the terms “first,” “second,” etc. may be used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. Further, in descriptions of the present invention, when detailed descriptions of related known configurations or functions are deemed to unnecessarily obscure the gist of the present invention, they will be omitted.

[0029] Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. FIG. 1 is an exemplary diagram illustrating the operation of a smart surveillance system according to an embodiment of the present invention, and FIG. 2 is a configuration diagram of an unmanned aerial vehicle (UAV) constituting the smart surveillance system according to the embodiment of the present invention.

[0030] As illustrated in FIG. 1, the smart surveillance system according to the embodiment of the present invention may include one UAV 100, a plurality of Internet of Things (IoT) terminals distributed in a surveillance area, a plurality of base stations distributed in the surveillance area, the Internet 30, and a data center 50. Here, the IoT terminals are a plurality of IoT terminals 201, 202, 203, 204, 205, 206, 207, and 208 including an IoT terminal 201, which are connected to each of a plurality of closed-circuit televisions (CCTVs) distributed in the surveillance area, and the IoT terminals may convert images captured through the CCTVs into surveillance data to transmit the converted surveillance data to the UAV 100.

[0031] In such a smart surveillance system, the UAV 100 may selectively collect the surveillance data generated by the plurality of IoT terminals using deep learning auction training, and may transmit the collected surveillance data to the data center 50 through the Internet 30 and a base station 10 that can communicate with the UAV 100 among a plurality of base stations.

[0032] The UAV 100 is an unmanned aerial vehicle that automatically or semi-automatically flies along a movement route without a pilot directly boarding therein, and may be, for example, a drone. As illustrated in FIG. 2, the UAV 100 may include a controller 110, a transceiver 120, and a storage unit 130.

[0033] Specifically, the controller 110 has a deep learning auction training network therein, and receives quotation weights from the plurality of IoT terminals through the transceiver 120 to perform deep learning auction training, sets the movement route to an IoT terminal corresponding to a final weight obtained as a result of the deep learning auction training to move the UAV 100, and receives surveillance data from the IoT terminal corresponding to the final weight through the transceiver 120 to transmit the surveillance data to the data center 50 through the Internet 30 and the base station 10 that can communicate with the UAV 100.

[0034] Here, the deep learning auction training network may include a plurality of monotonic networks 410, one allocation network 420 connected to the plurality of monotonic networks 410, and one payment network 430 connected to the plurality of monotonic networks 410, as illustrated in FIG. 4.

[0035] The transceiver 120 receives the quotation weights from the respective IoT terminals distributed in the surveillance area via a wireless network under the control of the controller 110 to transmit the quotation weights, and receives the surveillance data from the IoT terminal corresponding to the final weight to transmit the surveillance data. Here, examples of the wireless network include a 3.sup.rd Generation Partnership Project (3GPP) network, a Long-Term Evolution (LTE) network, a 5.sup.th generation (5G) mobile network, a Worldwide Interoperability for Microwave Access (WiMAX) network, the Internet, a local area network (LAN), a wireless local area network (WLAN), a wide area network (WAN), a personal area network (PAN), a Wi-Fi network, a Bluetooth network, a satellite broadcasting network, an analog broadcasting network, a digital multimedia broadcasting (DMB) network, and the like, but the present invention is not limited thereto.

[0036] The storage unit 130 is connected to the controller 110 to store deep learning auction training data, receives the quotation weights from the plurality of IoT terminals to store the received quotation weights, and stores the surveillance data received from the IoT terminal corresponding to the final weight.

[0037] The UAV 100 configured as described above may process the quotation weights received from the plurality of IoT terminals by performing the deep learning auction training to obtain the final weight, move to the IoT terminal corresponding to the final weight to selectively collect the surveillance data, and transmit the surveillance data to the data center 50 through the Internet 30 and the base station 10 that can communicate with the UAV 100.

[0038] Accordingly, in the smart surveillance system according to the embodiment of the present invention, the UAV 100 may selectively collect the surveillance data in consideration of distance and data similarity for the plurality of IoT terminals, and thus economic efficiency can be improved.

[0039] Hereinafter, a smart surveillance control method according to another embodiment of the present invention will be described with reference to FIGS. 3 to 6. FIG. 3 is a flowchart for describing a smart surveillance control method according to another embodiment of the present invention, FIG. 4 is a structural diagram of a deep learning auction training network according to the smart surveillance control method according to another embodiment of the present invention, FIG. 5 is an exemplary diagram illustrating a route setting process according to the smart surveillance control method according to another embodiment of the present invention, and FIG. 6 is a graph showing the efficiency of the smart surveillance control method according to another embodiment of the present invention.

[0040] In the smart surveillance control method according to another embodiment of the present invention, a UAV 100 starts smart surveillance control while wirelessly transmitting a surveillance data request signal to a plurality of IoT terminals 201, 202, 203, 204, 205, 206, 207, and 208 for collecting surveillance data (S210).

[0041] Upon receiving the surveillance data request signal, each of the plurality of IoT terminals 201, 202, 203, 204, 205, 206, 207, and 208 determines a quotation weight b.sub.i in consideration of a data redundancy s.sub.i and a displacement d.sub.i between the UAV 100 and the IoT terminal, and transmits the determined quotation weight b.sub.i to the UAV 100.

[0042] Here, the quotation weight b.sub.i may be defined as a relationship between the data redundancy s.sub.i and the displacement d.sub.i as shown in [Equation 1] below.

[00001]$\begin{array}{cc}{b}_i=\frac{s_i}{d_i}{d}_i=\sqrt{{\left(x_u\left(t\right)-x_i\right)}^2+{\left(y_u\left(t\right)-y_i\right)}^2}{s}_i=\mathrm{MSE}=\frac{1}{\mathrm{mn}}\underset{i=0}{\overset{m-1}{.Math.}}\underset{i=0}{\overset{n-1}{.Math.}}{\left[I\left(i,j\right)-K\left(i,j\right)\right]}^2& \left[\mathrm{Equation}1\right]\end{array}$

[0043] Here, the data redundancy s.sub.i is calculated through a mean square error for all pixels of an image having a size of m×n, K(i, j) denotes an image of surveillance data collected by the UAV 100 in a previous round, and I(i, j) denotes an image of surveillance data that each IoT terminal currently has.

[0044] Further, the displacement d.sub.i denotes a linear distance according to position coordinates (x.sub.u(t), y.sub.u(t)) of the UAV 100 at a time point t and coordinates (x.sub.i, y.sub.i) of each IoT terminal.

[0045] As the data redundancy s.sub.i decreases and the displacement d.sub.i increases, the quotation weight b.sub.i has a smaller value, and thus the possibility that the UAV 100 collects the surveillance data is reduced.

[0046] When the determined quotation weight b.sub.i is transmitted, the controller 110 of the UAV 100 receives the quotation weight b.sub.i from each of the plurality of IoT terminals 201, 202, 203, 204, 205, 206, 207, and 208 through the transceiver 120 (S220).

[0047] The controller 110 of the UAV 100 performs deep learning auction training by inputting the quotation weight b.sub.i to the deep learning auction training network provided therein (S230).

[0048] Specifically, as illustrated in FIG. 4, the controller 110 inputs the quotation weight b.sub.i to each of a plurality of monotonic networks 410 composed of max/min operations over several linear functions and converts the input quotation weight b.sub.i into a virtual evaluation weight b.sub.i using a virtual evaluation function ϕ.sub.i of [Equation 2] below.

ϕ.sub.i(b.sub.i)=max.sub.∀k∈Kmin.sub.∀j∈J(ω.sub.kj.sup.ib.sub.i+β.sub.kj.sup.i)  [Equation 2]

[0049] Here, ω.sup.i.sub.kj and β.sup.i.sub.kj denote neural network parameters of the monotonic networks 410, J denotes the number of linear functions, K denotes the number of groups, and each group consists of J linear functions.

[0050] Thereafter, the controller 110 inputs a plurality of virtual evaluation weights b.sub.i to one allocation network 420 to determine an IoT terminal corresponding to the highest virtual evaluation weight b.sub.i.

[0051] Specifically, the controller 110 inputs the plurality of virtual evaluation weights b.sub.i to the allocation network 420 and calculates allocation probabilities using a softmax function together with a dummy input, b.sub.N+1=0, as shown in [Equation 3] below.

[00002]$\begin{array}{cc}\begin{array}{c}{g}_i\left(\stackrel{\_}{b_i}\right)={\mathrm{softmax}}_i\left({\stackrel{\_}{b}}_1,.Math.,{\stackrel{\_}{b}}_{N+1};k\right)\\ =\frac{e^{k{\stackrel{\_}{b}}_i}}{{.Math.}_{j=1}^{N+1}{e}^{k{\stackrel{\_}{b}}_j}},\forall \in N\end{array}& \left[\mathrm{Equation}3\right]\end{array}$

[0052] The controller 110 may calculate the allocation probabilities using the softmax function, detect an allocation probability g.sub.i of each of the plurality of virtual evaluation weights b.sub.i, and determine an IoT terminal corresponding to the highest value among values of the allocation probabilities g.sub.i.

[0053] At the same time, the controller 110 inputs the plurality of virtual evaluation weights b.sub.i to one payment network 430 to determine a final weight p indicated by the IoT terminal having the highest virtual evaluation weight b.sub.i.

[0054] That is, the controller 110 may input the plurality of virtual evaluation weights b.sub.i to the payment network 430 and calculate a rectified linear unit (ReLU) activation function represented by [Equation 4] below to determine the final weight p.

[00003]$\begin{array}{cc}p\left(\stackrel{\_}{b_i}\right)=\mathrm{ReLU}\left(\underset{\forall j\ne i}{\max }\stackrel{\_}{b_j}\right),\forall \in N& \left[\mathrm{Equation}4\right]\end{array}$

[0055] In order to apply [Equation 4], the payment network 430 may be configured to convert the virtual evaluation function ϕ.sub.i used in the monotonic networks 410 into an inverse function, and the payment network 430 determines the second highest result value among the plurality of result values determined in this way as the final weight p.

[0056] Next, the controller 110 trains neural network parameters ω and β by configuring the plurality of quotation weights b.sub.i input to the monotonic networks 410 as a training set and minimizes a loss function {circumflex over (R)} of [Equation 5] below.

{circumflex over (R)}(ω,β)=−Σ.sub.i=1.sup.Ng.sub.i.sup.(ω,β)xp.sub.i.sup.(ω,β)  [Equation 5]

[0057] Here, g.sub.i denotes the allocation probability, p.sub.i denotes the final weight, and ω and β denote the neural network parameters.

[0058] The loss function {circumflex over (R)} may be minimized during the training process, and this may be considered the same as maximizing collection of data of the UAV 100.

[0059] After the controller 110 detects the IoT terminal having the final weight p in this way, the controller 110 sets the movement route to the IoT terminal having the final weight p (S240).

[0060] That is, as illustrated in FIG. 5, after the UAV 100 collects the surveillance data from the IoT terminal 201 and transmits the collected surveillance data, the controller 110 may detect a proximity IoT terminal 202 as the IoT terminal having the final weight p, and set the movement route to the proximity IoT terminal 202.

[0061] The controller 110 moves the UAV 100 to the proximity IoT terminal 202 along the movement route to receive the surveillance data from the proximity IoT terminal 202 as surveillance information and transmit the surveillance information to the data center 50 through the base station and the Internet 30 (S250).

[0062] Thereafter, in order to select a next IoT terminal that will receive the surveillance data, move the UAV 100, and collect the surveillance data, the controller 110 repeats operation S220 of receiving the quotation weight b.sub.i and operation S250 of transmitting the surveillance information.

[0063] Accordingly, as illustrated in FIG. 5, the UAV 100 collects the surveillance data as the surveillance information while flying through an optimized route moving from the proximity IoT terminal 202 to other IoT terminals 204, 205, and 207.

[0064] Such selection of the movement route is performed whenever each of the next IoT terminals that will receive the surveillance data is selected, and thus it is possible to improve economic efficiency by selectively collecting the surveillance data for the plurality of IoT terminals in consideration of the displacement d.sub.i and the data similarity s.sub.i.

[0065] The efficiency of the smart surveillance control method according to another embodiment of the present invention may be compared with reference to Comparative Example and Example.

COMPARATIVE EXAMPLE

[0066] As experimental conditions, a 6 km×6 km surveillance area is set and 10 IoT terminals are distributed in the surveillance area. A process of collecting, by one UAV, surveillance data from the IoT terminals in a conventional second price auction method is performed 300 times, and thus a yield is detected.

EXAMPLE

[0067] As experimental conditions, a 6 km×6 km surveillance area is set and 10 IoT terminals are distributed in the surveillance area. A process of collecting, by one UAV, surveillance data from the IoT terminals according to the smart surveillance control method according to another embodiment of the present invention is performed 300 times, and thus a yield is detected.

[0068] It can be seen that results of each of Comparative Example and Example may be shown in a result graph I of the Comparative Example and a result graph II of Example as illustrated in FIG. 6, and a difference Gap between the result graph I of Comparative Example and the result graph II of Example increases up to 300 times.

[0069] Accordingly, it can be seen that the smart surveillance control method according to another embodiment of the present invention has considerable efficiency in economic terms as compared to the conventional second price auction method.

[0070] According to the smart surveillance system according to the embodiment of the present invention, a UAV can selectively collect surveillance data in consideration of distance and data similarity for a plurality of IoT terminals, and thus economic efficiency can be improved.

[0071] According to the smart surveillance control method according to another embodiment of the present invention, surveillance data can be selectively collected using deep learning auction training, and thus economic efficiency can be improved.

[0072] Although the technical scope of the present invention has been specifically described with reference to the above exemplary embodiments, it should be noted that the above-described embodiments are for the purpose of explanation and not for the limitation thereof.

[0073] In addition, it will be understood by those skilled in the art that various embodiment may be made without departing from the scope of the present invention.