METHOD FOR CONTROLLING MOVEMENT OF AT LEAST ONE MOVABLE OBJECT, COMPUTER READABLE STORAGE MEDIUM AND APPARATUS CONFIGURED TO CONTROL MOVEMENT OF AT LEAST ONE MOVABLE OBJECT

Abstract

A method for controlling movement of at least one movable object, a computer readable storage medium and an apparatus configured to control movement of at least one movable object. Position information of the at least one movable object is retrieved. The retrieved position information is compared with a viewing zone of the camera to determine if the movable object is about to interfere with the viewing zone. An alert is generated for the at least one movable object in case the movable object is about to interfere with the viewing zone (4a, 4b) and movement of the at least one movable object is controlled responsive to the alert.

Claims

1. A method for controlling a movement of at least one movable object relative to a camera capturing a sequence of images of a scene, the method comprising: retrieving position information of the at least one movable object; comparing the retrieved position information with a viewing zone of the camera to determine if the movable object interferes with a security zone associated with the viewing zone; generating an alert for the at least one movable object in case the movable object interferes with a security zone associated with the viewing zone; and controlling movement of the at least one movable object responsive to the alert to avoid the viewing zone.

2. The method according to claim 1, wherein controlling the movement includes modifying a motion of the movable object such that the movable object avoids an interference with the viewing zone.

3. The method according to claim 1, wherein at least a first movable object and a second movable object, each having a camera, are controlled, the method comprising: retrieving first position information of the first movable object; comparing the first position information with a second viewing zone of the camera of the second movable object, to determine if the first movable object interferes with a security zone associated with the second viewing zone; generating an alert for the first movable object in case the first movable object interferes with a security zone associated with the second viewing zone; and controlling movement of the first movable object responsive to the alert to avoid the second viewing zone.

4. The method of claim 1, wherein the security zone associated with a viewing zone comprises the viewing zone plus a margin around the viewing zone.

5. A computer readable storage medium having stored therein instructions enabling controlling a movement of at least one movable object relative to a camera capturing a sequence of images of a scene, wherein the instructions, when executed by a computer, cause the computer to: retrieve position information of the at least one movable object; compare the retrieved position information with a viewing zone of the camera to determine if the movable object interferes with a security zone associated with the viewing zone; generate an alert for the at least one movable object in case the movable object interferes with a security zone associated with the viewing zone; and control movement of the at least one movable object responsive to the alert.

6. An apparatus configured to control movement of at least one movable object relative to a camera capturing a sequence of images of a scene, the apparatus comprising: an input unit configured to retrieve position information of the at least one movable object; a comparing unit configured to compare the retrieved position information with a viewing zone of the camera to determine if the movable object interferes with a security zone associated with the viewing zone; a notification unit configured to generate an alert for the at least one movable object in case the movable object interferes with a security zone associated with the viewing zone; and a movement controller configured to control movement of the at least one movable object responsive to the alert.

7. A vehicle comprising: a position sensor configured to determine position information of the vehicle; a data communication interface configured to receive local information about a viewing zone of a camera; a processing unit configured to compare the position information with the viewing zone of the camera to determine if the vehicle interferes with a security zone associated with the viewing zone, and to generate an alert in case the vehicle interferes with the security zone associated with the viewing zone; and a movement controller configured to control movement of the vehicle responsive to the alert such that an interference with the viewing zone is avoided.

8. The vehicle according to claim 7, further comprising a camera, wherein the data communication interface is further configured to establish a data link with further vehicles of the same or a similar type, wherein local information about a viewing zone of the camera is communicated to the further vehicles.

9. A system comprising a plurality of vehicles according to claim 8.

10. An apparatus configured to control movement of at least one movable object relative to a camera capturing a sequence of images of a scene, the apparatus comprising a processing device and a memory device having stored therein instructions, which, when executed by the processing device, cause the apparatus to: retrieve position information of the at least one movable object; compare the retrieved position information with a viewing zone of the camera to determine if the movable object interferes with a security zone associated with the viewing zone; generate an alert for the at least one movable object in case the movable object interferes with a security zone associated with the viewing zone; and control movement of the at least one movable object responsive to the alert.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0048] FIG. 1 a simplified schematic top view showing two movable objects each being equipped with a camera, said cameras having different viewing zones;

[0049] FIG. 2 schematically illustrates a method for controlling a movement of at least one movable object relative to a camera capturing a sequence of images of a scene;

[0050] FIG. 3 depicts a simplified block diagram of an apparatus configured to control movement of at least one movable object;

[0051] FIG. 4 schematically illustrates a situation in which two movable objects are controlled, so as to not interfere with the viewing zone of the respective object, where there is no security zone;

[0052] FIG. 5 schematically illustrates a situation in which two movable objects are controlled so as to not interfere with the viewing zone of the respective object, where there are two different types of security zones; and

[0053] FIG. 6 schematically illustrates another situation in which two movable objects are controlled so as to not interfere with the viewing zone of the respective object, where there are two different types of security zone.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

[0054] For a better understanding the principles of embodiments of the present solution shall now be explained in more detail in the following description with reference to the figures. It is understood that the present solution is not limited to these exemplary embodiments and that specified features can also expediently be combined and/or modified without departing from the scope of the present solution as defined in the appended claims. In the drawings, the same or similar types of elements or respectively corresponding parts are provided with the same reference numbers in order to prevent the item from needing to be reintroduced.

[0055] FIG. 1 depicts a simplified schematic top view showing a scenario with a first movable object 2a and a second movable object 2b. The movable objects 2a, 2b are each equipped with a camera (not shown), namely a first camera having a first viewing zone 4a and a second camera having a second viewing zone 4b. Each viewing zone is illustrated in FIG. 1 using a pair of two margin lines, namely the first margin lines 6a and the second margin lines 6b. These include a first viewing angle 8a and a second viewing angle 8b, respectively. The viewing angles 8a, 8b of the cameras shall each be defined by their respective focal length f, to which reference is made further below.

[0056] Each of the cameras captures a sequence of images or frames of a scene 10, which is located in the first viewing zone 4a and in the second viewing zone 4b.

[0057] Furthermore, in the scenario of FIG. 1, the movable objects 2a, 2b together with the cameras shall be moving, which means each movable object 2a, 2b has a direction of motion. The first movable object 2a has a first direction of motion 12a (indicated by a corresponding vector) and the second movable object has a second direction of motion 12b (also indicated by a vector).

[0058] In addition to this, there are a first security zone 14a and a second security zone 14b, comprising the first viewing zone 4a and the second viewing zone 4b, respectively. The first and second security zones 14a, 14b are depicted in FIG. 1 using a pair of dashed margin lines, i.e. the first security margin lines 16a and the second security margin lines 16b. The particular application of the security zones 14a, 14b will be explained in the description further below.

[0059] FIG. 2 schematically illustrates a method for controlling a movement of at least one movable object relative to a camera capturing a sequence of images of a scene. Position information of the at least one movable object 2a, 2b is retrieved 40. The retrieved position information is compared 41 with a viewing zone 4a, 4b of the camera to determine if the movable object 2a, 2b is about to interfere with the viewing zone 4a, 4b. In case the movable object 2a, 2b is about to interfere with the viewing zone 4a, 4b, an alert is generated 42 for the at least one movable object 2a, 2b. Movement of the at least one movable object 2a, 2b is then controlled 43 responsive to the alert.

[0060] FIG. 3 shows a simplified block diagram of an apparatus 18 configured to control movement of at least one movable object 2a, 2b. The apparatus 18 is in particular a part of the first and/or the second movable object 2a, 2b. The apparatus 18 comprises an input unit 20 configured to retrieve position information of at least one movable object 2a, 2b. The input unit 20 receives the position information, for example, from a position sensor 21 or via a data link. The position sensor can be a GPS-sensor or the like. The latter can also form part of the apparatus 18, in particular if the apparatus 18 is included in one of the movable objects 2a, 2b.

[0061] Further in particular, the movable objects 2a, 2b are vehicles, for example unmanned airborne vehicles, frequently referred to as UAVs. The retrieved position information is forwarded to a comparing unit 22. This in particular forms part of a processing unit 23, for example a microcontroller or another suitable unit. The comparing unit 22 is configured to compare the retrieved position information with data characterizing the position of at least one of the viewing zones 4a, 4b of the camera(s).

[0062] By way of an example, it is assumed that the apparatus 18 forms part of the first movable object 2a. Its comparing unit 22 compares the position information of this particular movable object 2a with data characterizing the second viewing zone 4b of the second movable object 2b (see FIG. 1). Within this comparison, it is determined if the first movable object 2a is about to interfere with the second viewing zone 4b. Details relative to this will be discussed further below with reference to FIGS. 4 to 6.

[0063] In addition to the mentioned units in the apparatus 18, there is a notification unit 24, which can also form part of the processing unit 23. The notification unit 24 is configured to generate an alert for the movable object 2a, 2b, in case the movable object 2a, 2b is about to interfere with the viewing zone 4a, 4b. When sticking back to the above example, the notification unit 24 is configured to generate an alert for the first movable object 2a, if this particular movable object 2a is about to interfere with the second viewing zone 4b of the second movable object 2b, i.e. of its camera.

[0064] In addition to this, there is a movement controller 26, which is configured to control movement of the movable object 2a, 2b, responsive to the alert. In particular, a direction of motion 12a, 12b is changed such that an interference with the viewing zone 4a, 4b is avoided. When making again reference to the above example, the first direction of motion 12a of the first movable object 2a will be changed in that the first movable object 2a avoids entering the second viewing zone 4b of the camera of the second movable object 2b.

[0065] In the above-described scenario, each movable object 2a, 2b has to be aware of data characterizing the local position of the viewing zones 4a, 4b of the respective other movable objects 2a, 2b. To enable the movable objects 2a, 2b to hold this information, there is a data link 28 between the objects (see FIG. 1). In particular, this is a wireless data link established using frequently known wireless communication technology such as WiFi, NFC, Bluetooth or the like. For establishing data communication via this data link 28, the apparatus 18 comprises a data communication interface 29. This is in particular configured to receive local information about the viewing zones 4a, 4b of the other cameras.

[0066] The movable objects 2a, 2b can be unmanned airborne vehicles forming a system comprising a plurality of similarly configured vehicles. The wireless data link 28 is then established between all the vehicles of the same type. These can form a system of vehicles representing a peer of cameras being useful for capturing of the scene 10, for example.

[0067] In a method for controlling the movement of a first movable object 2a and a second movable object 2b, each having a camera, as for example depicted in FIG. 1, first position information of the first movable object 2a is retrieved. This is, for example, performed using the position sensor 21. Then the first position information is compared with the second viewing zone 4b of the camera of the second movable object 2b to determine if the first movable object 2a is about to interfere with the second viewing zone 4b. An alert is generated for the first movable object 2a if the first movable object 2a is about to interfere with the second viewing zone 4b. Responsive to the alert, movement of the first movable object 2a is controlled in that a collision with the second viewing zone 4b is avoided.

[0068] A practicable solution for determining if a collision of one of the movable objects 2a, 2b with one of the viewing zones 4a, 4b of other movable objects 2a, 2b is about to occur is the use of security zones 14a, 14b. The security zones 14a 14b each comprise the respective viewing zone 4a, 4b and the retrieved position information is compared with the security zone 14a, 14b. It is determined if one of the movable objects 2a, 2b interferes with the security zone 14a, 14b and the alert is generated in case the movable object 2a, 2b interferes with the security zone 14a, 14b.

[0069] When making again reference to the above example, the first direction of motion 12a of the first movable object 2a is changed if the first movable object 2a interferes with the second security zone 14b of the second movable object 2b. Hence, information not only about the viewing zone 4a, 4b, but also about the security zone 14a, 14b is communicated via the data link 28 to the other movable objects 2a, 2b.

[0070] The apparatus 18 configured to control the movement of at least one movable object 2a, 2b in particular further comprises a memory device 30, in particular a non-volatile memory device, such as a flash memory or a hard disc, having stored therein instructions, which are executable by the processing unit 23, which is a processing device. These cause the processing unit 23 to retrieve position information, to compare the retrieved position information with data characterizing a position of the viewing zone 4a, 4b of the camera to generate an alert if the movable object 2a, 2b is about to interfere with the viewing zone 4a, 4b, and to control the movement of the movable object 2a, 2b in response to the alert.

[0071] Determination of a collision of one of the movable objects 2a, 2b with the viewing zone 4a, 4 of another movable object 2a, 2b and the security zone 14a, 14b of said movable objects 2a, 2b, respectively, will be now explained with reference to FIGS. 4, 5, and 6.

[0072] Firstly, imaging of a point M in space having the Cartesian coordinates X, Y and Z is considered. The imaging of this point on a sensor is defined by the below formula (1).

[00001]$\begin{array}{cc}m=\left[\begin{array}{ccc}f& 0& u_0\\ 0& f& v_0\\ 0& 0& 1\end{array}\right]\left[\begin{array}{c}X\\ Y\\ Z\end{array}\right]=\mathrm{KM}=\left[\begin{array}{c}{m}_x\\ m_y\\ m_w\end{array}\right]& \left(1\right)\end{array}$

[0073] K is the transformation matrix of the imaging system. In this matrix, f indicates the focal length of the imaging system defining the viewing angle. u.sub.0 and v.sub.0 represent the pixel coordinates in the image plane of the image sensor. Hence, m is the image of the point M on the image sensor. The image of the point M on the sensor is represented in homogeneous coordinates, which are m.sub.x, m.sub.y and m.sub.w. The pixel coordinates u.sub.0 and v.sub.0 are calculated according to formula (2) below.

[00002]$\begin{array}{cc}{p}_b=\left[\begin{array}{c}\frac{m_x}{m_w}\\ \frac{m_y}{m_w}\end{array}\right]=\left[\begin{array}{c}{u}_b\\ v_b\end{array}\right]& \left(2\right)\end{array}$

[0074] In other words, p.sub.b is the image of the point M on the image sensor. The transformation matrix or imaging matrix K is also defined by formula (3), wherein f is again the focal length and u.sub.0 and v.sub.0 (formula (1)) are calculated from the values of camW and camH.

[00003]$\begin{array}{cc}\left[\begin{array}{ccc}f& 0& \mathrm{camW}.Math.0.5\\ 0& f& \mathrm{camH}.Math.0.5\\ 0& 0& 1\end{array}\right]& \left(3\right)\end{array}$

[0075] These define the resolution of the image sensor in horizontal direction or in the direction of the width of the sensor (camW) and in vertical direction or in the direction of the height of the sensor (camH).

[0076] FIG. 4 illustrates the calculation according to which it is determined whether one movable object 2a, 2b is captured by the camera of another movable object 2a, 2b. In other words, answering the question whether the movable objects 2a, 2b can see each other. Firstly, there is an arbitrary initial global coordinate system 32, in which the Cartesian coordinates X, Y and Z of the point M are defined. In this coordinate system 32, the position of the first movable object 2a is defined by the vector C.sub.A. The position of the second movable object 2b is defined by the vector C.sub.B. A vector t.sub.BA pointing from the second movable object 2b towards the first movable object 2a is defined by formula (4) below.

t.sub.BA=C.sub.AC.sub.B(4)

[0077] For determination whether the first movable object 2a is in the viewing zone 4b of the second movable object 2b, the initial or world coordinate system 32 is converted into a second local coordinate system 34b of the second movable object 2b (illustrated by a dashed line in FIGS. 4 to 6). This is performed by inserting a transformation matrix R.sub.B into the calculation according to formula (1) above, which results in formula (5) below.

[00004]$\begin{array}{cc}{m}_b=K_B.Math.R_B.Math.t_{\mathrm{BA}}=\left[\begin{array}{c}{x}_b\\ y_b\\ w_b\end{array}\right]& \left(5\right)\end{array}$

[0078] K.sub.B is the imaging matrix of the camera of the second movable object 2b and R.sub.B, as mentioned, is the rotation and translation matrix transforming the global coordinate system 32 into the local coordinate system 34b.

[0079] The point, which is imaged onto the sensor of the camera of the second movable object 2b, is defined by the vector pointing from the second movable object 2b towards the first movable object 2a, i.e. t.sub.BA. Similar to the point M above, the position of the first movable object 2a is calculated in homogeneous coordinates x.sub.b, y.sub.b, and W.sub.b.

[0080] The corresponding point on the image sensor is represented by formula (6) below.

[00005]$\begin{array}{cc}{p}_b=\left[\begin{array}{c}\frac{x_b}{w_b}\\ \frac{y_b}{w_b}\end{array}\right]=\left[\begin{array}{c}{u}_b\\ v_b\end{array}\right]& \left(6\right)\end{array}$

[0081] u.sub.b and v.sub.b are the pixel coordinates of the image of the first movable object 2a on the image sensor of the camera of the second movable object 2b.

[0082] Finally, it is determined, whether these pixel coordinates of the first movable object 2a, are in the image plane, to be more precise in the range of the sensor of the camera of the second movable object 2b.

[0083] According to formula (7) below, it is determined whether the pixel coordinates of the position of the first movable object 2a, namely u.sub.b and v.sub.b, are in the image plane, which means if u.sub.b is between 0 and W.sub.B1 and v.sub.b is between 0 and H.sub.B1.

[00006]$\begin{array}{cc}\left(\left[\begin{array}{c}{u}_b\\ v_b\end{array}\right].Math..Math.\mathrm{is}.Math..Math.\mathrm{in}.Math..Math.\mathrm{the}.Math..Math.\mathrm{image}.Math..Math.\mathrm{plane}.Math.\left[0,0,W_B-1,H_B-1\right]\right)& \left(7\right)\end{array}$

[0084] If formula (7) is fulfilled, the first movable object 2a is visible in the camera of the second movable object 2b. This causes the first movable object 2a to take action in that it leaves the viewing zone 4b of the camera of the second movable object 2b. This is in particular performed by the apparatus 18 forming part of the movable object 2b controlling the movement.

[0085] Similarly, calculations can be performed with respect to the visibility of the second movable object 2b in the first viewing zone 4a of the camera of the first movable object 2a. For this purpose, a vector t.sub.BA is calculated and the initial and world coordinate system 32 is transformed into the first local coordinate system 34a of the first movable object 2a. Subsequently, the image matrix of the camera of the first movable object 2a is applied on the position of the second movable object 2b.

[0086] FIG. 5 refers to a situation in which a security zone 14 is defined. By way of an example only, this is depicted for the camera of the second movable object 2b only. The calculation is similar to that which was explained above with reference to FIG. 4. The only difference is a change in the imaging matrix, which is now K.sub.SB (see formula (8) below).

[00007]$\begin{array}{cc}{m}_b=K_{\mathrm{SB}}.Math.R_B.Math.t_{\mathrm{BA}}=\left[\begin{array}{c}{x}_b\\ y_b\\ w_b\end{array}\right]& \left(8\right)\end{array}$

[0087] By changing, for example, the focal length f (see formula (1)), the viewing angle of the camera of the second movable object 2b is widened. It is greater compared to the viewing angle defining the viewing zone 4b, which is the real viewing angle of the camera. By similar calculations, which were outlined with reference to the above formulas (1) to (8), it is determined whether the first movable object 2a is in the second security zone 14b of the second movable object 2b.

[0088] Another option for defining the security zone 14a, 14b is not altering the imaging matrix K.sub.B but by defining an offset of the position of the movable object 2a, 2b. This is illustrated in FIG. 6 and described by formula (9) below.

[00008]$\begin{array}{cc}{m}_b=K_B\left(R_B.Math.t_{\mathrm{BA}}-\left[\begin{array}{c}0\\ 0\\ \end{array}\right]\right)=\left[\begin{array}{c}{x}_b\\ y_b\\ w_b\end{array}\right]& \left(9\right)\end{array}$

[0089] In the local coordinate system 34b, the position of the second movable object 2b is shifted reversely by the amount . Hence, by application of the same imaging matrix K.sub.B, a security zone 14b is defined, which includes the real viewing zone 4b. Again, if a collision of the first movable object 2a with the second security zone 14b is detected, the first movement vector 12a (see FIG. 1) or the first direction of motion is changed in that the first movable object 2a keeps clear of the security zone 14b of the second movable object 2b.

[0090] The above example refers to a situation, in which a collision between the first movable object 2a and the second viewing zone 4b and the security zone 14b of the second movable object 2b, respectively, is explained. Similar mechanisms can be implemented for a plurality of movable objects 2a, 2b keeping clear of the viewing zone 4a, 4b and the security zone 14a, 14b of a camera or of a plurality of cameras. This in particular applies to a situation in which there is a peer of cameras, each camera being mounted on an unmanned airborne vehicle representing a movable object.

[0091] All named characteristics, including those taken from the drawings alone, and individual characteristics, which are disclosed in combination with other characteristics, are considered alone and in combination as important to the present solution. Embodiments according to the present solution can be fulfilled through individual characteristics or a combination of several characteristics. Features which are combined with the wording in particular or especially are to be treated as preferred embodiments.