METHOD OF DEPLOYING MULTIPLE MONITORING DEVICES

Abstract

The method includes a data collection step, a setup step, a positioning step, an analysis step, and an adjustment step. The data collection step collects spatial data about a scene to be monitored. The setup step installs a number of monitoring devices and a reference device, where the reference device includes a reflector or a calibration pattern. The positioning step determines respective positions of the monitoring devices relative to the reference device. The analysis step determines whether the FOVs of the monitoring devices jointly cover the scene by an algorithm module analyzing the scene data, the FOVs of the monitoring devices, and the relative positions of the monitoring devices against the reference device. The adjustment step provides suggestions about adding one or more monitoring devices or changing positions of the monitoring devices to cover the scene entirely if the FOVs of the monitoring devices do not cover the scene.

Claims

1. A method of deploying multiple monitoring devices, comprising: a data collection step: collecting spatial data about a scene to be monitored, where the spatial data includes the scene's length depth, and height; a setup step: installing a plurality of monitoring devices and a reference device, where the reference device comprises a reflector or a calibration pattern, and each monitoring device has a field of view (FOV); a positioning step: determining respective positions of the monitoring devices relative to the reference device through the monitoring devices detecting the reflector or the calibration pattern; an analysis step: determining whether the FOVs of the monitoring devices jointly cover the scene by an algorithm module analysing the scene data, the FOVs of the monitoring devices, and the relative positions of the monitoring devices against the reference device; and an adjustment step: providing suggestions about adding one or more monitoring devices or changing positions of the monitoring devices to cover the scene entirely if the FOVs of the monitoring devices do not cover the scene.

2. The method of deploying multiple monitoring devices according to claim 1, wherein each of the monitoring devices is a millimeter wave radar or an optical camera.

3. The method of deploying multiple monitoring devices according to claim 1, wherein all monitoring devices are millimeter wave radars or optical cameras.

4. The method of deploying multiple monitoring devices according to claim 1, wherein the reflector is a corner reflector, a Luneburg lens reflector, or a ball reflector.

5. The method of deploying multiple monitoring devices according to claim 1, wherein the calibration pattern is a chessboard pattern, an ArUco pattern, or a ChArUco pattern.

6. The method of deploying multiple monitoring devices according to claim 2, wherein, when a millimeter wave radar is used, the reflector is adopted; and the millimeter wave radar, based on a traversal time of a radio wave transmitted by the millimeter wave radar and reflected back by the reflector, a position of the millimeter wave radar comprising distance, angle, and height, relative to the reflector is determined.

7. The method of deploying multiple monitoring devices according to claim 2, wherein, when an optical cameras is used, the calibration. pattern is adopted; the optical cameras captures an image of the calibration pattern according to a method of camera calibration; the calibration pattern has an actual location in the scene and a pixel location in the image; and, according to a correspondence relation between the actual and pixel locations, the position of the optical camera relative to the calibration pattern is determined.

8. The method of deploying mulfiple monitoring devices according to claim 1, wherein the setup step further comprises providing a turntable on the reference device, and placing the reflector or the calibration pattern on the turntable.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0016] FIG. 1 is a flow diagram showing the steps of a method of deploying multiple monitoring devices according to an embodiment of the present invention.

[0017] FIG. 2 is a schematic diagram showing the field of views (FOVs) of a number of millimeter wave radars.

[0018] FIG. 3 is a schematic diagram showing radio wave transmitted from the millimeter wave radars of FIG. 2 to a reflector.

[0019] FIG. 4 is a schematic diagram showing a reflector spun by a turntable.

[0020] FIG. 5 is a schematic diagram showing the FOVs of a number of optical camera.

[0021] FIG. 6 is a schematic diagram showing a calibration pattern shot by the optical cameras of FIG. 5.

[0022] FIG. 7 is a schematic diagram showing a calibration pattern spun by a turntable.

[0023] FIG. 8 is a schematic diagram showing the FOVs of a number of millimeter wave radars and optical cameras.

[0024] FIG. 9 is a schematic diagram showing radio wave transmitted from the millimeter wave radars of FIG. 8 to a reflector and a calibration pattern shot by the optical cameras of FIG. 8.

[0025] FIG. 10 is a schematic diagram showing a reflector and a calibration pattern spun by a turntable.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0026] The following descriptions are exemplary embodiments only, and are not intended to limit the scope, applicability or configuration of the invention in any way. Rather, the following description provides a convenient illustration for implementing exemplary embodiments of the invention. Various changes to the described embodiments may be made in the function and arrangement. of the elements described without departing from the scope of the invention as set forth in the appended claims.

[0027] As shown in FIG. 1, a method of deploying multiple monitoring devices according to an embodiment of the present invention includes the following steps: a data collection step S1, a setup step S2, a positioning step S3, an analysis step S4, and an adjustment step S5.

[0028] The data collection step S1 collects spatial data about a scene to be monitored, where the spatial data includes the scene's length, depth, and height.

[0029] FIG. 2 is a schematic diagram showing the field of views (FOVs) of a number of millimeter wave radars. FIG. 5 is a schematic diagram showing the FOVs of a number of optical cameras. FIG. 8 is a schematic diagram showing the FOVs of a number of millimeter wave radars and optical cameras.

[0030] The setup step S2 installs a number of monitoring devices, each having a FOV, and a reference device 3. The reference device 3 includes a reflector 11 or a calibration pattern 21. Specifically, the monitoring devices may all be millimeter wave radars for optical cameras 2, or some monitoring devices are millimeter wave radars 1 and some monitoring devices are optical cameras 2.

[0031] FIG. 3 is a schematic diagram showing radio wave transmitted from the millimeter wave radars to the reflector. When the monitoring devices are millimeter wave radars 1, the reflector 11 should be employed. The reflector 11 may be a radar reflector to reflect the radio wave from the millimeter wave radars 1, The radar reflector may be a corner reflector, a Luneburg lens reflector, or a ball reflector. The corner reflector includes three mutually perpendicular plates of high radio reflectivity. Radio wave from a millimeter wave radar 1 incident at a specific angle would be almost entirely reflected back to the millimeter wave radar 1. The millimeter wave radar 1 as such may determine the location of the corner reflector. The Luneburg lens reflector is a spherical reflector coated with a metallic layer. Radio wave from a millimeter wave radar 1 may be reflected along the same incident path.

[0032] FIG. 6 is a schematic diagram showing the reflector shot by the optical cameras 2. When the monitoring devices are optical camera 2, the calibration pattern 21 should be employed. The calibration pattern 21 may be a chessboard pattern, an ArUco pattern, or a ChArUco pattern. For example, a chessboard pattern has alternating squares in dark and light colors arranged in an array. Each square represents a 3D coordinate.

[0033] A user may choose to use millimeter wave radars 1 along with a reflector 11, or to use optical cameras 2 along with a calibration pattern 21. Alternatively, as shown in FIG. 9, if both millimeter wave radars 1 and optical cameras 2 are employed, both a reflector 11 and a calibration pattern 21 should be adopted.

[0034] As shown in FIGS. 2 and 5, the FOV of each monitoring device indicates that an object can only be detected or shot by the monitoring device when it is located within a coverage of the monitoring device. An object outside this coverage of FOV cannot be detected or shot by the monitoring device. In other words, each millimeter wave radar 1 can only detect, and each optical camera 2 can only shoot, objects within its FOV.

[0035] FIG. 4 is a schematic diagram showing the reflector spun by a turntable. FIG. 7 is a schematic diagram showing the calibration pattern spun by the turntable. FIG. 10 is a schematic diagram showing a reflector and a calibration pattern spun by a turntable.

[0036] The setup step may also include having a turntable 31 on the reference device 3 where the reflector 11 or the calibration pattern 21 is positioned. The turntable 31 spins the reflector 11 or the calibration pattern 21 so that the monitoring devices detects or shoots the reflector 11 or the calibration pattern 21 from different angles.

[0037] As shown in FIGS. 1, 3, and 6, the positioning step S3 determines the respective positions of the monitoring devices relative to the reference device 3.

[0038] As shown in FIG. 3, when the monitoring devices are millimeter wave radars 1, the millimeter wave radars 1, based on the traversal times of their respective radio wave reflected back by the reflector 11, may determine their respective positions (including distances, angles, and heights) relative to the reflector 11.

[0039] As shown in FIG. 4, by spinning the reflector 11 by the turntable 31, the monitoring devices may detect the reflector 11 from varying angles. According to the Doppler effect, the reflected radio wave from the reflector 11 may undergo frequency and amplitude changes, depending the movement of the reflector 11. For example, if the reflector 11 approaches a millimeter wave radar 1, the reflected radio wave would have a higher frequency. If the reflector 11 moves away from a millimeter wave radar 1, the reflected radio wave would have a lower frequency. Based on the frequency difference between the transmitted and reflected radio wave, the spinning speed of the reflector 11 may be determined The relative positions of the millimeter wave radars 1 against the reference device 3 may then be determined.

[0040] As shown in FIG. 6, when the monitoring devices are optical cameras 2, the optical cameras 2 captures an image of the calibration pattern 21 according to a method of camera calibration. The calibration pattern 21 has an actual location in the scene and a pixel location in the image. According to the correspondence relation between the actual and pixel locations, the respective positions of the optical cameras 2 relative to the calibration pattern 21 (i.e., reference device 3) may he determined.

[0041] As the 3D scene and the 2D image are of different dimensions, the method of camera calibration is conducted to achieve dimension conversion and to establish correspondence therebetween. Then, what occurs in the 3D scene can be reconstructed by multiple images subsequently taken.

[0042] The method of camera calibration is as follows.

[0043] Step 1: converting a world coordinate system into a camera coordinate system through principle of lens imaging, which includes scaling, rotation, and translation, where the world coordinate system is the 3D coordinate system of the real world and the camera coordinate system is another 3D coordinate system presented in an optical camera 2.

[0044] Step 2: converting the camera coordinate system into an image coordinate system, also known as projection, Where 3D coordinate system is projected to a screen's 2D coordinate system without the dimension of height.

[0045] Step 3: sampling the image coordinates into pixel coordinates discretely, where the pixel coordinates are also 2D coordinates.

[0046] Camera calibration is a known prior art and not the main gist of the present invention. The detail is, therefore, omitted here.

[0047] As shown in FIGS. 1, 3, and 6, the analysis step S4 combines the FOVs of the monitoring devices and determines whether these overlapped FOVs of the monitoring devices cover the scene by an algorithm module's analyzing the scene data, the FOVs of the monitoring devices, the relative positions of the monitoring devices against the reference device 3. Specifically, the algorithm model establishes a geometric model of the scene using the scene data, locates the positions of the monitoring devices in the geometric model based on their relative positions against the reference device 3, and overlaps the FOVs of the monitoring devices in the geometric model to see if there is any blind spot. If there is blind spot, the algorithm module provides adjustment suggestion according to the scene data, the FOVs of the monitoring devices, and the relative positions of the monitoring devices against the reference device 3.

[0048] The adjustment step S5 suggests the addition of more monitoring devices or the position change to the monitoring devices to cover the scene if the existing FOVs of the monitoring devices do not cover the entire scene. A user then can install one or more monitoring devices or change the positions of the monitoring device according to the suggestion provided by the algorithm module, thereby significantly reducing the time and effort in trial and error and enhancing the perfonnance and precision of the scene's surveillance.

[0049] While certain novel features of this invention have been shown and described and are pointed out in the annexed claim, it is not intended to be limited to the details above, since it will be understood that various omissions, modifications, substitutions and changes in the and details of the device illustrated and in its operation can be made by those skilled in the art without departing in any way from the claims of the present invention.