Systems and methods for detecting flying animals

Abstract

A system for detecting flying animals, the system comprising: a detection module configured to detect flying animals; a local processing module associated with the detection module and configured to remove background noise from data gathered by the detection module; an analysis module configured to receive data from the local processing module and process said data to determine presence of flying animals and classify such flying animals by species; and a deterrent configured to repel one or more species of flying animal based on the classification of detected flying animals.

Claims

1. A system for detecting flying animals, the system comprising: a detection module configured to detect flying animals; a local processing module associated with the detection module and configured to remove background noise from data gathered by the detection module; an analysis module configured to receive data from the local processing module and process said data to determine presence of flying animals and classify such flying animals by species; and a deterrent configured to repel one or more species of flying animal based on the classification of detected flying animals.

2. A system according to claim 1, wherein the detection module comprises at least a first movement recorder and a second movement recorder.

3. A system according to claim 2, wherein a memory is associated with each of the first movement recorder and second movement recorder and is configured to store movement data detected by the first movement recorder and second movement recorder.

4. A system according to claim 2, wherein at least one of the first movement recorder and/or second movement recorder is a camera.

5. A system according to claim 4, wherein at least one of the first movement recorder and/or second movement recorder is a stereoscopic camera, directional camera, spherical camera or thermal camera.

6. A system according to claim 3, wherein at least one of the first movement recorder and/or second movement recorder is a radar.

7. A system according to claim 6, wherein at least one of the first movement recorder and/or second movement recorder is a Doppler radar, noise radar or frequency modulated continuous wave radar.

8. A system according to claim 2, wherein at least one of the first movement recorder and/or second movement recorder is an acoustic device.

9. A system according to claim 2, wherein at least one of the first movement recorder and/or second movement recorder is a directional microphone or an ultrasound recorder.

10. A system according to claim 3, wherein the local processing module is configured to compare data recorded by the first movement recorder and the second movement recorder to identify constant background noise.

11. A system according to claim 1, wherein the local processing module is further configured to apply thresholding to the data recorded by the first movement recorder and the second movement recorder to segment the data and blur background noise.

12. A system according to claim 1, wherein the local processing module is further configured to calculate the size of an object and its position in two dimensional space.

13. A method of detecting objects in real time image data, the method comprising: i) capturing at least two image streams of an environment; ii) converting each associated frame from each image stream into binary form; iii) comparing associated binary frames from each image stream to identify non-environmental objects; iv) applying thresholding to each frame to divide data into segments; and v) determining the size and two dimensional position of each non-environmental object.

14. A method of classifying objects in real time image data, the method comprising: i) combining a plurality of time coded image frames to derive a composite image; ii) using the composite image to determine Cartesian coordinates for one or more non-environmental objects; iii) grouping individual non-environmental objects; iv) applying Kalman filtering to each group of non-environmental objects; v) classifying each non-environmental object by animal species; vi) predicting object flight paths; and vii) providing a probability of collision with a pre-determined target.

15. A method according to claim 14, the method comprising the further steps of: viii) generating an alarm if the probability of collision is greater than 80%; and ix) deploying a deterrent to repulse the classified animal species.

16. A method of predicting bird or bat flight paths, the method comprising: i) comparing an instantaneous subset of data for an identified animal species against a stored subset of data for the same animal species; ii) extracting flight path data from the stored subset of data; iii) extrapolating flight path data from the instantaneous subset data based on flight path data from the stored subset of data; and iv) generating on ore more flight path predictions.

17. A method according to claim 16, wherein the method further comprises the step of: v) predicting the probability of collision of the animal with a target object.

18. A system for detecting, classifying and deterring animal species, the system comprising: two or more image capturing devices; a local processing unit for comparing and converting image frames captured by the two or more image capturing devices into binary format and determining the approximate size and two dimensional position of objects of interest; a central processing unit for determining the three dimensional coordinates and predicted trajectory of the objects of interest and for classifying the objects of interest based on size, appearance and other identifying features; and a deterrent module configured to deploy deterrent countermeasures according to the object classification.

19. A system according to claim 18 further comprising a radar for calculating the height of an object.

20. A system according to claim 18 further comprising one or more environmental sensors for measuring environmental conditions.

21. A system according to claim 20 wherein the one or more environmental sensors comprise one of or a combination of temperature sensors, humidity sensors, wind direction sensors, wind speed sensors, atmospheric pressure sensors, precipitation sensors or gas analysers.

Description

FIGURES

[0019] The invention will now be described by way of reference to the following figures:

[0020] FIG. 1 shows a simplified view of a system according to embodiments of the invention;

[0021] FIG. 2 shows a camera based system according to embodiments of the invention;

[0022] FIG. 3 illustrates an embodiment of the invention utilising a Kalman filter and an artificial neural network;

[0023] FIG. 4 illustrates a particular embodiment of the invention of FIGS. 1 and 2; and

[0024] FIG. 5 illustrates a system for detecting bats.

DESCRIPTION

[0025] A simplified version of a system 10 according to the present invention is shown in FIG. 1. The system comprises a master unit 12 to which one or more pair of slave units 14 are in communication with. Slave units 14 are arranged in pairs with each pair defining a detection module. Each slave unit 14 comprises a movement detector 16 configured to detect and record movement in a target environment, a central processing unit (CPU) or microprocessor 18 for processing recorded data and a memory 20 for storing both recorded and processed data. Each slave unit is connected to the master unit by appropriate communication means 22 such as Ethernet, WIFI, Bluetooth or GSM. Each movement detector associated 16 with a slave unit 14 is focused on the same surveillance area around an object of interest.

[0026] In the embodiment illustrated in FIG. 2, each movement detector 16 includes a camera which could be, directional cameras, spherical cameras, thermal cameras, or a combination thereof. Each camera records image data relating to the same target area. Recorded image data is stored in the memory 20 associated with each respective slave unit 14 and processed by the CPU or microprocessor 18 to convert each frame of the image data into binary format. Each binary image is split into pixels that either have a white value or a black value. A first thresholding process is used to identify whether each pixel should be set as white or black. Each slave unit 14 utilises dedicated software to process image frames recorded by the associated camera 16.

[0027] The binary image data from each slave unit 14 is compared to identify any differences between captured frames. Thresholding is then applied to each frame to segment the binary image. Any constant background elements are blurred and thresholding is applied again to each frame to clearly identify just that portion of each frame that is of interest. In other words, as the number of frames stored in relation to each recording is n>2 or n=2, the absolute difference value between the frames can be calculated to ensure that only moving objects within the captured frames are retained for further analysis. The resulting image is then blurred using Gaussian blur or Gaussian smoothing to remove any discontinuities entered into the image by the first thresholding process. A second thresholding step is then applied to remove unwanted noise from the resulting image and to reduce the number of details that do not relate to moving objects.

[0028] A contour is applied to the image following completion of the second thresholding step. The contour is analysed by the slave unit to determine whether the contour size measured in pixels falls between upper and lower pixel thresholds. If the contour size is less than a lower threshold or higher than upper threshold, the image is not analysed further as the object is assumed by the system not to be a bat or a bird. Provided that the contour size is between the lower and upper thresholds, the localisation of the object is calculated using the centroid method. The centroid of an image (object contour) X is calculated by well-known image moment method. Centroid:

[00001]$\left\{\stackrel{\_}{x},\stackrel{\_}{y}\right\}=\left\{\frac{M_{1.Math.0}}{M_{00,}},\frac{M_{0.Math.1}}{M_{0.Math.0}}\right\}$

[0029] Central moments of digital image f(x,y) ate defined as:

[00002]$M_{\mathrm{ij}}=\underset{x}{.Math.}.Math.\underset{y}{.Math.}.Math.x^i.Math.x^j.Math.I\left(x,y\right)$

[0030] The calculated contour size of the object and its position are then sent to the master unit for further processing.

[0031] The master unit 12 combines data received from each of the pair of slave units 14 associated with a detection module. The combined data is analysed to remove all non-bird/bat objects. The master unit 12 determines the contour size, velocity, position and x, y, z co-ordinates of an object using the 3D Cartesian system. 3D co-ordinates are determined by using the 2D image data sent from the pair of slave units 14 and analysing the absolute difference between the position of the detected object as identified in the 2D image data. All objects identified in the 2D image data are grouped based on the mean square error (mse) between positions:

mse=(y.sub.ay.sub.b)+(x.sub.ax.sub.b)

and their true positions are calculated using a Kalman filter applied to a plurality of frames recorded over a period of time. Where a and b in the equation defines the frames from each of a pair of slave units 14.

[0032] The analysed data is used to calculate a predicted flight path of each recorded bird/bat object and to calculate the probability of a collision with an object of interest, i.e. aircraft, wind turbine or other high value installation. In the event that a bird/bad object is determined to be likely to collide with an object of interest or enter a pre-determined target zone the master unit is configured to automatically trigger an alarm to warn of the risk of a bird/bat strike and/or deploy deterrent countermeasures to repulse birds/bats in the vicinity away from the object of interest.

[0033] All recorded data relating to birds/bats is stored in a centralized data base along with calculated flight paths, time of day and other relevant information such as audio recordings.

[0034] The stored data is used to classify the type of bird/bat that has been recorded by species. This classification is used to select an appropriate deterrent countermeasure for the type of bird/bat recorded. The stored data is also used to improve the accuracy of flight path prediction and collision probability calculations by applying an appropriate algorithm to a sub-set of data relating to a particular bird/bat species.

[0035] To minimise errors in calculated position and to predict the flight path of a bird/bat object, an extended Kalman filter combined with a neural network is used. The following algorithm is used to predict the flight path of a bird/bat object:

[0036] Where:

[0037] x,yCartesian coordinated determined by the cameras

[0038] r, polar coordinates of the detected object.

[0039] The Jacobian of the measurement equation:

[00003]$\hat{x}\left(k+1\right)=\left[\begin{array}{c}{\hat{x}}_{\mathrm{pos}}\left(k+1\right)\\ {\hat{x}}_{\mathrm{vel}}\left(k+1\right)\\ {\hat{y}}_{\mathrm{pos}}\left(k+1\right)\\ {\hat{x}}_{\mathrm{vel}}\left(k+1\right)\end{array}\right]=\left[\begin{array}{cccc}1& d.Math.t& 0& 0\\ 0& 1& 0& 0\\ 0& 0& 1& d.Math.t\\ 0& 0& 0& 1\end{array}\right]\left[\begin{array}{c}{\hat{x}}_{\mathrm{pos}}\left(k\right)\\ {\hat{x}}_{\mathrm{vel}}\left(k\right)\\ {\hat{y}}_{\mathrm{pos}}\left(k\right)\\ {\hat{x}}_{\mathrm{vel}}\left(k\right)\end{array}\right]=F.Math.\hat{x}\left(k\right)$

[0040] The measurement update equation:

[00004]${\hat{m}}_k=\left[\begin{array}{c}{\hat{r}}_k\\ {\hat{}}_k\end{array}\right]=\left[\begin{array}{c}\sqrt{{\hat{x}}_{p.Math.o.Math.s}^2+{\hat{y}}_{p.Math.o.Math.s}^2}\\ a.Math..Math.\tan .Math..Math.2.Math.\left({\hat{y}}_{p.Math.o.Math.s}/{\hat{x}}_{p.Math.o.Math.s}\right)\end{array}\right]$

[0041] Jacobian matrix:

[00005]$H_k=\frac{\stackrel{}{}}{\hat{x}}.Math.{|}_x=\left[\begin{array}{cccc}\cos \left({\hat{}}_k\right)& 0& \sin \left({\hat{}}_k\right)& 0\\ -\sin \left({\hat{}}_k\right)/{\hat{r}}_k& 0& \cos \left({\hat{}}_k\right)/{\hat{r}}_k& 0\end{array}\right]$

[0042] In the FIG. 3 the combination of Kalman filter with the artificial neural network is presented. The determined polar coordinates r, from a number of previous positions of the object are applied to artificial feedforward neural network (ANN) of twenty five neurons in a first layer and ten in a second layer. The task of ANN is to predict the trace vector of the bird in the range of the detection zone 24. The accuracy and precision of the prediction of the bird trace depends on the number of previous points applied to the network and increases with observation time. The neural network is additionally supplied with information about size and/or velocity of the detected object. During the first week of the system work the neural network weights and biases are tuned with a Levenberg-Marquardt backpropagation algorithm (trainlm) to adopt to the behavior of particular sapiens occurring in a given geographical area. As shown in FIG. 4, the detection zone comprises a first detection zone 24a approximately 500-1000 metres from an object of interest. When a flying object enters the first detection zone 24a the system determines whether the flying object is a bird or bat, as described above, and its relative size, position and predicted flight path. As the bird or bat enters the second detection zone 24b the system activates a first deterrent which may be a visual deterrent such as strobe lights. If the bird or bat then subsequently enters a third detection zone 24c which is less than 250 metres from the object of interest the system activates a second deterrent which may be an audible deterrent such as a siren.

[0043] While the invention has been described with reference to the slave units 14 each comprising a movement detector 16 which is a camera, it will be appreciated that other devices may be used as movement detectors, in particular radar or acoustic devices. Doppler radar, noise radar or frequency modulated continuous wave radar devices or directional microphone or ultrasound recorder devices could be used in place of one or more of the cameras of a slave unit 14.

[0044] Multiple detection modules comprising a combination of cameras, radar and acoustic devices may be connected to the master unit 12. In one example, a detection unit comprises two slave units 14 with each slave unit 14 comprising a stereoscopic camera for calculating size and 2D position of an object. In another embodiment one slave unit 14 comprises a stereoscopic camera and the other slave unit comprises a radar unit for detecting height of an object or at least one acoustic device for recording bird/bat calls to assist in classification of objects.

[0045] In order to improve local analysis of recorded data by each slave unit 14, an environmental monitoring module may be associated with each slave unit. The environmental monitoring module comprises a number of different sensors for monitoring environmental factors such as temperature, humidity, wind direction, wind speed, atmospheric pressure, precipitation volume, atmospheric gas content, for example. Each of these factors is accounted for by the software that processes recorded data to more accurately calculate the size and position of recorded objects.

[0046] The ability of each slave unit 14 to communicate with the master unit is critical for real-time detection, classification and deterrence of bird/bat objects. To protect against communication failure, each slave unit 14 communicates with the master unit 12 using several communication protocols including: GPS, radio/satellite, wireline communication, IR illumination, Ethernet or GSM, for example.

[0047] For a wind turbine application, detection modules and environmental sensors are placed on each turbine to provide protection for an entire wind farm installation. Upon detection of a flying object by a detection module, the associated data is processed to calculate the size and position of the object in two dimensions. The resulting data is sent to the master unit 12 for object classification and analysis of Cartesian position and predicted flight path. The position and predicted flight path are used to calculate probability of collision with the wind turbine blades. If the detected object is classified as a protected species of bird or bat, the master unit either instructs the wind turbine blades to slow or stop or emits a deterrent to encourage specific species of bird or bat to move away from the wind turbine.

[0048] For an airport installation, detection modules and environmental sensors are placed around the airport such that at least one detection module covers each segment of airspace surrounding the airport. Upon detection of a flying object by a detection module, the associated data is processed to calculate the size and position of the object in two dimensions. The resulting data is sent to the master unit 12 for object classification and analysis of Cartesian position and predicted flight path. The position and predicted flightpath are used to calculate probability of collision with an aircraft or ground installation or probability of the object entering a pre-determined airspace. The master unit 12 is configured to generate an alarm to alert ground control of bird/bat strike threat and to deploy deterrent countermeasures to repulse birds/bats from the area.

[0049] In relation to a system for detecting bats as shown in FIG. 5, the movement detector is a sound sensor for measuring an ultrasonic signal emitted by bats during flight. Such a system comprises a plurality of ultrasonic sensors 26 placed around each slave unit, typically spaced apart around a one metre perimeter. The recorded ultrasonic signal is sampled at a frequency of 250 kHz and 16 bit resolution. The frequency spectrum is determined by the Fourier Transform on the 100 ms time window.

[0050] Each slave unit 14 is equipped with relevant sound sensor. The data is processed in real-time by the slave unit 14 (simultaneously with the image analysis). To the master unit 12 only information about particular frequencies occurring in the signal is transferred. If the bat detection is confirmed by the number of observations, the signal in time domain is transferred to the master unit 12 in order to calculate the actual bat position. The position is calculated basing on the phase shift between the signal and time difference of arrival.

[0051] It will further be appreciated by those skilled in the art that although the invention has been described by way of example with reference to several embodiments it is not limited to the disclosed embodiments and that alternative embodiments could be constructed without departing from the scope of the invention as defined in the appended claims.