DETECTION OF MODULATING ELEMENTS

Abstract

Methods and apparatus for detecting a modulating element of an object are described in which signal values are obtained from one or more optical detectors. A determination is made if there is a time dependent variation in the obtained signal values at the one or more optical detector and a modulating element is detected based on the determination.

Claims

1. A method for detecting a modulating element of an object, the method comprising: obtaining signal values from one or more optical detectors; determining if there is a time dependent variation in the obtained signal values at the one or more optical detectors; and detecting a modulating element based on the determination.

2. The method according to claim 1, wherein the one or more optical detectors are one or more non-imaging optical detectors.

3. The method according to claim 2, wherein the one or more non-imaging optical detectors operate in a continuous detection mode.

4. The method according to claim 3, comprising using the determination to track the position of the modulating element.

5. The method according to claim 4, comprising providing a control signal based on the tracking of the modulating element for guiding an apparatus towards the detected element.

6. The method according to claim 2, wherein the one or more non-imaging optical detectors comprise four non-imaging optical detectors arranged as a quadrant detector.

7. The method according to claim 2, wherein the one or more non-imaging optical detectors comprise a position sensing device.

8. The method according to claim 1, wherein the one or more optical detectors are pixels of an imaging sensor.

9. The method according to claim 8, wherein the determination is made by comparing pixel values in successive image frames.

10. The method according to claim 9, wherein detecting the modulating element comprises determining a location of the modulating element within a frame based on the determination at respective pixels of the image sensor.

11. The method according to claim 1, comprising processing the signal values to deemphasize or remove an unmodulated background signal component.

12. The method according to claim 11, wherein the processing comprises filtering the obtained signal values.

13. The method according to claim 12, wherein said filtering uses one or both of a band-pass filter and a high-pass filter.

14. The method according to claim 11, wherein the processing comprises mixing a signal generated at a local oscillator with the obtained signal values.

15. The method according to claim 1, wherein said detection includes comparing the determined time-varying signal with a threshold.

16. The method according to claim 1, wherein signal values are obtained from a plurality of optical detectors, the method comprising triangulating the location of the modulating element based on the detection of the modulating element at two or more optical detectors.

17. The method according to claim 1, wherein the modulating element is one or more of: a rotor of a unmanned aerial vehicle (UAV) or other vehicle, an emission from a vehicle exhaust, and a portion of a human face.

18. Apparatus for detecting a modulating element of an object, comprising: means for obtaining signal values from one or more optical detectors; means for determining if there is a time dependent variation in the obtained signal values at the one or more optical detectors; and means for detecting a modulating element based on the determination.

19. The apparatus according to claim 18, wherein: the one or more optical detectors include one or more non-imaging optical detectors, or one or more pixels of an image sensor; the means for obtaining signal values includes a readout circuit; the means for determining if there is a time dependent variation includes a processor; and/or the means for detecting a modulating element includes a processor.

20.-38. (canceled)

39. A computer program product including one or more non-transitory machine-readable mediums encoding instructions that when executed by one or more processors cause a process to be carried out for detecting a modulating element of an object, the process comprising: obtaining signal values from one or more optical detectors; determining if there is a time dependent variation in the obtained signal values at the one or more optical detectors; and detecting a modulating element based on the determination.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Embodiments will now be described, by way of example only, with reference to the accompanying drawings, in which:

[0032] FIG. 1 shows a schematic diagram of a propeller of a vehicle;

[0033] FIGS. 2a and 2b shows schematic diagrams illustrating a pixel grid upon which a rotating propeller is imaged according to an embodiment;

[0034] FIG. 3 is a plot of contrast against time for a signal received at an optical detector from modulating element of an object according to an embodiment;

[0035] FIG. 4 is a schematic diagram of a side-view of a unmanned aerial vehicle showing two visible propellers;

[0036] FIG. 5 is a block diagram showing a readout circuit according to an embodiment;

[0037] FIG. 6 is a block diagram showing a readout circuit according to an embodiment; and

[0038] FIG. 7 shows schematic of a quadrant detector according to an embodiment.

DESCRIPTION

[0039] Example embodiments are described below in sufficient detail to enable those of ordinary skill in the art to embody and implement the systems and processes herein described. It is important to understand that embodiments can be provided in many alternate forms and should not be construed as limited to the examples set forth herein.

[0040] Accordingly, while embodiments can be modified in various ways and take on various alternative forms, specific embodiments thereof are shown in the drawings and described in detail below as examples. There is no intent to limit to the particular forms disclosed. On the contrary, all modifications, equivalents, and alternatives falling within the scope of the appended claims should be included. Elements of the example embodiments are consistently denoted by the same reference numerals throughout the drawings and detailed description where appropriate.

[0041] The terminology used herein to describe embodiments is not intended to limit the scope. The articles “a,” “an,” and “the” are singular in that they have a single referent, however the use of the singular form in the present document should not preclude the presence of more than one referent. In other words, elements referred to in the singular can number one or more, unless the context clearly indicates otherwise. It will be further understood that the terms “comprises,” “comprising,” “includes,” and/or “including,” when used herein, specify the presence of stated features, items, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, items, steps, operations, elements, components, and/or groups thereof.

[0042] Unless otherwise defined, all terms (including technical and scientific terms) used herein are to be interpreted as is customary in the art. It will be further understood that terms in common usage should also be interpreted as is customary in the relevant art and not in an idealized or overly formal sense unless expressly so defined herein.

[0043] The rotation of a propeller that has a different contrast, colour, or temperature to the object behind it, will result in the pixels that capture the propeller exhibiting a time dependent variation in intensity (or colour or temperature). This modulation will be a compound effect of the rotational speed of the propeller, the frame update rate, the pixel stare time, and in some cases occlusion e.g. by structural elements of a UAV.

[0044] A readout circuit can be designed to detect, using electronic or computational means, this regular modulation resulting from the motion of the propeller. This will provide a mechanism from which the object will be detected.

[0045] FIG. 1 depicts a rotating propeller 1 against a lower intensity background 2. The propeller 1 contains two rotor blades 11, 12 which is typical of a lightweight UAV. However, other UAVs or vehicles may have a greater number of blades. The blades 11,12 themselves occupy a segment of a circular rotational region 3 of the propeller. When imaging this propeller with a high-resolution imaging device (e.g. one or more optical detectors), a pixel may only occupy a small region of the propeller rotation circle. If the distance to the propeller is large, however, then the propeller may be sub-pixel in size, in which case some form of occlusion will be required for modulation to be detectable, such that the cross-sectional area being imaged changes at a characteristic frequency.

[0046] FIGS. 2a and 2b show an example of such a high-resolution imaging device having an image sensor that is formed of a grid of pixels 200 (optical detectors). A shaded row 201 and column 202 of pixels intersect at a single pixel 203 that will exhibit a high or low intensity state depending on the contrast, or intensity, of the propeller blade 11,12 relative to its background. For example, in FIG. 2a the pixel of interest 203 will exhibit a high value signal at the pixel due to the pixel being occupied by the blade 12. At a subsequent point in time the propeller will have rotated such that the blade 12 no longer covers the pixel 203 and thus, the pixel will output a low signal value e.g. having an intensity corresponding to the background contrast level. In other words, with propeller rotation the value of the pixel will undergo a detectable modulation as shown in FIG. 3.

[0047] FIG. 3 shows a plot with contrast on the ordinate axis and time measured along the abscissa axis. The modulation can be seen in the form of the peaks in contrast (which will translate to signal value received at the pixels) forming a pulse train in which the pulses are separated by gaps (mark-space ratio) along the abscissa which represent the time period for the propeller 11 to rotate such that the following blade 11, 12 reaches the pixel 203 and causes the contrast (signal intensity) to increase. The mark-space ratio and repetition frequency of the pulse train will vary in proportion to the factors including propeller blade occupancy, rotation speed, and sensor (pixel) stare time.

[0048] FIG. 1 and FIG. 2 depict a bottom up (or top down) view of a propeller 1 with two rotor blades 11, 12. The principle of detection of a modulating element similarly works with a side on, or also an oblique view of the air platform of the UAV. FIGS. 4a and 4b depict a quadrotor UAV 400 with two visible rotating propellers 401, 402, each having two blades. As the blades rotate, the side-on visibility of the rotor blade also modulates as the blades (in this example a dual bladed rotor element) rotate from the blade being parallel to imaging plane (FIG. 4a) i.e. the blade is side on, to being perpendicular to the image plane (FIG. 4b) i.e. the blade is end on.

[0049] Dual bladed rotors (propellers) such as that depicted in FIGS. 1 and 4a, 4b are typical on small UAVs, but the principle of operation of this method also works for rotors with three or more blades.

[0050] Other non-modulating image elements will not vary in such a repetitive manner, this makes the detection and location of the modulating object within an image scene easy to detect, even when it occupies a small proportion of the image. The modulated signal should be detectable even when the pixel (optical detector) is large compared to the size of the target object on the image. A single pixel (optical detector) that occupies a large proportion of the propeller circle region will still contain a modulated component.

[0051] In particular the case where the pixel size fully encompasses the bladed region, and the optical detection/imaging is performed when observing the object (e.g. UAV) from above or below the ability to continue to detect the modulating element will depend on the presence of occluding elements (i.e. struts or other structural elements of the UAV), or on the angle of the blades relative to the field of view of the imager. In the case of FIGS. 4a and 4b, a modulating signal will still be present due to the change in contrast at the single detector site as the blade rotates from side on to end on with respect to the optical detector.

[0052] FIG. 5 shows a circuit for readout of a signal value from an optical detector e.g. a pixel. The combination of the photodetector 501 and amplifier 502 may exemplify a single pixel (optical detector) of an imager focal plane array. This is followed by a filter 503, e.g. a high pass filter setting a filter pass threshold to a level that will remove the unmodulated background signal of the image. The output of this filter is then integrated by an integrator 504 and a threshold detection circuit 505 will determine whether this modulated signal is a true detection. The threshold detection circuit being configured to only return signals that cross a threshold to be detected, thus avoiding false detections from noise or other signal artefacts.

[0053] If the pixel modulation frequency is determinable, the filter 503 may be a band-pass filter. This would have the advantage of reducing in-band noise. Further, the filter may be configured to only let through modulating signals having a frequency or frequency range that is indicative of the type of object being detected. In other words, the filtering could be tuned to the frequency of a target category of object (i.e. based on a known rotation speed or range of speeds of a UAV). In another embodiment, the filter could by configured to track the modulating frequency one it has been detected or to adaptively narrow its bandwidth to improve the signal to noise ratio.

[0054] In another embodiment, shown in FIG. 6, the pixel photodetector 501, 502 may be connected to a heterodyne mixing circuit 601, such that the received signal is mixed with a local oscillator signal provided by a local oscillator 602. The output from the mixer 602 may then be filtered prior to threshold detection. This circuit arrangement has the advantage of that the characteristic frequency of the modulation is moved to a lower value such that it can be detected with low bandwidth electronics which are typically cheaper and less complex). The circuit of FIG. 6 shows both mixer and oscillator 601, 602 and a filter 503, however, in some embodiments, the filter may be omitted and only the mixer and oscillator 601, 602 used.

[0055] While the camera is staring, electrons may be collected in a capacitor associated with the pixel photodetector 501, 502 (not shown). During read out electronic switches (not shown) may be configured to discharge the capacitor to an analogue to digital converter (not shown) and isolate it from the pixel before resetting. This circuitry is typically part of a focal plane array of a camera and thus is difficult/expensive to change. However, the filter 503, integrator 504 and threshold circuitry 505 or the mixer 601 and local oscillator 602, could easily be implemented using digital signal processing techniques.

[0056] In an imaging sensor, the filtering in FIGS. 5 and 6 may happen after the frame is read out via the read-out integrated circuit 501, 502. In other words, the filtering is applied to each pixel upon readout of the whole image frame from the sensors grid of pixels. In an embodiment, the filtering capability is included on the analogue or digital electronics of each pixel i.e. on a pixel-by-pixel basis. Therefore, only modulating pixels are ‘triggered’ (e.g. by the threshold detection circuit 505) and these ‘triggered’ pixels can be read out without reading the non-‘triggered’ pixels i.e. rather than reading out the whole frame. This would be advantageous because it would allow a higher data rate and for accessing high modulation rates (frequencies) without aliasing. In an embodiment, the implementation would use a dedicated ASIC readout integrated circuits (ROICs). The additional expense of the dedicated circuitry would be justified in high value or critical applications, however.

[0057] The above circuit may also be implemented digitally within a digital signal processor, or other suitable processing means. For example, it may be embodied within the digital signal processing unit of an optical detector (e.g. an imaging sensor). It may be encoded within the firmware of an imaging readout sensor, or implemented as an analogue detection component of a focal plane array readout circuit such as those of FIGS. 5 and 6.

[0058] In an embodiment, the invention may be implemented using a non-imaging optical detector. An optical detector configuration that could use either a single, bi, or quadrant detector arrangement rather than an imaging sensor. These single-pixel variants may operate in a continuous detection mode and would continue to observe a modulation signal relative to the rotor blade motion.

[0059] One example of which may be a single pixel detector. Non-imaging optical detectors typically have a mode in which a signal can be continuously obtained which is advantageous as it is not necessary to take into account frame rates and artefacts that may result from using an imaging optical detector such as a camera image sensor which obtains frames at discrete time intervals.

[0060] In a further embodiment a quadrant detector may be used. A quadrant detector may be able to locate as well as detect the modulating signal. The principle of a quadrant detector is widely used as part of a feedback loop in tracking systems. In this application, the quadrants of the detector may operate continuously (as opposed to the discrete time mode of an imaging pixel). The signal received on each detector may operate with the receiver circuit shown in FIG. 5 for example.

[0061] The propeller would be defocussed at the imaging plane of the quadrant sensor and part of it may need to be occluded in order to observe modulation, particularly if the propeller is being viewed from directly above or below. The defocussing allows the modulating signal needs to be present in all four quadrants simultaneously for accurate detection of location.

[0062] FIG. 7 shows a quadrant detector 700 according to an embodiment. The quadrant detector has four non-imaging optical detectors (usually photodiodes) 701a, 701b, 701 c, 701d arranged about four quadrants of a square. A quadrant detection mode of the quadrant detector 700 enables the detector to locate and track the UAV. For example, a source (target) in the scene may be defocussed and is imaged onto the four quadrants together. The relative values in each of the four quadrants 701a, 701b, 701 c, 701d provide the position. That is to say, where the defocused image of the modulating element is not located centrally about the plane of the quadrant detector 700 the respective amplitudes of the detected modulation in signal level in the respective areas 701a, 701b, 701 c, 701d may be different and these differences will result in different detected signal values that can be used to determine the position of the modulating element to provide the a relative displacement to the origin of the quadrant detector 700.

[0063] In an embodiment, the quadrant detector may be attached to a control module or other circuitry for controlling or guiding the movements of an apparatus towards (or away) from the modulating element. For example, the quadrant detector may provide signals that are used to control a mechanical steering arrangement to enable other sensors or actuators to be directed towards the modulating object. In a military environment this could be used as part of a weapon or guidance system to track and engage a target object having a modulating element. For example, to track and engage a target hostile UAV or other object.

[0064] Considering the example of FIG. 7, the optically defocused image circle of a propeller 1 is at a first position 702a. Due to the image blurring the depth of signal modulation in each quadrant should be proportional to the amount of the blurred propeller each quadrant contains. At this position the signal at the upper-right detector 701b will be highest due to the defocussed image 702a occupying a larger portion of the upper right quadrant 701b than any other quadrant. The signal at the lower-left quadrant 701c will be lowest due to having the smallest portion of the blurred image circle 702a covering that area. The signals at the upper-left 701a and lower-right 701d will be somewhere between these two extremes as the image circle 702a modestly encroaches into these quadrants. These relative signal levels will allow a displacement to be determined from the signal values e.g. by a control module, which may send the signal to direct a steering arrangement to guide the apparatus with the quadrant detector to the target propeller 1 according to the displacement. After the steering mechanism is thus controlled, the defocused image circle of the propeller 1 may be located at the centre of the quadrant detector at position 702b, for example.

[0065] Although a quadrant detector is shown in the example of FIG. 7, in other embodiments, the number of optical detectors may be different. For example, a bi-cell detector may be used. Alternatively, the number of detectors may be greater than four, for example, where the optical detectors are arranged as segments of a circular sensor region about a point of origin (not shown).

[0066] In another embodiment, a type of non-imaging optical detector (sensor) called a position sensing device (also called a position sensitive device or position sensitive detector) may be used. A position sensing device is an optical position sensor that includes a semiconductor diode with North and South contacts on the top surface and East and West contacts on the bottom surface. Illumination of a portion of the diode surface may cause a change in local resistance and thus changes in electrical current flowing through the pairs of contacts. The relative electrical currents through the pairs of contacts relate to the position of illumination through the change in resistances. As will be appreciated, position sensing devices may operate isotropically and provide a continuous signal output or may be configured as a discrete sensor.

[0067] In another embodiment, the position may be determined (and thus the object tracked) using the outputs of two or more detectors together with a suitable baseline (i.e. point of reference) to triangulate the location of the target object according to method of triangulation as known to those skilled in the art.

[0068] The modulating element may generate a modulating signal at the optical detector as a result of a change in contrast, intensity, colour, or temperature for example. In the latter case the sensor would be responsive to the thermal band.

[0069] The methods and apparatus described may be advantageous in other applications than detection of UAVs. For example, the modulating element may be the thermal emissions of a vehicle exhaust that are modulated by the engine revolutions, enabling the detection of vehicles from a modulating exhaust signature. In a further potential application, the modulating element may be the colour of a human face resulting from the heartbeat signature. Detection of the modulating element may enable face detection in cluttered scenes.

[0070] In the above description we refer to pixels when describing certain embodiments with imaging and non-imaging optical detectors but this is not intended to be limiting. Any means of optical detection may be used e.g. a photo sensor or diode, which is capable of detecting light intensity levels and producing a measurable signal output in proportion to the amount of light detected.

[0071] Examples in the present disclosure can be provided as methods, systems or machine readable instructions, such as any combination of software, hardware, firmware or the like. Such machine readable instructions may be included on a computer readable storage medium (including but not limited to disc storage, CD-ROM, optical storage, etc.) having computer readable program codes therein or thereon.

[0072] The methods described herein may be realised by machine readable instructions. The machine readable instructions may, for example, be executed by a general purpose computer, a special purpose computer, an embedded processor or processors of other programmable data processing devices to realize the functions described in the description and diagrams. In particular, a processor or processing apparatus may execute the machine readable instructions. Thus modules of apparatus (for example, an imaging device such as an optical detector or a camera with an image sensor) may be implemented by a processor executing machine readable instructions stored in a memory, or a processor operating in accordance with instructions embedded in logic circuitry. The term ‘processor’ is to be interpreted broadly to include a CPU, processing unit, ASIC, logic unit, or programmable gate set etc. The methods and modules may all be performed by a single processor or divided amongst several processors. Such machine readable instructions may also be loaded onto a computer or other programmable data processing devices, so that the computer or other programmable data processing devices perform a series of operations to produce computer-implemented processing, thus the instructions executed on the computer or other programmable devices provide an operation for realizing functions specified herein.

[0073] Such machine readable instructions may also be stored in a computer readable storage that can guide the computer or other programmable data processing devices to operate in a specific mode. For example, the instructions may be provided on a non-transitory computer readable storage medium encoded with instructions, executable by a processor.

[0074] Further, the teachings herein may be implemented in the form of a computer software product, the computer software product being stored in a storage medium and comprising a plurality of instructions for making a computer device implement the methods recited in the examples of the present disclosure.

[0075] The present inventions can be embodied in other specific apparatus and/or methods. The described embodiments are to be considered in all respects as illustrative and not restrictive. In particular, the scope of the invention is indicated by the appended claims rather than by the description and figures herein. All changes that come within the meaning and range of equivalency of the claims are to be embraced within their scope.