HIT PERFORMANCE WHILE APPROACHING A TARGET

Abstract

The present invention relates to a computer-implemented method for targeting missiles, to a corresponding computer program, to a corresponding computer-readable medium and to a corresponding data processing device, as well as to a missile.

Claims

1. Computer-implemented method (10) for targeting missiles, comprising the steps of: a) receiving, once and prior to the departure of a missile (40), a template T including a target point of aim; b) repeatedly receiving, during the flight of the missile (40) and at a predefined image cycle rate f.sub.B, image data I from a camera (44) of the missile (40) and inertial range estimations D.sup.IM.sub.neu from an inertial measurement; c) per image cycle of the predefined image cycle rate f.sub.B, calculating a pre-scaled starting parameter vector p* for this image cycle using a last calculated range correction ΔD; d) per image cycle of the predefined image cycle rate f.sub.B, carrying out an iterative Lucas-Kanade method in order to calculate an estimated parameter vector p, including a current scale s.sub.neu based on the current image data I and on the template T, from the calculated pre-scaled starting parameter vector p* by means of mapping W.sub.p, wherein the target point of aim is improved by means of the mapping W.sub.p using the estimated parameter vector p; f) per image cycle of the predefined image cycle rate f.sub.B, calculating a range correction ΔD for the next image cycle from a current scale s.sub.neu, a previous scale s.sub.alt, a current inertial range estimation D.sup.IM.sub.neu and a previous inertial range estimation D.sup.IM.sub.alt; and h) per image cycle of the predefined image cycle rate, controlling the missile (40) in a closed-loop manner in order to target the missile (40) based on the improved target point of aim.

2. Method (10) according to claim 1, further comprising the step of: e) per image cycle of the predefined image cycle rate f.sub.B, compensating, by means of an offset and optionally a scaling factor for the next image cycle, for differences in brightness between the template T and the image data I scaled using the mapping W.sub.p.

3. Method (10) according to claim 1, wherein steps f) and c) and/or e) are carried out only when changes in the scale s become significant, in particular when $\frac{s_{neu}}{s_{alt}}-1>S,$ where S is a predefined threshold value.

4. Method (10) according to claim 1, further comprising the step of: g) selecting, in a scale-controlled manner, a section, replacing the template T, in the current image data I as a new template T for the next image cycle.

5. Method (10) according to claim 1, wherein in step f) an interval of size N is considered and averages over a predefined number M of scales s are used at the respective interval ends to calculate the range correction ΔD.

6. Method (10) according to claim 1, wherein a learning filter is additionally applied in step f).

7. Computer program, comprising instructions which, when the program is executed by a computer, cause the computer to carry out the steps of the method (10) according to claim 1.

8. Computer-readable medium (20) on which the computer program according to claim 7 is stored.

9. Data processing device (30), comprising means (31, 32) for executing the method (10) according to claim 1.

10. Missile (40), comprising: a camera (44); and a data processing device (30) according to claim 9, wherein the camera (44) is communicatively connected to the data processing device (30) and is designed to repeatedly send image data I to the data processing device (30) at the predefined image cycle rate f.sub.B.

Description

[0080] The present invention will be described in greater detail below with reference to the embodiments shown in the schematic drawings, in which:

[0081] FIG. 1 shows a schematic flow diagram of a computer-implemented method for targeting missiles;

[0082] FIG. 2 is a schematic view of a computer-readable medium;

[0083] FIG. 3 is a schematic view of a data processing device; and

[0084] FIG. 4 is a schematic side view of a missile.

[0085] The accompanying figures are intended to provide further understanding of the embodiments of the invention. They illustrate embodiments and, in conjunction with the description, serve to explain principles and concepts of the invention. Other embodiments and many of the advantages mentioned can be seen in the drawings. The elements of the drawings are not necessarily shown to scale with one another.

[0086] In the figures of the drawings, identical, functionally identical and identically acting elements, features and components are each provided with the same reference signs, unless stated otherwise.

[0087] FIG. 1 shows a schematic flow diagram of a computer-implemented method 10 for targeting missiles. The computer-implemented method 10 comprises the following steps: [0088] a) receiving, once and prior to the departure of a missile, a template T including a target point of aim; [0089] b) repeatedly receiving, during the flight of the missile and at a predefined image cycle rate f.sub.B, image data I from a camera of the missile and inertial range estimations D.sup.IM.sub.neu from an inertial measurement; [0090] c) per image cycle of the predefined image cycle rate f.sub.B, calculating a pre-scaled starting parameter vector p* for this image cycle using a last calculated range correction ΔD; [0091] d) per image cycle of the predefined image cycle rate f.sub.B, carrying out an iterative Lucas-Kanade method in order to calculate an estimated parameter vector p, including a current scale s.sub.neu based on the current image data I and on the template T, from the calculated pre-scaled starting parameter vector p* by means of mapping W.sub.p, wherein the target point of aim is improved by means of the mapping W.sub.p using the estimated parameter vector p; [0092] e) per image cycle of the predefined image cycle rate f.sub.B, compensating, by means of an offset and optionally a scaling factor for the next image cycle, for differences in brightness between the template T and the image data I scaled using the mapping Wp. [0093] f) per image cycle of the predefined image cycle rate f.sub.B, calculating a range correction ΔD for the next image cycle from a current scale s.sub.neu, a previous scale s.sub.alt, a current inertial range estimation D.sup.IM.sub.neu and a previous inertial range estimation D.sup.IM.sub.alt; and [0094] g) selecting, in a scale-controlled manner, a section, replacing the template T, in the current image data I as a new template T for the next image cycle. [0095] h) per image cycle of the predefined image cycle rate f.sub.B, controlling the missile in a closed-loop manner in order to target the missile based on the improved target point of aim.

[0096] In step a) the template T and the target point of aim to which the missile is to be steered are received. The target to be hit is described by the template (signature), i.e. an image section from at least one image of a target area, on which image the target to be reached is at least partially or completely mapped. The image of the target area was recorded by an IR camera of the missile and transmitted to a monitoring system. The template may have been selected or “cut out” automatically or by a user (“manually”) in the image of the target area (for example, on a screen on which the image of the target area is displayed by the monitoring system, the user can select the template of the target to be reached by “cutting out” the target to be reached from the image of the target area using a cursor that he controls via a mouse). The point of aim is usually selected in the middle of the template T.

[0097] In step b), the image data I from the IR camera are repeatedly/continuously received at the predefined image cycle rate f.sub.B. The IR camera of the missile accordingly sends, at the predefined image cycle rate f.sub.B, the recorded image data I of the target area in which the target to be reached is located. In addition, at the predefined image cycle rate f.sub.B, current inertial range estimations D.sup.IM.sub.neu or D.sup.IM.sub.t (inertial range estimation at the current point in time or image cycle t) are continuously received. The inertial range estimations D.sup.IM or the changes in the inertial range estimations ΔD.sup.IM are based on the known speed v of the missile (for example 300 km/h) and the elapsed time Δt.

ΔD.sup.IM=vΔt

[0098] In step c), in each image cycle of the predefined image cycle rate f.sub.B, the pre-scaled starting parameter vector p* for the Lucas-Kanade method of this image cycle is calculated by adjusting the previously calculated parameter vector p.sub.alt or p.sub.t−k (k equals 1 or more image cycles) by correcting/pre-scaling the scale s of the previously calculated parameter vector p.sub.alt using the last calculated range correction ΔD.

[0099] In step d), the received template T is tracked in each image cycle using the Lucas-Kanade method (automated target tracking means/tracker of the Lucas-Kanade type) with the four-parametric parameter vector p.

[00013]$p=\left(\begin{array}{c}\Delta x_h\\ \Delta x_v\\ \alpha \\ s\end{array}\right)$

[0100] where Δx.sub.h is the translation in the X direction, Δx.sub.v is the translation in the Y direction, α is the rotation/angle of rotation and s is the scale (zoom factor).

[0101] In the Lucas-Kanade method, the parameter vector p is iteratively estimated/improved until a predefined minimum error reduction ΔE.sub.min for the functional E(p) (see below) is met as the termination criterion.

[00014]$\Delta E_{\min }=\frac{(E_{n-1}\left(p\right)-E_n\left(p\right)}{E_{n-1}\left(p\right)}$

[0102] The Lucas-Kanade method is used to measure, as precisely as possible from image to image, the specified target point of aim of the target to be reached. In each image cycle of the predefined image cycle rate f.sub.B, the target point of aim is improved by means of the mapping W.sub.p using the (iteratively) estimated parameter vector p, by searching for the template T in the current image data I of the current image cycle and, based on this, iteratively estimating the parameter vector p. The point of aim is mapped onto the current image data I, i.e. the current image, via the mapping (warp) W.sub.p according to the particular estimation of the parameter vector p. A difference with respect to a control point is determined there, and the missile is navigated/controlled on the basis of this difference (step h)). The four-parametric parameter vector p is iteratively estimated/changed until the mapping W.sub.p transfers/maps the points x of the template T as precisely as possible to the corresponding points in the current image data I.

W.sub.p=sR(α)+h

W.sub.p=f(p)

where R(α) is a rotation matrix for rotation α and h is a translational movement (translation) in the horizontal direction x.sub.h and in the vertical direction x.sub.v, with

[00015]$h=\left(\begin{array}{c}\Delta x_h\\ \Delta x_v\end{array}\right).$

[0103] The functional E(p) is to be minimised, with x passing through all image points of the template T.

E(p)=Σ.sub.x|I(W.sub.p(x)−T(x))|.sup.2

[0104] Since the changes between two successive image data I of a video sequence are only small, the optimisation problem is solved iteratively using a Taylor series and a compensation calculation over all image points by means of a simple Gauss-Newton or Newton-Raphson descent method. The iteration continues until the predefined minimum error reduction ΔE.sub.min is met as the termination criterion.

[0105] The starting point of the method for the second image data I (the template T is “punched out” from the first image) is the parameter vector p.sub.0.

[00016]$p_0=\left(\begin{array}{c}{x}_{\mathrm{TL},h}\\ x_{\mathrm{TL},v}\\ 0\\ 1\end{array}\right)$

[0106] The initial translation ho corresponds to the top left corner of the punched out template T, with

[00017]$h_0=\left(\begin{array}{c}{x}_{\mathrm{TL},h}\\ x_{\mathrm{TL},v}\end{array}\right).$

Each new image is the result parameter vector of the last image.

[0107] The starting point of the method for all subsequent images/image data I of all subsequent image cycles at the predefined image cycle rate f.sub.B is the (final) estimated parameter vector p from the previous image cycle.

[0108] In step e), for the next image cycle, brightness differences between the template T and the image data I scaled using the mapping W.sub.p are compensated for by means of an offset and a scaling factor (gain). As a result, the difference image, which is taken into account in the compensation calculation for the geometric mapping parameters of the mapping W.sub.p, is kept free from influences of brightness (only the “target structure” is taken into account), since the expression for calculating ΔD (see above) is numerically unstable for very small real scale changes (for long ranges) and thus small estimation errors for the scales can lead to extremely large corrections. By means of the brightness compensation, the scale change is estimated even more effectively and the inertial-based range estimation D.sup.IM is considerably improved as a result.

[0109] In step f), in each image cycle of the predefined image cycle rate f.sub.B, the range correction ΔD is calculated for the next image cycle. In order to reduce the number of iterations required in the Lucas-Kanade method for each image cycle, prior knowledge about the distance to the target is applied in advance to the last estimated scale s.sub.alt or s.sub.t−k. The target distances from the inertial measurements D.sup.IM are related to the scales s of the Lucas-Kanade method. This happens based on the ratio of the current inertial range estimation D.sup.IM.sub.neu or D.sup.IM.sub.t to the previous inertial range estimation D.sup.IM.sub.alt or D.sup.IM.sub.t−k and the ratio of the current scale s.sub.neu or s.sub.t to the previous scale s.sub.alt:

[00018]$\frac{s_{\mathrm{neu}}}{s_{alt}}=\frac{D_{\mathrm{alt}}}{D_{\mathrm{neu}}}\stackrel{\mathrm{def}}{=}\frac{D_{alt}^{IM}+\Delta D}{D_{\mathrm{neu}}^{\mathrm{IM}}+\Delta D}$

[0110] The current scale s.sub.neu is the scale of the parameter vector p calculated in this image cycle in step d). The previous scale s.sub.alt is the scale of the parameter vector p.sub.alt or p.sub.t−k calculated in the previous image cycle in step d). The current inertial range estimation D.sup.IM.sub.neu is the inertial range estimation received in this image cycle. The previous inertial range estimation D.sup.IM.sub.alt is the inertial range estimation received in the previous image cycle. D.sub.alt denotes the previous actual range to the target and D.sub.neu denotes the current actual range to the target.

[0111] Based on this, the range correction ΔD is calculated, which precisely corrects the “incorrect” ranges (=inertial measurements) D.sup.IM integrated from inertial measurements, as follows:

[00019]$\Delta D=\frac{s_{neu}{D}_{neu}^{\mathrm{IM}}-s_{alt}{D}_{alt}^{\mathrm{IM}}}{s_{alt}-s_{neu}}$

[0112] Using the calculated range correction ΔD, the starting parameter vector p* and in particular the scale s of the starting parameter vector p* is pre-scaled in the next image cycle in step c) using the following formula:

[00020]$s_{\mathrm{neu}}=\frac{D_{alt}^{IM}+\Delta D}{D_{\mathrm{neu}}^{\mathrm{IM}}+\Delta D}{s}_{alt}$

[0113] The number of iterations of the Lucas-Kanade method that are necessary to find a sufficiently accurate estimated parameter vector p is thus significantly reduced. By introducing the known scale change, which is as precisely estimated as possible, into the actual tracking method (Lucas-Kanade method) in the course of the pre-scaling, the number of necessary iterations can be significantly reduced. This exact scale change/pre-scaling in turn requires precise range estimation by calculating the range correction ΔD. The scale s estimated in this way is used to correct the inertial-based range estimation D.sup.IM in the subsequent image cycle, which leads to substantial improvements, in particular in the case of moving targets.

[0114] In step f), an interval of size N is also considered and averages over a predefined number M of scales s are also used at the respective interval ends in order to calculate the range correction ΔD. For M=2*k+1 the correction formula for an image at the point in time t is:

[00021]$\Delta D_t=\frac{\left(1\text{/}M\underset{i=-k}{\overset{k}{.Math.}}{s}_{t-N+k+i}\right)*D_{t-N+k}^{\mathrm{IM}}-\left(1\text{/}M\underset{i=-k}{\overset{k}{.Math.}}{s}_{t-k+i}\right)*D_{t-k}^{\mathrm{IM}}}{s_{t-N+k}-s_{t-k}}$

[0115] In step f), a learning filter is additionally applied in order to further protect the correction value from occasional outliers of individual estimations. For this purpose, the effective correction value at the point in time t is calculated as follows:

ΔD.sub.eff,t=(1−α)ΔD.sub.eff,t−1+αΔD.sub.t

where α∈]0,0.5].

[0116] The aim of all of the aforementioned measures is to use the correction estimation method as early as possible or as early as is useful in order to reduce the number of iterations of the Lucas-Kanade method as quickly as possible. The specific parameterisation depends largely on the image quality and the image point resolution.

[0117] In addition, steps c), e) and f) are carried out only if

[00022]$\frac{s_{neu}}{s_{alt}}-1>S,$

where S is a predefined threshold value (changes in the scale s become significant). This also contributes to the numerical stability of the method.

[0118] In step g), a section, replacing the template T, in the current image data I is selected, in a scale-controlled manner, as a new template T for the next image cycle (resampling), in order to refine the resolution of the target on the template T. In particular, as a function of the scale s, resampling of the template T is repeatedly carried out, which subsequently also renders the scale estimation more reliable. During the mentioned resampling by punching out, the four parameters of the parameter vector p have to be correspondingly reset to

[00023]$p_0=\left(\begin{array}{c}{x}_{\mathrm{TL},h}\\ x_{\mathrm{TL},v}\\ 0\\ 1\end{array}\right)$

(as at the start of the method, see above). In addition, the values of the scale buffer (s.sub.alt) have to be divided by the last calculated scale value s.

[0119] In step h), in order to target the missile, in each image cycle of the predefined image cycle rate f.sub.B, the missile is controlled in a closed-loop manner based on the improved target point of aim, by a difference with respect to a control point being determined and the missile being navigated/controlled on the basis of this difference. For this purpose, control commands are transmitted to actuating mechanisms of the missile in order to actuate aerodynamic control means (flaps on winglets/wings), and to drives (e.g. jet engine, propeller, etc.) of the missile. The control commands are derived from the estimated parameter vector p.

[0120] FIG. 2 shows a schematic representation of a computer-readable medium 20.

[0121] A computer program is stored on the computer-readable medium, which program comprises instructions which, when the program is executed by a computer, cause the computer to carry out the steps of the computer-implemented method for (image-based) targeting or flight guidance of missiles according to FIG. 1. By way of example, the computer program is stored on a computer-readable storage disk 20 such as a Compact Disc (CD), Digital Video Disc (DVD), High Definition DVD (HD DVD) or Blu-ray Disc (BD). However, the computer-readable medium can also be a data memory such as a magnetic memory (e.g. magnetic core memory, magnetic tape, magnetic card, magnetic strip, magnetic bubble memory, roll memory, hard disk, floppy disk or removable storage device), an optical memory (e.g. holographic memory, optical tape, Tesa Film tape, LaserDisc, Phasewriter (Phasewriter Dual, PD) or Ultra Density Optical (UDO)), a magneto-optical memory (e.g. MiniDisc or Magneto-Optical Disk (MO-Disk)), a volatile semiconductor/solid-state memory (e.g. Random Access Memory (RAM), Dynamic RAM (DRAM) or Static RAM (SRAM)) or a non-volatile semiconductor/solid-state memory (e.g. Read Only Memory (ROM), Programmable ROM (PROM), Erasable PROM (EPROM), Electrically EPROM (EEPROM), Flash-EEPROM (e.g. USB stick), Ferroelectric RAM (FRAM), Magnetoresistive RAM (MRAM) or Phase-change RAM).

[0122] FIG. 3 shows a schematic representation of a data processing device 30.

[0123] The data processing device 30 comprises means for executing the computer-implemented method for (image-based) targeting or flight guidance of missiles according to FIG. 1 or for executing the aforementioned computer program. The data processing device (data processing system) 30 may be a personal computer (PC), a laptop, a tablet, a server, a distributed system (e.g. cloud system) and the like. The data processing system 30 comprises a central processing unit (CPU) 31, a memory which has a random access memory (RAM) 32 and a non-volatile memory (MEM, e.g. hard disk) 33, a human-machine interface (human interface device, HID, e.g. keyboard, mouse, touchscreen, etc.) 34 and an output device (MON, e.g. monitor, printer, loudspeaker, etc.) 35. The CPU 31, the RAM 32, the HID 34 and the MON 35 are communicatively connected via a data bus. The RAM 32 and the MEM 33 are communicatively connected via another data bus. The computer program can be loaded into the RAM 32 from the MEM 33 or from another computer-readable medium 20. According to the computer program, the CPU 31 executes steps a) to h) of the computer-implemented method as shown schematically in FIGS. 1 to 8. The execution can be initialised and controlled by a user via the HID 34. The status and/or the result of the executed computer program can be displayed to the user by the MON 35. The result of the executed computer program can be permanently stored on the non-volatile MEM 33 or another computer-readable medium 20.

[0124] In particular, the CPU and the RAM 33 for executing the computer program can comprise a plurality of CPUs 31 and a plurality of RAMs 33, for example in a computer cluster or in a cloud system. The HID 34 and the MON 35 for controlling the execution of the computer program can be comprised by another data processing system, such as a terminal, which is communicatively connected to the data processing system 30 (e.g. cloud system).

[0125] FIG. 4 is a schematic side view of a missile 40.

[0126] The missile is here, by way of example, a rocket 40, which comprises the data processing device 30 according to FIG. 3, a plurality of winglets 41 having flaps, a plurality of wings 42 having flaps, a plurality of drives 43 (e.g. jet engine, propeller, etc.) and an IR camera 44. The data processing device 30 is communicatively connected to the winglets 41, wings 42 and drives 43, such that, in each image cycle of the predefined image cycle rate f.sub.B, said winglets, wings and drives are controlled in a closed-loop manner based on the control commands of the data processing device 30 for targeting the missile. The IR camera 44 is communicatively connected to the data processing device 30 and sends image data I during the flight of the rocket 40 at the predefined image cycle rate f.sub.B to the data processing device 30. At the predefined image cycle rate f.sub.B, the inertial range estimations D.sup.IM can be determined and transmitted/provided by a separate device (not shown) that is communicatively connected to the data processing device 30, or by the data processing device 30 itself during the flight of the missile.

[0127] As already described above, the difference with respect to the control point of the rocket 40 is determined and, on the basis of this difference, the rocket 40 is navigated/controlled by the control commands, which are derived from the estimated parameter vector p, being transmitted to actuating mechanisms of the rocket 40 in order to actuate the flaps of the winglets 41 and of the wings 42, and to the drives 43.

[0128] In the preceding detailed description, various features have been summarised in one or more examples in order to improve the cogency of the presentation. It should be clear, however, that the above description is merely illustrative and in no way restrictive in nature. It serves to cover all alternatives, modifications, and equivalents of the various features and embodiments. Many other examples will be immediately and directly apparent to a person skilled in the art on the basis of his technical knowledge in view of the above description.

[0129] The embodiments were selected and described in order to be able to present the principles on which the invention is based and their possible applications in practice as effectively as possible. This enables persons skilled in the art to optimally modify and use the invention and its various embodiments with regard to the intended use.

[0130] In the claims and the description, the terms “including” and “having” are used as neutral terms for the corresponding term “comprising”. Furthermore, the use of the terms “a” and “an” should not fundamentally exclude a plurality of features and components described in this way.

[0131] Without further elaboration, it is believed that one skilled in the art can, using the preceding description, utilize the present invention to its fullest extent. The preceding preferred specific embodiments are, therefore, to be construed as merely illustrative, and not limitative of the remainder of the disclosure in any way whatsoever.

[0132] In the foregoing and in the examples, all temperatures are set forth uncorrected in degrees Celsius and, all parts and percentages are by weight, unless otherwise indicated.

[0133] The entire disclosures of all applications, patents and publications, cited herein and of corresponding German application No. 102020001234.5, filed Feb. 25, 2020, are incorporated by reference herein.

[0134] The preceding examples can be repeated with similar success by substituting the generically or specifically described reactants and/or operating conditions of this invention for those used in the preceding examples.

[0135] From the foregoing description, one skilled in the art can easily ascertain the essential characteristics of this invention and, without departing from the spirit and scope thereof, can make various changes and modifications of the invention to adapt it to various usages and conditions.

LIST OF REFERENCE SIGNS

[0136] 10 computer-implemented method [0137] 20 computer-readable medium [0138] 30 data processing device (data processing system) [0139] 31 CPU [0140] 32 RAM [0141] 33 MEM [0142] 34 HID [0143] 35 MON [0144] 40 rocket [0145] 41 winglets [0146] 42 wings [0147] 43 drives [0148] 44 IR camera