METHOD FOR TRAINING A SUPERVISED ARTIFICIAL INTELLIGENCE INTENDED TO IDENTIFY A PREDETERMINED OBJECT IN THE ENVIRONMENT OF AN AIRCRAFT

Abstract

A method for training an artificial intelligence intended to identify a predetermined object in the environment of an aircraft in flight. The method comprises steps of identifying at least one predetermined object in representations representing at least one predetermined object and its environment, establishing a training set and a validation set, the training set and the validation set comprising a plurality of representations from the representations representing at least one predetermined object, training the artificial intelligence with the training set and validating the artificial intelligence with the validation set. The artificial intelligence may then be used, in a method for assisting the landing of the aircraft, to identify a helipad where the landing operation may be performed. The artificial intelligence may also be used, in a method for avoiding a cable, to identify cables situated on or close to the trajectory of the aircraft.

Claims

1. A method for training a supervised artificial intelligence intended to identify a predetermined object in the environment of an aircraft in flight, wherein the method includes the following steps carried out using a calculator: identifying at least one predetermined object by processing representations representing at least one predetermined object and at least part of its environment, the representations comprising a plurality of representations of the same predetermined object with different values of at least one characteristic parameter of the representations; establishing a training set and a validation set to feed the supervised artificial intelligence, comprising the following sub-steps: selecting a plurality of representations from the representations to form the training set; and selecting a plurality of representations from the representations to form the validation set; training in order to train the supervised artificial intelligence, using at least the training set; and validating in order to validate the supervised artificial intelligence, using at least the validation set.

2. The method according to claim 1, wherein the step of identifying at least one predetermined object by processing the representations comprises the following sub-steps: processing the representations by applying one or more image processing methods from a Sobel filter, a Hough transform, the least squares method, the snake method and the image matching method, in order to identify at least one parametrizable geometric shape; identifying, in each of the representations, at least one predetermined object, by means of the geometric shape(s); and storing, for each of the representations, the representation and the identified predetermined object(s).

3. The method according to claim 1, wherein the characteristic parameter(s) of the representations comprise(s) one or more criteria from an accumulation criterion for the representations, a noise criterion for the representations, a similarity factor criterion for the representations, a distance of the predetermined object in the representations and an angle of view of the predetermined object in the representations, the sub-step of selecting the training set being carried out according to at least one characteristic parameter of the representations.

4. The method according to claim 1, wherein the step of identifying at least one predetermined object comprises a sub-step of automatically labelling the predetermined object(s), the labelling of the predetermined object(s) comprising at least one labelling parameter from a geometric shape of the predetermined object(s), definition parameters of a geometric shape of the predetermined object(s), positioning parameters of the predetermined object(s), and the sub-step of selecting the training set is performed according to at least one labelling parameter.

5. The method according to claim 1, wherein the sub-steps of selecting the training set and selecting the validation set are carried out by random selection from the representations, the representations of the validation set being different from the representations of the training set.

6. The method according to claim 1, wherein the representations are limited to predetermined objects situated in a determined geographical area.

7. The method according to claim 1, wherein the supervised artificial intelligence comprises a multilayer neural network or a support-vector machine.

8. The method according to claim 1, wherein the predetermined object, and its geometric characteristics, are known previously.

9. The method for assisting the landing of the aircraft, the aircraft including at least one on-board specific calculator and at least one specific image capture device connected to the specific calculator, the method being implemented by the specific calculator, wherein the method comprises the following steps: acquiring at least one image of an environment of the aircraft using the specific image capture device(s); and identifying at least one helipad in the environment by processing the image(s) with the supervised artificial intelligence by means of the specific calculator, the supervised artificial intelligence being defined using the training method according to claim 1, the predetermined object being a helipad, the supervised artificial intelligence being stored in a specific memory connected to the specific calculator.

10. The method according to claim 9, wherein the method comprises a step of displaying, on a specific display device of the aircraft, a first identification marker in overlay on the identified helipad(s) in an image representing the environment of the aircraft or indeed in a direct view of the environment through the specific display device.

11. The method according to claim 9, wherein the method comprises a step of determining at least one helipad available for a landing operation from the identified helipad(s) and a step of displaying, on a specific display device of the aircraft, a second identification marker in overlay on the available helipad(s) in an image representing the environment of the aircraft or indeed in a direct view of the environment through the specific display device.

12. The method according to claim 9, wherein the method comprises the following additional steps: selecting a helipad in order to carry out a landing operation on the helipad selected from the identified helipad(s); determining a position of the selected helipad; determining a setpoint for guiding the aircraft to the selected helipad using the specific calculator; and automatically guiding the aircraft towards the selected helipad by means of an autopilot device of the aircraft.

13. The method according to claim 12, wherein the method includes a final step of automatically landing the aircraft on the selected helipad.

14. The method according to claim 9, wherein the method comprises a step of calculating a distance between the identified helipad(s) and the aircraft, using the specific calculator as a function of one or more geometric characteristics of the helipad(s), the geometric shapes associated with the geometric characteristics represented in the image(s), and characteristics of the specific image capture device, and a step of displaying, on the specific display device, the calculated distance of the identified helipad(s).

15. The method for assisting the avoidance of a cable with an aircraft, the aircraft including at least one on-board designated calculator and at least one designated image capture device connected to the designated calculator, the method being implemented by the designated calculator, wherein the method comprises the following steps: acquiring at least one image of an environment of the aircraft using the designated image capture device(s); and identifying at least one cable in the environment by processing the image(s) with the supervised artificial intelligence by means of the designated calculator, the supervised artificial intelligence being defined using the training method according to claim 1, the previously known predetermined object being a cable, the supervised artificial intelligence being stored in a designated memory connected to the designated calculator.

16. The method according to claim 15, wherein the method comprises a step of displaying, on a designated display device of the aircraft, an identification symbol in overlay on the identified cable(s) in an image representing the environment of the aircraft or indeed in a direct view of the environment through the designated display device.

17. The method according to claim 15, wherein the method comprises the following additional steps: determining a position of the identified cable(s); determining a guidance setpoint for the aircraft avoiding the identified cable(s), using the designated calculator; and automatically guiding the aircraft according to the guidance setpoint by means of an autopilot device of the aircraft.

18. The method according to claim 15, wherein the method comprises a step of calculating a distance between the identified cable(s) and the aircraft, using the designated calculator as a function of one or more geometric characteristics of the cable(s), the geometric shapes associated with the geometric characteristics represented in the captured image(s), and the characteristics of the designated image capture device, and a step of displaying, on the designated display device, the calculated distance of the cable(s).

19. A system for assisting the landing of an aircraft, the system including: at least one on-board specific calculator; at least one specific memory connected to the specific calculator; and at least one specific image capture device connected to the specific calculator, wherein the system is configured to implement the method for assisting the landing of an aircraft according to claim 9.

20. An aircraft, wherein the aircraft comprises the system for assisting the landing of the aircraft according to claim 19.

21. A system for assisting the avoidance of a cable with the aircraft, the system including: at least one on-board designated calculator; at least one designated memory connected to the designated calculator; and at least one designated image capture device connected to the designated calculator, wherein the system is configured to implement the method for assisting the avoidance of a cable according to claim

15.

22. An aircraft, wherein the aircraft includes the system for assisting the avoidance of a cable with the aircraft according to claim 21.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0152] The disclosure and its advantages appear in greater detail in the context of the following description of embodiments given by way of illustration and with reference to the accompanying figures, in which:

[0153] FIG. 1 is a view of an aircraft comprising a system according to the disclosure;

[0154] FIG. 2 is a block diagram of a method for training a supervised artificial intelligence intended to identify a predetermined object;

[0155] FIG. 3 is a block diagram of a method for assisting the landing of an aircraft;

[0156] FIG. 4 is a view comprising helipads;

[0157] FIG. 5 is a block diagram of a method for assisting cable avoidance with an aircraft; and

[0158] FIG. 6 is a view comprising suspended cables.

DETAILED DESCRIPTION

[0159] Elements that are present in more than one of the figures are given the same references in each of them.

[0160] FIG. 1 shows an aircraft 1 provided with an airframe 4. A pilot 2 is positioned inside the airframe 4. The aircraft 1 shown in FIG. 1 is an aircraft provided with a rotary wing. In the context of the disclosure, the aircraft 1 may be another type of aircraft, and may comprise, for example, a plurality of rotary wings.

[0161] The aircraft 1 also comprises a system 10 for assisting the landing of the aircraft 1 and a system 40 for assisting cable avoidance with the aircraft 1.

[0162] The system 10 for assisting the landing of the aircraft 1 comprises an on-board specific calculator 11, a specific memory 12, a specific image capture device 15, at least one specific display device 14 and possibly an autopilot device for the aircraft 1. The specific calculator 11 is connected to the specific memory 12, to the specific image capture device 15, to each specific display device 14 and to the possible autopilot device 18, via wired or wireless links. The specific calculator 11 can thus communicate with these elements of the system 10 for assisting the landing of the aircraft 1.

[0163] The system 40 for assisting cable avoidance with an aircraft 1 comprises a designated calculator 41, a designated memory 42, a designated image capture device 45, at least one designated display device 44 and the possible autopilot device 18 of the aircraft 1. The designated calculator 41 is connected to the designated memory 42, to the designated image capture device 45, to each designated display device 44 and possibly to any autopilot device 18, via wired or wireless links. The designated calculator 41 can thus communicate with these elements of the system 40 for assisting cable avoidance.

[0164] By way of example, the calculators 11, 41 may comprise at least one processor and at least one memory, at least one integrated circuit, at least one programmable system, or at least one logic circuit, these examples not limiting the scope to be given to the term “calculator”. The term “processor” may refer equally to a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), a microcontroller, etc.

[0165] Said at least one specific display device 14 comprises, for example, a specific screen 16 positioned on an instrument panel 5 of the aircraft 1 and/or a specific viewing device 17 arranged on a helmet 7 of the pilot 2.

[0166] Said at least one designated display device 44 comprises, for example, a designated screen 46 positioned on the instrument panel 5 and/or a designated viewing device 47 arranged on the helmet 7 of the pilot 2.

[0167] The screens 16, 46 positioned on an instrument panel 5 may be separate, as shown in FIG. 1. The screens 16, 46 may alternatively form a single screen.

[0168] The viewing devices 17, 47 arranged on the helmet 7 of the pilot 2 form a single viewing device 17, 47 allowing information to be displayed in overlay on a direct view of the landscape outside the aircraft 1.

[0169] The image capture devices 15, 45 are positioned so as to capture images of a front zone of the environment of the aircraft 1. The image capture devices 15, 45 are, for example, fastened to the airframe 4 of the aircraft 1 and oriented towards the front of the aircraft 1. The image capture devices 15, 45 may be separate as shown in FIG. 1. The image capture devices 15, 45 may alternatively form a single image capture device. The image capture devices 15, 45 may include, for example, a camera or a photographic device.

[0170] The autopilot device 18 is shared by the two systems 10, 40. The autopilot device 18 can act automatically on the control members of the aircraft 1 in order to transmit one or more setpoints to these control members so as to fly along an expected trajectory towards a target point, for example.

[0171] The specific memory 12 stores a supervised artificial intelligence configured to identify predetermined objects 20, 30 in the environment of the aircraft 1, and more precisely to identify helipads 20, 25 in the environment of the aircraft 1.

[0172] The designated memory 42 stores a supervised artificial intelligence configured to identify predetermined objects 20, 30 in the environment of the aircraft 1, and more precisely to identify cables 30 in the environment of the aircraft 1.

[0173] Each of these supervised artificial intelligences may comprise a multilayer neural network provided with at least two hidden layers or a support-vector machine.

[0174] FIG. 2 shows a block diagram relating to a method for training a supervised artificial intelligence to identify a predetermined object, which may be known previously. To reiterate, a predetermined object may, for example, be a helipad 20, 25 or a cable 30 situated in the environment of the aircraft 1.

[0175] The method for training the supervised artificial intelligence comprises several steps as follows, carried out by means of a dedicated calculator that may be separate from the on-board calculators 11, 41.

[0176] Firstly, a step 100 of identifying at least one predetermined object 20, 30 is carried out by processing the representations representing at least one predetermined object 20, 30 and at least part of its environment. The representations used during this identification step 100 comprise a plurality of representations of the same predetermined object 20, 30 with different values of at least one characteristic parameter of these representations.

[0177] The representations may originate from different sources and be of different types. The representations may comprise images captured by an aircraft in flight by a camera or a photographic device, images from a terrain database, or else synthetic images, for example.

[0178] These representations may also be limited to predetermined objects located in a given geographical area such as a country, a region or a city.

[0179] This at least one characteristic parameter of these representations comprises, for example, an accumulation criterion, a noise criterion for said representation, a similarity factor criterion for these representations, the estimated distance of the predetermined object 20, 30 in each representation, the angle of view relative to the predetermined object, the weather conditions of these representations or indeed the colors, the contrast and/or the brightness of these representations, etc.

[0180] The representations as a whole may optionally form a database.

[0181] This identification step 100 may comprise sub-steps.

[0182] A sub-step of automatically labelling at least one predetermined object may, for example, be carried out by the calculator or may indeed have been carried out beforehand. This labelling comprises at least one labelling parameter for each predetermined object 20, 30, for example the geometric shape of the predetermined object 20, 30, definition parameters of such a geometric shape, such as parameters of the equation defining said geometric shape or its dimensions, and positioning parameters of the predetermined object 20, 30, such as a distance of the predetermined object 20, 30, a focal distance of a lens or a bias of an installation used to capture the predetermined object 20, 30.

[0183] A sub-step 102 of processing these representations is carried out, for example, by the calculator by applying one or more image processing methods such as a Sobel filter, a Hough transform, the least squares method, the snake method and the image matching method. This processing sub-step 102 makes it possible to detect, in each representation, at least one parametrizable geometric shape, such as a line segment, an ellipse, a catenary or other particular geometric shapes. A parametrizable geometric shape can be defined by a number of points of the geometric shape that is to be found.

[0184] Next, a sub-step 103 of identifying at least one predetermined object 20, 30 in each of the representations is carried out by the calculator, using the parametrizable geometric shape.

[0185] An ellipse may correspond to a circle drawn on a helipad 20, 25 and seen at certain angles of view in a representation, and may thus make it possible to identify a helipad 20, 25.

[0186] A line segment may also correspond to a circle drawn on a helipad 20, 25 and seen at a long distance according to a representation, and may thus make it possible to identify a helipad 20, 25. A line segment may also correspond to elements of the letter “H” printed on a helipad 20, 25 and may thus make it possible to identify a helipad 20, 25.

[0187] Such a line segment may also correspond to a building or to an element of a metal structure situated in the environment of the predetermined object. Similarly, a particular geometric shape may also correspond to such a building or such an element of a metal structure.

[0188] A catenary may correspond to a suspended cable 30 and thus make it possible to identify this cable 30.

[0189] A sub-step 104 of storing the representation and this at least one identified predetermined object 20, 30 in a memory connected, for example, in a wired or wireless manner to the calculator, is carried out for each of the representations. Each identified predetermined object 20, 30 can be stored with geometric characteristics associated with the parametrizable geometric shape that made it possible to identify this predetermined object 20, 30, namely a line segment, an ellipse, a catenary or a particular geometric shape.

[0190] Next, a step 110 of establishing a training set and a validation set is carried out, and comprises two sub-steps.

[0191] During a selection sub-step 115, a plurality of representations are selected from all the identified representations in order to form the training set.

[0192] During a selection sub-step 116, a plurality of representations are selected from all the identified representations in order to form the validation set.

[0193] The sub-steps 115, 116 of selecting the training and validation sets may be carried out according to one or more characteristic parameters of these representations, according to at least one labelling parameter or else by random selection from the representations as a whole.

[0194] The selections 115, 116 may be made manually by an operator. These selections 115, 116 may also be made automatically by the calculator, for example as a function of these characteristic parameters of these representations or of a labelling parameter.

[0195] Furthermore, the training and validation sets may be identical or else comprise separate representations.

[0196] The training and validation sets are then used to feed the supervised artificial intelligence.

[0197] Thus, during a training step 120, the training set is used to train the supervised artificial intelligence. During this training step 120, the supervised artificial intelligence is thus trained in order to identify one or more predetermined objects in the representations forming the training set.

[0198] Then, during a validation step 130, the validation set is used to validate the supervised artificial intelligence by using the validation set. During a validation step 130, the efficiency and reliability of the supervised artificial intelligence are verified.

[0199] This supervised artificial intelligence defined in this way can be stored in the specific memory 12 of the system for assisting the landing of the aircraft 1 such that this system 10, using the specific calculator 11, implements the method for assisting the landing of an aircraft, a block diagram of which is shown in FIG. 3. This method for assisting the landing of an aircraft comprises several steps.

[0200] During an acquisition step 210, at least one image of an environment of the aircraft 1 is captured using the specific image capture device 15.

[0201] Then, during an identification step 220, at least one helipad, which may be known previously, is identified in the environment by processing said at least one captured image with the supervised artificial intelligence by means of the specific calculator 11.

[0202] In this way, the supervised artificial intelligence automatically and rapidly identifies, in the captured images, one or more helipads 20 present in the environment of the aircraft 1, and possibly known previously, by identifying the helipads 20, for example by means of geometric characteristics of the helipads 20, or even characteristic elements of the environment.

[0203] The method for assisting the landing of an aircraft may comprise additional steps.

[0204] For example, during a display step 225, a first identification marker 21 is displayed on the specific display device 14, as shown in FIG. 4. The first identification marker 21 may be displayed in overlay on each identified helipad 20 in an image representing the environment of the aircraft 1 on the screen 16 or indeed in a direct view of the environment on the viewing device 17 of the helmet 7. In this way, the pilot can view the presence and the position of each helipad 20 present in front of the aircraft 1. The first identification marker 21 is, for example, elliptical in shape. In FIG. 4, the identified helipads 20 are located at the top of a building 50.

[0205] During a step 230 of determining at least one helipad 25 available for a landing operation, each helipad 25 available for a landing operation from each identified helipad 20 is determined by the specific calculator 11 by means of the supervised artificial intelligence by analyzing the images captured by the specific image capture device 15. This availability of a helipad 20 is determined, for example, by establishing that the letter “H” printed on the helipad 25 is totally visible.

[0206] Next, during a display step 235, a second identification marker 26 may be displayed on the specific display device 14 for each available helipad 25. The second identification marker 26 is displayed in overlay on each available helipad 25 in an image representing the environment of the aircraft 1 on the screen 16 or indeed in a direct view of the environment on the viewing device 17 of the helmet 7. The second identification marker 26 is, for example, in the form of a dot, and may be displayed in a specific color, for example green.

[0207] During this display step 235, a third identification marker 29 may be displayed on the specific display device 14 for each helipad 28 occupied by a vehicle and therefore not available for a landing operation. The third identification marker 29 is displayed in overlay on each occupied helipad 28 in an image representing the environment of the aircraft 1 on the screen 16 or indeed in a direct view of the environment on the viewing device 17 of the helmet 7. The third identification marker 29 is, for example, in the form of a cross, and may be displayed in a specific color, for example red.

[0208] The method for assisting the landing of an aircraft may also comprise additional steps in order for the aircraft 1 to automatically approach an identified helipad 20, 25, or even automatically land on this helipad 20, 25.

[0209] During a selection step 240, a helipad 20, 25 is selected from said at least one identified helipad 20, 25 in order to carry out a landing operation.

[0210] This selection may be made manually by a pilot or a co-pilot of the aircraft 1, for example on the screen 16 provided with a touch panel or by means of an associated pointer. This selection may also be made automatically, in particular when only one helipad 20 is identified or when only one helipad 25 of the identified helipads 20 is available.

[0211] During a determination step 250, a relative position of the selected helipad 20, 25 is determined with respect to the aircraft 1. This relative position may be determined using the specific calculator 11, the images captured by the specific image capture device 15, and optionally the characteristics of the specific image capture device 15 and/or one or more geometric characteristics of the selected helipad 20, 25.

[0212] During a determination step 260, a setpoint for guiding the aircraft to the selected helipad 20, 25 is determined using the specific calculator 11. This setpoint is determined as a function of the relative position of the selected helipad 20, 25 and one or more stored control laws, the guidance setpoint being transmitted to the autopilot device 18.

[0213] During an automatic guidance step 270, an approach phase in which the aircraft 1 approaches the selected helipad 20, 25 is carried out automatically by means of the autopilot device 18.

[0214] During a final automatic landing step 280, the aircraft 1 can be landed on the selected helipad automatically by means of the autopilot device 18, by applying one or more stored control laws.

[0215] The method for assisting the landing of an aircraft may also include a step of calculating a distance between each identified helipad 20, 25 and the aircraft 1 and a step of displaying the calculated distance or distances on the specific display device 14. Each distance is calculated by the specific calculator 11 as a function of one or more geometric characteristics of this helipad 20, 25, the geometric shapes associated with these geometric characteristics represented on said at least one captured image, and the characteristics of the specific image capture device 15.

[0216] The supervised artificial intelligence intended to identify a predetermined object may also be stored in the designated memory 42 of the system 40 for assisting cable avoidance with an aircraft 1 such that this system 40, using the designated calculator 41, implements the method for assisting cable avoidance with an aircraft 1, a block diagram of which is shown in FIG. 5. This method for assisting cable avoidance with an aircraft 1 comprises several steps.

[0217] During an acquisition step 310, at least one image of an environment of the aircraft 1 is captured using the designated image capture device 45.

[0218] Then, during an identification step 320, at least one cable 30, which may be known previously, is identified in the environment by processing said at least one captured image with the supervised artificial intelligence by means of the designated calculator 41.

[0219] In this way, the supervised artificial intelligence makes it possible to automatically and rapidly identify, in the captured images, one or more cables present in the environment of the aircraft 1, by identifying the cable or cables, for example geometric characteristics of the cable or cables, or even characteristic elements of the environment.

[0220] The method for assisting cable avoidance with an aircraft may comprise additional steps.

[0221] For example, during a display step 325, an identification symbol 31 is displayed on the designated display device 44, as shown in FIG. 6. The identification symbol 31 can be displayed in overlay on each identified cable 30 in an image representing the environment of the aircraft 1 on the screen 46 or indeed in a direct view of the environment on the viewing device 47 of the helmet 7. In this way, the pilot can view the presence and the position of each cable 30 present in front of the aircraft 1 or close to its trajectory. The identification symbol 31 has, for example, an elongate shape following the path of the cable 30. In FIG. 6, the identified cables 30 are located high up, between two pylons 34.

[0222] The method for assisting cable avoidance with an aircraft may also comprise additional steps in order for the aircraft 1 to follow a trajectory avoiding an identified cable 30, if necessary.

[0223] During a determination step 350, a position of each identified cable 30 is determined. This position may be relative to the aircraft 1 or absolute in a terrestrial reference frame, for example.

[0224] This position of each identified cable 30 is, for example, determined using the designated calculator 41, the images captured by the designated image capture device 45, and optionally the characteristics of the designated image capture device 45 and/or one or more geometric characteristics of each identified cable 30.

[0225] During a determination step 360, a guidance setpoint enabling the aircraft 1 to avoid each identified cable 30 is determined using the designated calculator 41. This setpoint is determined as a function of the position of each identified cable 30 and one or more stored control laws, the guidance setpoint being transmitted to the autopilot device 18.

[0226] During an automatic guidance step 370, the aircraft 1 can, by means of the autopilot device 18, automatically follow a trajectory avoiding each identified cable 30, by applying the previously determined guidance setpoint.

[0227] The method for assisting cable avoidance may also include a step of calculating a distance between one or more identified cables 30 and the aircraft 1, and a step of displaying the calculated distance or distances on the designated display device 44. Each distance is calculated by the designated calculator 41 as a function of one or more geometric characteristics of this cable 30, the geometric shapes associated with these geometric characteristics represented on said at least one captured image, and the characteristics of the designated image capture device 45.

[0228] The aircraft 1 can thus navigate safely while avoiding any cable identified by the system 40 for assisting cable avoidance with an aircraft.

[0229] Naturally, the present disclosure is subject to numerous variations as regards its implementation. Although several embodiments are described above, it should readily be understood that it is not conceivable to identify exhaustively all the possible embodiments. It is naturally possible to replace any of the means described with equivalent means without going beyond the ambit of the present disclosure and the claims.