METHOD, AERIAL VEHICLE AND SYSTEM FOR DETECTING A FEATURE OF AN OBJECT WITH A FIRST AND A SECOND RESOLUTION

Abstract

Embodiments according to a first and second aspect of the present invention are based on the core idea of flying along the object for detecting a feature of an object and detecting at least a part of the object with a capturing unit with a first resolution and providing, for those areas of the object that comprise the feature, images with the second resolution that is higher than the first resolution.

Claims

1. A method for detecting a damage of an object, comprising: (a) flying along the object and optically detecting at least a part of the object by at least one capturing unit with the first resolution to generate a plurality of images, wherein each image represents an at least partly different area of the object, (b) evaluating the plurality of images to classify the generated images into images that do not comprise the damage and into images that comprise the damage, and (c) optically detecting again those areas of the object whose allocated images comprise the damage with a second resolution that is higher than the first resolution.

2. The method according to claim 1, wherein step (b) is performed after flying along the object, and step (c) comprises approaching those areas of the object whose allocated images comprise the damage.

3. The method according to claim 2, wherein in step (a) and in step (c), the capturing unit generates one image each with the same focal length, in step (a), the object is approached such that the first capturing unit has a first distance to the object when generating an image, and in step (c), the object is approached such that the capturing unit has a second distance to the object that is lower than the first distance when generating an image.

4. The method according to claim 2, wherein in step (a) and in step (c), the object is approached such that the capturing unit has the same or similar distance to the object when generating an image, in step (a), the capturing unit generates an image with a first focal length, and in step (c), the capturing unit generates an image with a second focal length that is greater than the first focal length.

5. The method according to claim 2, wherein optically detecting the area again in step (c) comprises generating a plurality of partial images of the area, each with the second resolution.

6. The method according to claim 2, wherein position and/or location information of the capturing unit is allocated to each image generated in step (a), and in step (c), the areas of the object that are to be flown along are determined by using the position and/or location information of the images comprising the damage.

7. The method according to claim 2, wherein an unmanned aerial vehicle, such as a drone comprising the capturing unit, flies along the object, and step (b) comprises transmitting the images generated in step (a) from the unmanned aerial vehicle to a computer, for example a laptop computer, and evaluating the images by the computer; and evaluating the images comprises evaluating the images in an automated manner.

8. The method according to claim 7, wherein in step (a), the unmanned aerial vehicle flies along the object autonomously, step (b) comprises generating waypoints by using the position and/or location information of the images comprising the damage and transmitting the waypoints to the unmanned aerial vehicle, and in step (c), the unmanned aerial vehicle approaches the areas of the object autonomously by using the waypoints.

9. The method according to claim 2, wherein an unmanned aerial vehicle, e.g. a drone comprising the capturing unit, flies along he object autonomously, the unmanned aerial vehicle comprises a computer, wherein step (b) comprises evaluating the images and generating waypoints by using the position and/or location information of the images comprising the damage by the computer of the unmanned aerial vehicle, wherein evaluating the images comprises evaluating the images in an automated manner, and in step (c), the unmanned aerial vehicle approaches the areas of the object autonomously by using the waypoints.

10. The method according to claim 1, wherein flying along the object in step (a) is flying along the object autonomously by an unmanned aerial vehicle, wherein the unmanned aerial vehicle comprises the at least one capturing unit; and wherein step (b) comprises generating waypoints by using the position and/or location information of the images comprising the damage and transmitting the waypoints to the unmanned aerial vehicle; and step (c) comprises flying along the area of the object autonomously by using the waypoints.

11. The method according to claim 1, wherein steps (a) to (c) are performed during flying along the object such that in step (a), an image of an area is generated, in step (b), the image generated in step (a) is classified for a further area prior to generating an image, when the image is classified as comprising the damage in step (b), prior to generating the further image, the area is optically detected again in step (c) before an image is generated for the further area and when the image is classified as not comprising the damage in step (b), an image is generated for the further area.

12. A method for detecting a damage of an object, the method comprising: (a) flying along the object and optically detecting at least a part of the object by at least one capturing unit to generate a plurality of images, wherein each image represents an at least partly different area of the object, and wherein, for one area, an image with the first resolution and a plurality of partial images, each with a second resolution that is higher than a first resolution, are generated, (b) evaluating the plurality of images to classify the generated images into images that do not comprise the damage and into images that comprise the damage, and (c) providing the partial images of those areas of the object whose allocated images comprise the damage.

13. The method according to claim 12, wherein an unmanned aerial vehicle, e.g., a drone comprising the capturing unit, flies along the object autonomously, step (b) comprises transmitting the images and partial images generated in step (a) from the unmanned aerial vehicle to a computer, e.g., laptop computer, and evaluating the images by the computer, wherein evaluating the images comprises evaluating the images in an automated manner and step (c) comprises providing the partial images of the area allocated to the image by the computer.

14. The method according to claim 12, wherein an unmanned aerial vehicle, e.g., a drone comprising the capturing unit, flies along the object autonomously, the unmanned aerial vehicle comprises a computer, wherein step (b) comprises evaluating the images and the partial images by the computer of the unmanned aerial vehicle, wherein evaluating the images and the partial images comprises evaluating the images and the partial images in an automated manner and in step (c), the unmanned vehicle transmits the partial images to an evaluating unit, e.g., for classifying or cataloging the detected damages.

15. The method according to claim 1, wherein step (b) comprises AI or machine learning.

16. An unmanned aerial vehicle, e.g., drone for detecting a damage of an object, comprising: at least one capturing unit for generating images by optical detection, wherein the unmanned aerial vehicle can be controlled to fly along the object and to optically detect at least part of the object by the capturing unit with a first resolution to generate a plurality of images, wherein each image represents an at least partly different area of the object and optically detect again those areas of the object whose allocated images comprise the damage with a second resolution that is higher than a first resolution; wherein the unmanned aerial vehicle is configured to transmit the plurality of images to an external computer, e.g., laptop computer that classifies the generated images into images that do not comprise the damage and into images that comprise the damage, receive information from the external computer that indicate the areas of the object to be optically detected by the second resolution, or wherein the unmanned aerial vehicle comprises a computer that is configured to evaluate the plurality of images to classify the generated images into the images that do not comprise the damage and into the images that comprise the damage.

17. An unmanned aerial vehicle, e.g., drone, for detecting a damage of an object comprising: at least one capturing unit for generating images by optical detection, wherein the unmanned aerial vehicle can be controlled to fly along the object and optically detect at least a part of the object by the capturing unit to generate a plurality of images, wherein each image represents an at least partly different area of the object and generate, for each area, an image with a first resolution and a plurality of partial images, each with a second resolution that is higher than the first resolution.

18. A system for detecting a damage of an object comprising: an unmanned aerial vehicle, e.g., a drone, wherein the unmanned aerial vehicle can be controlled to fly along the object to optically detect at least a part of the object by at least one capturing unit with a first resolution to generate a plurality of images, wherein each image represents an at least partly different area of the object, wherein the system is configured to evaluate the plurality of images to classify the generated images into images that do not comprise the damage and into images that comprise the damage and wherein the unmanned aerial vehicle can be controlled to optically detect again those areas of the object whose allocated images comprise the damage with a second resolution that is higher than a first resolution.

19. The system according to claim 18, wherein the unmanned aerial vehicle comprises the at least one capturing unit and wherein the unmanned aerial vehicle can be controlled to fly along the object autonomously and approach those areas of the object autonomously whose allocated images comprise the damage by using the waypoints; and wherein the unmanned aerial vehicle is configured to transmit the plurality of images to a computer; wherein the system comprises a computer; and wherein the computer is configured to evaluate the plurality of images at least partly in an automated manner to classify the generated images into images that do not comprise the damage and into images that comprise the damage and wherein the computer is configured to generate waypoints by using the position and/or location information of the images comprising the damages; and wherein the computer is configured to transmit the waypoints to the unmanned aerial vehicle.

20. A system for detecting a damage of an object, comprising: an unmanned aerial vehicle, e.g., drone, wherein the unmanned aerial vehicle can be controlled to fly along the object and optically detect at least a part of the object by the capturing unit to generate a plurality of images, wherein each image represents an at least partly different area of the object and generate, for each area, an image with a first resolution and the plurality of partial images, each with a second resolution that is higher than the first resolution, wherein the system is configured to evaluate the plurality of images to classify the generated images into images that do not comprise the damage and into images that comprise the damage and provide the partial images of those areas of the object whose allocated images comprise the damage, e.g., for classifying or cataloging the detected damages.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Embodiments of the present invention will be detailed subsequently referring to the appended drawings, in which:

[0057] FIG. 1 is a schematic illustration of an object and a capturing unit with a flight trajectory according to an embodiment of the present invention;

[0058] FIG. 2 is a schematic illustration of a section of FIG. 1 with amended flight trajectory according to an embodiment of the present invention;

[0059] FIG. 3 is a schematic illustration of an image classification according to an embodiment of the present invention;

[0060] FIG. 4 is a schematic side view of a wind power plant having a damage that is detected with the help of embodiments according to the present invention;

[0061] FIG. 5 is a flow diagram of a method for detecting a feature of an object 20 according to an embodiment of the present invention; and

[0062] FIG. 6 is a flow diagram of a further method for detecting a feature of an object according to an embodiment of the present invention.

DETAILED DESCRIPTION OF THE INVENTION

[0063] Before embodiments of the present invention will be discussed in more detailed below based on the drawings, it should be noted that identical, functionally equal or equal elements, objects and/or structures in the different figures are provided with the same or similar reference numbers, such that the description of these elements illustrated in different embodiments is inter-exchangeable or inter-applicable.

[0064] FIG. 1 shows an object 110, an unmanned aerial vehicle 120 with a capturing unit 130 as well as a computer 140. Here, the aerial vehicle 120 is to be considered as merely optional, as possible implementation of a moveable capturing unit, for example in the form of a drone with a camera.

[0065] Starting from a starting point S, the aerial vehicle 120 flies along the object to detect one or several features 110a. The flight trajectory is indicated by waypoints WP1 to WP4. The flight trajectory can originate for example from a preceding trajectory planning. Here, for example waypoint generation can be performed, for example by the computer 140, based on a 3D model of the object, which is provided to the aerial vehicle 120 via a connection 140a. Here, it should be noted that the computer 140 could also be part of the aerial vehicle 120, such that the aerial vehicle plans its own trajectory, for example autonomously. Further, the trajectory can also be manually predetermined by a human pilot, for example due to the lack of a 3D model of the object 110.

[0066] When flying along the object 110, the capturing unit 130 detects the front side 110b of the object 110 or generally a part of the object. Here, the capturing unit 130 generates a plurality of images B1-B4 with a first resolution, wherein each image represents at least partly a different area of the object 110 or the front side 110b of the object. Accordingly, the images B1-B4 can partly overlap as shown in FIG. 1 or, in other words, the same can comprise information on a partly equal image section or a partly equal area of the object. Further, the images might also not overlap. As an example in FIG. 1, each of the waypoints WP1-WP4 is associated to one of the images B1-B4. Simply put, a plurality of waypoints can be determined from where an object is detected, for example captured.

[0067] According to the invention, for detecting the feature 110a, a plurality of methods and method steps are available whose features, as long as not stated otherwise, are inter-exchangeable and can be used together in any combination. Some inventive options will be discussed below with the help of FIGS. 2 and 3. However, this does not represent a limiting list of method steps but merely serves for an improved understanding of inventive concepts and their configurations.

[0068] FIG. 2 shows the images B1-B4 with the same resolution. The images B1-B are classified 140c into images 210 that do not include the feature 110a including images B1, B3, B4 and into images 220 that include the feature 110a, including image B2. The classification is implemented in FIG. 2 as one example with the computer 140 but can be performed by any classification unit. The classification can in particular be implemented by methods of machine learning. The classification can for example be performed by artificial intelligence (AI).

[0069] Again, as shown in FIG. 1, the results of the classification can be communicated 140b during the flight to the aerial vehicle 120 or the capturing unit 130 or can be communicated 140a after flying along the object 110 and the subsequent landing at the starting point S. Again, it should be noted that the computer 140 or also a respective classification unit could be part of the aerial vehicle 120, such that a respective communication 140a, 140b is considered to be optional. Further, the bi-directionality of the communication 140a, 140b shown in FIG. 1 is also optional, any combination of information can be transmitted via the aerial vehicle to the computer or vice versa. Depending on existing hardware, a plurality of possible task distributions (for example with respect to image classification, calculation of waypoints, classification of the feature) can be realized between aerial vehicle 120 and computer 140 and the communication 140b, 140b can be configured accordingly.

[0070] Based on the classification, the capturing unit 130 can optically detect again the area of the object 110, which includes the feature by the image B2 allocated to the area on which the area 110a has been detected with a second resolution that is higher than the first resolution.

[0071] If the classification is performed during flying along the object 110, the aerial vehicle can stay on the waypoint WP2 after generating the image B2 and after detecting the feature on image B2 and can detect the respective area of the object again. However, the aerial vehicle 120 does not have to stay on the waypoint WP2 but, for example, can merely maintain a same or similar distance to the object 110. For increasing the resolution when detecting again, the capturing unit 130 can increase the focal length, for example by adapting a zoom setting or by changing a lens. Thereby, an image B21 with higher resolution can be generated, which is, for example, a partial image of the image B2. For this, apart from the evaluation regarding a classification of images B1-B4, which include the feature 110a, the position of the feature can be evaluated by using position and location data of the aerial vehicle 120 to direct the capturing unit 130 to the feature 110a for generating the image B21. Here, it should be noted that the image B21 does not necessarily have to include a partial area of the image B2. The area of the object detected with image B21 can be selected for example only in dependence on a position of the feature 110a, such that the image section of image B21 can be selected independently of areas of the images B1-B4.

[0072] If, for example, the positon of the feature 110a is not known, an amount of partial images B21-B24 can be generated with the second resolution. Further, also independent of a classification and detection of the feature 110a on the images B1-B4, a plurality of areas of the object or, for example, each area of the object can be detected by a plurality of partial images of the area, each with the second resolution (for example for image B1 the partial images B11-B14, for image B2 the partial images B21-B24 etc.).

[0073] Respective partial images can be transmitted, for example via the communication 140b to an evaluation unit, for example in the form of the computer 140, for example, for classifying or cataloging the detected feature 110a.

[0074] Alternatively, the classification of images B1-B4 can also take place after flying along the waypoints WP1-WP4 and subsequently landing on the starting point S. By the communication 140a, subsequently, based on the classification at allocated positon and/or location information about the feature 110a, a second flight trajectory 150 can be provided to the aerial vehicle 120. For easier understanding, FIG. 1 shows the trajectory 150 as flight from starting point S to waypoint WP2 and back to starting point S. At the waypoint WP2, the above-discussed detecting again of the area of the object takes place. Here, it should again be noted that an area of the object including the feature 110a can be captured with a single image B21 or with a raster of partial images of the image B2, for example including partial images B21-B24 with the second resolution. Increasing the resolution can again be obtained by changing a zoom setting or by changing the lens of the capturing unit 130. These adaptations can be performed, for example, manually during the landing, after flying along the object 110 for the first time and prior to approaching the object 110 via the flight trajectory 140. The flight itself can again take place autonomously or manually.

[0075] For increasing the resolution when detecting the object again, the aerial vehicle 120 or the capturing unit 130 can also reduce a distance to the object d. The same will be discussed below with reference to FIG. 3. FIG. 3 shows a section of FIG. 1 with the object 110, the aerial vehicle 120, the capturing unit 130 and the computer 140. The waypoint WP2 has a first distance d.sub.1 to the object 110. For detecting again, the aerial vehicle 120 can approach the object on the waypoint WP2a, such that the distance is reduced to the distance d.sub.2. Thereby, the object 110 or the area of the object comprising the feature 110a can be captured with a higher resolution, for example without changing a zoom setting or without changing a lens.

[0076] Adapting the trajectory of the aerial vehicle 120 can take place during the flight, as shown optionally in FIG. 3. By communication 140b with the computer 140, after classifying the image B2, the trajectory can be changed during the flight. Here, the capturing unit or the aerial vehicle can communicate the image B2 to the computer 140 and subsequently receive new waypoints WP2a. Alternatively, the aerial vehicle as mentioned above can itself include the computer 140 and perform classification and trajectory adaptation itself. Here, any combination is possible, such that the aerial vehicle or the capturing unit, for example, perform the classification individually, communicate the classification result to an external computer and can receive waypoints in return. A respective trajectory adaptation with distance reduction can also take place during an intermediate landing via the communication 140a of FIG. 1 according to the invention. Further, the flight trajectory can also consist a priori of waypoints with different distances to the object 100, such that, for example, independent of a classification of the images or a detection of the feature 110a, a raster of partial images (B11-B14, B21-B24, etc.) is generated for a plurality of areas of the object with the second resolution by reducing the distance.

[0077] Further embodiments include an AI supported inspection of wind turbines with drones and will be discussed below based on FIG. 4. FIG. 4 shows a wind power plant 400 with a tower 410, a pod 420, a rotor 430, rotors or blades 440, rotor blade or blade tips 450 and rotor blade flanges 460. One of the rotors 440 has a damage 110a. This damage 110a or the defect can be a crack, for example. Regarding defect detection, the damage 110a can be the feature of the wind power plant that is to be detected. Regarding a generation of a model of the wind power plant 400, e.g., by a calibration flight of a drone, the blade tips 450 and/or the rotor blade flanges 460 can be features of the wind power plant 400 that are to be detected. From the detection, for example, via a known position of the aerial vehicle at the time of the detection and the mapping geometry, a 3D model of the wind power plant can be generated. From this 3D model, again waypoints can be generated for autonomous inspection flights. Above that, detected images 210 are shown that do not include the feature and detected images 220 that include the feature in the form of image 220a for the case that the feature is the damage 110a and alternatively or additionally images 220b for the case that the feature are the blade tips 450.

[0078] For the case that the feature is the damage 110a, the partial image 470 is shown as an example for illustration. According to embodiments, areas with images 220a including the feature can be detected again with higher resolution, wherein, however, not the entire previously scanned area of the object has to be scanned again but also merely a partial area of the original image section or the area of the object can be detected. Here, it should noted, in comparison to FIG. 1, generally, in embodiments not necessarily an image Bx has to be completely divided into partial images Bxx by renewed detection with increased resolution. Further, a respective partial image 470 does also not have to be completely within the image with the first resolution from the evaluation of which the position of the feature has been detected. Simply put, in comparison, “raster detection” is indicated for detecting the blade tips 450 for the images 220b.

[0079] The basic concept of defect detection or pattern detection according to embodiments takes place by detecting and subsequently excluding the defect-free areas with the help of AI. In other words, defect detection i.e. for example, detection of the damage 110a, takes place by detecting defect-free areas (image 210), which in this context are referred to as patterns, wherein with this pattern detection for example the defect 110a can be inferred. Subsequently, the areas of the wind power plant 400 or patterns that have been detected by the AI as not defect-free, i.e., defective and therefore not excluded are approached again for generating high-resolution defect images. For approaching again, automatic generation of waypoints can be used. With reference to FIG. 2, areas of the object or the wind power plant 400 associated with the images 210 are excluded and areas associated with images 210 are approached again. With reference to FIG. 1, the high-resolution defect images, can, for example be the image with the second resolution B21 or the plurality of partial images B21-B24. As already discussed above, generation of high-resolution defect images can take place by using zoom lenses and/or a closer approach to the areas that have been detected as not defect-free. Further, an exchange of the lens of the capturing unit is also possible.

[0080] Regarding the wind power plant 400, the AI-supported image/defect detection or pattern detection can be used, for example, in the following tasks or missions. [0081] 1. Calibration flight [0082] 2. Tower inspection [0083] 3. Blade inspection

[0084] According to embodiments, the AI support can be used in several stages. As example, three stages will be explained below, wherein features of the individual stages are inter-exchangeable or combinable in any manner, as far as not indicated otherwise. They are merely to explain the idea regarding the usage of AI with respect of feature detection and are therefore not to be considered as being limiting.

[0085] Stage 1:

[0086] After landing of the drone and transmitting the image data, subsequent further processing of the image data by the AI is performed, which computes on a remote computer (laptop) or runs on the same. The AI generates waypoints for inspection or an inspection flight, for example, after a calibration flight or waypoints for a defect flight, i.e. approaching the wind power plant 400 for detecting the defect 110a or another feature (for example rotor tips 540) with increased resolution after an inspection flight.

[0087] The drone performs the inspection flight autonomously and transmits the images to the remote computer after the landing, for example during a battery change. The image/defect detection or pattern detection takes place on the remote computer. The results, for example waypoints for a subsequent inspection flight, for example based on a generically generated CAD model of the wind power plant after the calibration flight or waypoints for the subsequent defect flight, i.e., approaching the detected defects 110a for example at a short distance and capturing the defects 100a with high resolution are retransmitted to the drone after the calculation.

[0088] Stage 2 Local Intelligence:

[0089] AI runs or computes on an additional computing unit in real time on the drone and controls the inspection flight after calibration or the defect approach during the inspection flight. The additional computing unit can be, for example, an add on GPU (graphics processing unit) Board (additional graphic processor board) or CPU (central processing unit) or specific AI boards. Further, the above-described computer can also be part of the drone and hence provide the hardware for operating the AI.

[0090] The drone is provided with its own local intelligence, for example by the additional computing unit and performs calculations onboard, for example locally on its own drone hardware in real time. Above that, the drone can perform actions, for example calculation of the waypoints for the subsequent inspection flight directly during the calibration flight and further perform the inspection flight, for example, directly afterwards. Defects are detected in real time and instantaneous direct approaching or zooming in at the detected defects and capturing the defect with respective high resolution takes place. Subsequently, the inspection flight is continued up to the next defect.

[0091] By using a 300 DJI drone 1 with P1 full format camera and 50 mm lens or alternatively a zoom lens, the inspection flight can be performed, for example, with a distance to the blade 440 of 8 m and approaching the detected defect again with a distance of 3 m.

[0092] After the inspection flight is terminated, the drone transmits the data, for example all image data of the inspection flight having, for example, a low or the first resolution (for example corresponding to images B1-B4 of FIG. 1) and all generated defect or partial images with, for example, a high or the second resolution (for example one or several of the partial images B11, . . . , B44 of FIG. 1) to a cloud or to an external computer. From the defect images, the defects 110a can be categorized by an operator and the defect protocols can be generated interactively.

[0093] Stage 3 Evaluation:

[0094] Detected and high-resolution defect images are categorized by the AI. The defect images stored in the cloud can be automatically categorized with the help of AI and the defect protocols can be generated automatically. According to the different stages of the AI support, the AI can be used as discussed below in the three above stated tasks.

[0095] 1. Calibration flight: in embodiments, the basic principle of the AI support is the optical recognition and detection of the optical recognition and detection of the blade tips 450 by AI and the calculation of the position of the blade tips 450 as well as the optical recognition and detection of the blade flanges 460 by AI and calculation of the positions, distances and angles of the blade flanges 460.

[0096] Alternatively or additionally, the pitch angles of the blades can be detected and/or calculated. With these values, final calculation or modification of a generic model, for example a CAD model of the wind power plant 400 including the positioning and orientation of the plant as well as the bending of the blades 440 can take place. From these data, the waypoints for the inspection flights can be calculated. This can take place with intermediate landing (remote—stage 1) or in real time without intermediate landing (local intelligence—stage 2).

[0097] 2. Tower inspection: In tower inspection, the usage of AI can be particularly advantageous due to the large amount of images. Based on the above-stated hardware (300 DJI drone with P1 full format camera and 50 mm lens or alternatively zoom lens) for example 400 images can be generated at a distance of 9 m between capturing unit and wind power plant with a resolution of 1.25 pixel/mm at a tower height of 145 m. On the other hand, the possible variations of damages 110a or in other words defect classes are manageable and the defects mostly large-scale, such that the AI can be reached relatively quickly.

[0098] 3. Blade inspection: Inventive methods for blade inspection are similar to the methods for tower inspection, for example at a significantly lower number of images. For obtaining, for example, needed or advantageous resolution of approximately 1.6 pixel/mm for a first inspection, for example for the usage of AI, for example at a blade length of approximately 7 m at distance of the drone or capturing unit of 7 m to the blade approximately 25 images can be generated or needed per side. For obtaining an improved resolution, for example the above-discussed second resolution, for example a resolution of more than 3.5 pixel/mm requested by a reviewer, detected defects should be approached again or immediately at a distance of approximately 3 m to the blade. However, as described above, for obtaining the improvement of the resolution a change of a zoom setting or a change of the used lens could be performed.

[0099] Based on the following table, aspects of embodiments according to the invention will be summarized briefly again and their advantages are illustrated based on numerical examples. The numerical values are based on the above-described 300 DJI drone with P1 full format camera and 50 mm lens. The image sensor has a width of 35.9 mm with 8197 pixel and a height of 24 mm with 5460 pixel.

TABLE-US-00001 P1 P1 P1 P1 P1 P1 Image Full Distance Image Section Pixel/ sensor format mm Width Height Drone Width Height 35.9 mm 24 mm mm mm Pixel 45 Mio 8.197 5.460 Tower e.g. 9 m 6.400 4.320 1.28 38 Images Inspection Height per Tower 145 m Side Total 8 Blades per 45 Degree 304 20 cm Images Overlap Blade e.g. 7.0 m 5.026 3.360 1.63 25 Images/ Inspection Length Blade 70 m Side Total 3 Sides 75 20 cm Images Overlap Defect 3.0 m 2.120 1.420 3.87 Flight Calibra- 25 m 18.000 12.000 0.46 tion

[0100] In the table, the above-described possible tasks or missions of inventive methods regarding a wind power plant are plotted in the form of a tower inspection, a flight inspection and a calibration (calibration flight). Further, an example for an above-described defect flight is entered. For each of these tasks, in the fourth column, a distance of the aerial vehicle or the drone to the wind power plant, in the fifth column, the image width covered by the respective image section with respect to the surface of the wind power plant in mm, in the sixth column, the image height covered by a respective image portion regarding the surface of the wind power plant in mm and in the seventh column, the respective resolution in pixel/mm is entered.

[0101] The inspection of a wind power plant can start, for example with the calibration or the calibration flight. For this, a pilot flies along the wind power plant at a distance of 25 m with the aerial vehicle, for example the drone. Here, the wind power plant is optically detected, wherein an image in an image area corresponds to a width of 18 m and a height of 12 m in reality. Accordingly, images of this optical detection have a resolution of 0.46 pixel/mm. Based on known position and location information of the aerial vehicle connected to the captured images, by recognizing characteristic features of the wind power plant, such as the blade tips, a CAD model of the wind power plant can be generated or a generated model can be modified. Due to the high distance and the low resolution, this step can be performed with little time effort. Detecting the features can be performed in particularly by using methods of machine learning. It should be noted that the distance could also be in an area or interval such that the distance is for example at most 25 m or at most 20 m or is in a range of 20 m to 25. Further, the distance can also be for example 20m.

[0102] Based on the CAD model, subsequently, waypoints for the inspection flights for example for tower inspection and/or blade inspection can be generated. These waypoints can also again be generated by using AI. Both the evaluation as well as the waypoint generation can be performed after landing after the calibration flight or also during the calibration flight by the aerial vehicle itself. In order to be able to detect defects with sufficient accuracy, in the for example subsequently autonomously performed inspection flights, the distance of the aerial vehicle to the wind power plant is reduced. For blade inspection, for example a distance of 7 m can be set, such that a generated image corresponds to a width of approximately 5 m and a height of approximately 3.3 m in reality, which results in a resolution of 1.63 pixel/mm. In 25 images per blade side with 3 sides and an overlap of 20 cm, with an exemplary length of the blades of 70 m, 75 images result. Analogously, during tower inspection, 304 images result at a resolution of 1.28 pixel/mm.

[0103] For evaluating the images of the inspection flights, again, AI can be used. As already mentioned above, the same can already itself categorize the images during the flight and divide them into images showing damages and into images showing no damages. Alternatively, this can also take place after landing on an external computer. The large advantage of methods of machine learning becomes obvious based on the large number of images, which would result in a high time effort when evaluation is performed by persons. By the inventive idea of separately detecting areas of the wind power plant comprising damages again, direct time-intensive and data-intensive generation of images with high resolution during tower and/or blade inspection or for the entire surface and/or blades can be omitted. By the position and location information of the drone that can be connected to the respective images on which damages have been detected, respective parts can be approached again during a defect flight. Alternatively, detecting can take place with the second or higher resolution also during the tower or blade inspection. The distance of the aerial vehicle to the object can be reduced to 3 m or a respective lens can be attached (e.g. during an intermediate landing) or a zoom setting (e.g. during the flight) can be adapted accordingly. Thereby, a resolution of 3.87 pixel/mm can be obtained.

[0104] With such a high resolution, even smallest damages can be detected and categorized. Thus, for example strict legal requirements regarding the safety of plants can be implemented. By reducing the number of detailed images during renewed optical detection with the increased or second resolution, an inventive method is combined with low time and resource effort due to the preselection of the areas of the wind power plant to be considered. Above that, a respective method is very well scalable due to the usage of a plurality of autonomously flying drones. A further option of scaling is the accuracy, wherein both the inspection flights and the defect flights can be improved with even further reduced distances and improved image sensors. Further, also with zoom lenses, more images with higher resolution can be generated.

[0105] In the following FIGS. 5 and 6 the inventive methods will be briefly summarized.

[0106] FIG. 5 shows a flow diagram of a method for detecting a feature of an object according to an embodiment of the present invention. FIG. 5 shows the method steps 510-530 in an order, which is merely exemplary. Accordingly, the method steps can also be used in amended order. Step 510 includes flying along the object and optically detecting at least part of the object at least by a capturing unit 130 with a first resolution to generate a plurality of images, wherein each image represents an at least partly different area of the object. Step 520 includes, for example, automated evaluation of the plurality of images to classify the generated images into images that do not include the feature and into images that include the feature. Step 530 includes optically detecting again those areas of the object whose allocated images include the feature with a second resolution that is higher than a first resolution.

[0107] FIG. 6 shows a flow diagram of a further method for detecting a feature of an object according to an embodiment of the present invention. FIG. 6 shows the method steps 610-630 in an order, which is merely exemplary. The method steps can accordingly be applied in amended order. Step 610 includes flying along the object and optically detecting at least a part of the object by at least one capturing unit to generate a plurality of images, wherein each image represents an at least partly different area of the object and wherein, for one area, an image with the first resolution and a plurality of partial images each, with a second resolution that is higher than the first resolution are generated. Step 620 includes, for example, automated evaluation of the plurality of images to classify the generated images into images that do not include the feature and into images that include the feature. Step 630 includes providing the partial images of those areas of the object whose allocated images include the feature.

[0108] Further, it should be noted that optically detecting according to embodiments could also include detection in the infrared range, for example by means of infrared cameras.

[0109] All listings of materials, environmental influences, electric characteristics and optical characteristics stated herein are to be considered as exemplary and not as limiting.

[0110] Although some aspects have been described in the context of an apparatus, it is obvious that these aspects also represent a description of the corresponding method, such that a block or device of an apparatus also corresponds to a respective method step or a feature of a method step. Analogously, aspects described in the context of a method step also represent a description of a corresponding block or detail or feature of a corresponding apparatus. Some or all of the method steps may be performed by a hardware apparatus (or using a hardware apparatus), such as a microprocessor, a programmable computer or an electronic circuit. In some embodiments, some or several of the most important method steps may be performed by such an apparatus.

[0111] Depending on certain implementation requirements, embodiments of the invention can be implemented in hardware or in software. The implementation can be performed using a digital storage medium, for example a floppy disk, a DVD, a Blu-Ray disc, a CD, an ROM, a PROM, an EPROM, an EEPROM or a FLASH memory, a hard drive or another magnetic or optical memory having electronically readable control signals stored thereon, which cooperate or are capable of cooperating with a programmable computer system such that the respective method is performed. Therefore, the digital storage medium may be computer readable.

[0112] Some embodiments according to the invention include a data carrier comprising electronically readable control signals, which are capable of cooperating with a programmable computer system, such that one of the methods described herein is performed.

[0113] Generally, embodiments of the present invention can be implemented as a computer program product with a program code, the program code being operative for performing one of the methods when the computer program product runs on a computer.

[0114] The program code may, for example, be stored on a machine-readable carrier.

[0115] Other embodiments comprise the computer program for performing one of the methods described herein, wherein the computer program is stored on a machine readable carrier.

[0116] In other words, an embodiment of the inventive method is, therefore, a computer program comprising a program code for performing one of the methods described herein, when the computer program runs on a computer.

[0117] A further embodiment of the inventive method is, therefore, a data carrier (or a digital storage medium or a computer-readable medium) comprising, recorded thereon, the computer program for performing one of the methods described herein. The data carrier, the digital storage medium, or the computer-readable medium are typically tangible or non-volatile.

[0118] A further embodiment of the inventive method is, therefore, a data stream or a sequence of signals representing the computer program for performing one of the methods described herein. The data stream or the sequence of signals may, for example, be configured to be transmitted via a data communication connection, for example via the Internet.

[0119] A further embodiment comprises a processing means, for example a computer, or a programmable logic device, configured to or adapted to perform one of the methods described herein.

[0120] A further embodiment comprises a computer having installed thereon the computer program for performing one of the methods described herein.

[0121] A further embodiment in accordance with the invention includes an apparatus or a system configured to transmit a computer program for performing at least one of the methods described herein to a receiver. The transmission may be electronic or optical, for example. The receiver may be a computer, a mobile device, a memory device or a similar device, for example. The apparatus or the system may include a file server for transmitting the computer program to the receiver, for example.

[0122] In some embodiments, a programmable logic device (for example a field programmable gate array, FPGA) may be used to perform some or all of the functionalities of the methods described herein. In some embodiments, a field programmable gate array may cooperate with a microprocessor in order to perform one of the methods described herein. Generally, the methods are performed by any hardware apparatus. This can be a universally applicable hardware, such as a computer processor (CPU) or hardware specific for the method, such as ASIC.

[0123] The apparatuses described herein may be implemented, for example, by using a hardware apparatus or by using a computer or by using a combination of a hardware apparatus and a computer.

[0124] The apparatuses described herein or any components of the apparatuses described herein may be implemented at least partly in hardware and/or software (computer program).

[0125] The methods described herein may be implemented, for example, by using a hardware apparatus or by using a computer or by using a combination of a hardware apparatus and a computer.

[0126] The methods described herein or any components of the methods described herein may be performed at least partly by hardware and/or by software.

[0127] While this invention has been described in terms of several advantageous embodiments, there are alterations, permutations, and equivalents, which fall within the scope of this invention. It should also be noted that there are many alternative ways of implementing the methods and compositions of the present invention. It is therefore intended that the following appended claims be interpreted as including all such alterations, permutations, and equivalents as fall within the true spirit and scope of the present invention.