METHOD AND SYSTEM FOR PERFORMING DATA ANALYSIS FOR PLANT PHENOTYPING

Abstract

The invention relates to a method for performing data analysis for plant phenotyping of single plants in a field and a data acquisition and evaluation system for performing data analysis for plant phenotyping of single plants in a field. Further, the invention relates to a mobile platform for use in said method and/or in said data acquisition and evaluation system and a use of the mobile platform for said method and/or said data acquisition and evaluation system. The method comprises the steps of capturing spectral data via a hyperspectral imaging sensor, capturing image data via an image sensor, capturing georeference data via an inertial measurement unit, spatializing the image data to generate georeferenced image data and a digital surface model, spatializing the spectral data, generating georeferenced spectral data based on the spatialized spectral data and the digital surface model and overlaying the georeferenced image data and georeferenced spectral data with field plan information to generate a high-resolution analysis data set.

Claims

1. A method for performing data analysis for plant phenotyping of single plants in a field, comprising the steps of: capturing spectral data via a hyperspectral imaging sensor (1.3), capturing image data via an image sensor (1.2), capturing georeference data via an inertial measurement unit (1.1), spatializing the image data (2.1, 2.2) to generate georeferenced image data (2a) and a digital surface model (2b), spatializing the spectral data (3.1, 3.2), generating georeferenced spectral data (3a) based on the spatialized spectral data and the digital surface model (2b) and overlaying the georeferenced image data and the georeferenced spectral data with field plan information (4a, 4b) to generate a high-resolution analysis data set.

2. The method according to claim 1, wherein the image sensor is a RGB sensor.

3. The method according to claim 1, wherein spatializing the image data comprises assigning spatial coordinates to the image data and preferably spatially correcting the image data.

4. The method according to claim 1, wherein spatializing the spectral data comprises a first step of spatializing the spectral data, which comprises assigning spatial coordinates to spectral data and preferably radiometrically correcting the spectral data and preferably a second step of spatializing the spectral data, which comprises spatially correcting the spectral data.

5. The method according to claim 1, wherein the field plan information comprise field information for defining field locations and field dimensions, in particular field piece information for defining field piece locations and field piece dimensions.

6. The method according to claim 1, wherein overlaying the georeferenced spectral data and the georeferenced image data with the field plan information comprises an assignment of field piece information according to georeference coordinates.

7. The method according to claim 1 comprises the step of capturing additional data via at least one additional sensor, preferably via a thermal sensor and/or an electro-magnetic sensor.

8. The method according to claim 1, wherein generating the digital surface model comprises multiple recording of an individual picture element by capturing the image data and combining said multiple recorded individual picture elements to a three dimensional image.

9. The method according to claim 1, wherein the method comprises the step of using a computer algorithm for phenotyping that preferably identifies direct traits and/or leave diseases and/or insect damages and/or virus infections by symptoms and/or abiotic stress effects.

10. The method according to claim 1, wherein the hyperspectral imaging sensor for capturing spectral data and the image sensor for capturing image data and the inertial measurement unit for capturing georeference data are arranged on a mobile platform, wherein the mobile platform is a ground-based device and/or an aerial device, preferably an autonomous mobile platform.

11. The method according to claim 1, wherein the method comprises pre-processing and/or processing the data on the mobile platform and/or an agricultural station and/or a main station during the operating process and/or in a separate step offline.

12. The method according to claim 1, wherein the captured data and/or the pre-processed data and/or the processed data are transferred from the mobile platform to the main server and/or from the agricultural station to the main server via a wire connection and/or a wireless connection.

13. A data acquisition and evaluation system for performing data analysis for plant phenotyping of single plants in a field, comprising: a hyperspectral imaging sensor for capturing spectral data (1.3), an image sensor for capturing image data (1.2), an inertial measurement unit for capturing georeference data (1.1) and a control unit, which is adapted to: spatialize the image data (2.1, 2.2) to generate georeferenced image data (2a) and a digital surface model (2b), spatialize the spectral data (3.1, 3.2), generate georeferenced spectral data (3a) based on the spatialized spectral data and the digital surface model (2b), overlay the georeferenced image data and georeferenced spectral data with field plan information (4a, 4b) to generate a high-resolution analysis data set.

14. A mobile platform for use in a method according to claim 1, comprising a hyperspectral imaging sensor for capturing spectral data (1.3), an image sensor for capturing image data (1.2) and an inertial measurement unit for capturing georeference data (1.1).

15. A mobile platform for use in a data acquisition and evaluation system according claim 13, comprising a hyperspectral imaging sensor for capturing spectral data (1.3), an image sensor for capturing image data (1.2) and an inertial measurement unit for capturing georeference data (1.1).

16. A method for selecting a plant, said method comprising: a) growing a plant population; b) performing the method of claim 1 for phenotyping the population of plants based on the high-resolution analysis data set; and c) selecting a plant from the population having a desired phenotype.

17. A method for selecting plant individuals in a breeding program, said method comprising: a) growing a plant population of training individuals; b) performing the method of claim 1 for phenotyping the population of training individuals based on the high-resolution analysis data set and generating a phenotype training data set; c) associating the phenotype training data set with a genotype training data set comprising genetic information across the genome of each training individual; d) genotyping a population of breeding individuals; e) selecting breeding pairs from the population of breeding individuals based on plant genotypes using the association training data set to select breeding pairs likely or able to generate offspring with one or more desired traits; f) optionally, crossing the breeding pairs to generate offspring; and growing the offspring with the one or more desired traits.

18. The method of claim 17, wherein said genotypic information for the candidate is obtained by genotyping using SNP markers.

19. The method of claim 17, wherein said genotypic information for the candidate is obtained by analyses of gene expression, metabolite concentration, or protein concentration.

20. The method of claim 17, wherein said breeding individuals are homozygous.

21. The method of claim 17, further comprising a genetically diverse population that includes individuals carrying one or more transgenes or a genetically diverse population that includes individuals with DNA edited with random or targeted mutagenesis.

22. The method of claim 17, wherein said plant population of training individuals and/or the population of breeding individuals is genetically diverse.

23. A method for selecting an inbred plant, the method comprising: a) quantitatively assessing the distribution of two or more traits in a population of inbred plants, wherein assessing the distribution of at least one trait is performed on bases the high-resolution analysis data set generated by the method of claim 1; b) constructing a relationship matrix for each inbred plant parent for the two or more traits of interest; c) applying the relationship matrix in a multivariate mixed model analysis for the population of inbred plants; d) obtaining a predicted value for said inbred plant; and e) selecting one or more inbred plants based on the predicted value.

24. The method of claim 23, wherein the population of inbred plants is separated into male and female lines.

25. The method of claim 23, wherein the traits comprise a plurality of correlated attributes.

26. The method of claim 25, wherein the plurality of correlated attributes comprises grain yield, moisture content, total leaf number and/or biomass.

27. The method of claim 23, further comprising determining the general combining ability and/or the specific combining ability for said plant.

28. The method of claim 23, further comprising calculating a BLUP using the model.

29. The method of claim 23, further comprising calculating the accuracy of prediction for each said predicted value.

30. The method of claim 23, further comprising selecting a hybrid progeny plant based on predicted values obtained from two parent inbred plants.

Description

[0096] Preferred embodiments of the invention shall now be described with reference to the attached drawings, in which

[0097] FIG. 1: shows a schematic flow diagram of an exemplary method for performing data analysis for plant phenotyping of single plants in a field;

[0098] FIG. 2: shows a geotiff recorded by an aerial device which is overlaid with corresponding field plan information;

[0099] FIG. 3: shows merging of captured image data and thermal data;

[0100] FIG. 4: shows an example of measurements of a pathogen infection;

[0101] FIG. 5A: shows merging of spectral data with RGB data for measuring the pathogen infection according to FIG. 4;

[0102] FIG. 5B: shows a spectral comparison of leaf disease, healthy leaf and soil according to FIG. 4 and FIG. 5;

[0103] FIG. 6: shows an example for a single plant analysis; and

[0104] FIG. 7: shows a measurement of leave coverage and/or a biomass.

[0105] In the figures, elements with the same or comparable functions are indicated with the same reference numerals.

[0106] FIG. 1 shows a schematic flow diagram of the method for performing data analysis for plant phenotyping of single plants in a field is shown. The flow diagram describes the processing of data after capturing these data 1a. The processing can be performed on the mobile platform, an agricultural station and/or a main server. In particular, pre-processing which is a part of processing, can be performed on the mobile platform, wherein the remaining part of processing can be performed on the agricultural station and/or the main server, and/or the agricultural station, wherein the remaining part of processing can be performed on the main server.

[0107] The method describes steps of processing after capturing georeference data 1.1 via an inertial measurement unit, image data 1.2 via an image sensor and spectral data 1.3 via a hyperspectral imaging sensor. The first step according to FIG. 1 is spatializing the image data 2.1, 2.2 and spatializing the spectral data 3.1, 3.2, 3.3. Spatializing the image data 2.1, 2.2 comprises assigning spatial coordinates to the image data 2.1 and spatially correcting the image data 2.2 to generate georeferenced image data 2a and a digital surface model 2b by using the georeference data 1.1. Spatializing the spectral data 3.1, 3.2, 3.3 comprises assigning spatial coordinates to spectral data 3.1, a radiometric correction 3.3 and spatial correction 3.2 of the spectral data 1.3. For generating georeferenced spectral data 3a, the spectral data 1.3 where spatialized by using. The next step of the method according to FIG. 1 comprises overlaying the georeferenced image data 2a and the georeferenced spectral data 3a with field plan information 4 to generate a high-resolution analysis data set by assigning of plot information according to geo coordinates 4a, 4b. In a phenotyping analysis 5 the high-resolution analysis data set is analyzed for identifying plant traits. For example, the high-resolution analysis data set can be characterized and plant traits can be determined by means of a database analysis.

[0108] FIG. 2 shows a field sector 20.1 in geotiff format 20 recorded by an aerial device, which is overlaid with the corresponding field plan information 4 for analyzing single plants in this field sector 20.1. The field plan information 4 are mapped out as a shapefile defining the plot locations as well as dimensions and cover plots which are not part of the geotiff. Further, axes 21, 22 of the field plan information 4 indicate the north/south and the east/west position of plots and region, which are covered by the image. The scale of the easting axis 21 and the northing axis 22 is meters of a distance to a reference point line. According to FIG. 2, the field plan information 4 define a field splitting into field pieces 23 which have dimensions to ensure capturing high-resolution data. These field pieces 23 form a grid of the field. Further, the field plan information 4 shows blocks 25 which comprise multiple lines 24 of field pieces 23, wherein each line 24 comprises multiple field pieces 23.

[0109] As can be seen in FIG. 3, segmentation 32 for separating between a plant 33 and a background 34 can be made by merging captured data. During processing the data captured via different sensor units are preferably merged. Hereby, differences in size, scale often originating from using different lenses and/or sensor units, changes in physical position as well as interferences origination from different environmental conditions, e.g. sunlight, clouds, temperature, etc, can be eliminated. FIG. 3 shows a thermography 30.1 and a RGB image 30.2 of a maize plant 33 which are merged 31 and segmented 32. This segmentation 32 separates the maize plant 33 from the background 34 to assign leaves 35 of the plant 33 and preferably to ascertain the quality and/or quantity of infections or drought or osmotic stress.

[0110] With reference to FIGS. 4, 5a and 5b, a measurement of a pathogen infection is made by merging.

[0111] FIG. 4 shows an image of a field 40 after merging captured spectral data and captured RGB data as described above. Due to high ground resolution and known geospatial sensor information it is possible to get information about single plants. The information which are received from one data-capturing-process can be used to setup a time resolved series of the plant and/or trait development during a vegetation period. A visual indication of e.g. different plant traits 41, 42 provides a high-resolution and a less subjective phenotyping analysis. Due to the visual indications, the soil, healthy leaves and leaf diseases can be differentiated. Plant traits as well as the soil often have unique fingerprints in the electromagnetic spectrum. Known as spectral signatures, these fingerprints enable identification of the plant traits of single plants of the field.

[0112] In FIG. 5a the soil 50, healthy leaves 51 and leaf diseases 52 are marked. The detection of plant traits can be achieved by comparing each pixel-spectrum with a database, in which reference spectra of different plant traits are deposed. Spectral data allow a differentiation of pixels by its underlying chemical composition. Plants, part of the plants or other targets can show the same visual color while having completely different chemical components, e.g. a brown soil and a brown necrotic leaf tissue.

[0113] As can be seen in FIG. 5b, a spectral comparison shows that if a similarity of the spectra is high enough the pixel can be classified as leaf disease-pixel. On the contrary, if the similarity of the spectra is not high enough the pixel can be classified as healthy leaf-pixel. Therefore, a x-axis 55 scale is wavelength in nm and a y-axis 56 scale is a normalized intensity. After classification of all pixels of a plant and/or a part of the plant, the pixels of leaf disease and healthy leaf can be used to calculate a ratio describing the amount of infestation.

[0114] FIG. 6 shows an example of a single plant analysis. The method captures contours 60 and midpoints 61 of the biomass distribution 62 of single plants. Therefore, the method provides the possibility to measure the biomass for single plants. Further, a growth rate of single plants can be calculated on basis of time series biomass measurements.

[0115] With reference to FIG. 7, a leaf coverage and/or biomass can be measured by comparing captured and merged data of single plants in one trial plot with a spectral reference scale 70. The figure shows single plants in different stages of development. In comparison to the lower part of the trial plot 72, the upper part of the trial plot 71 comprises a canopy between neighbored plants which has been partially closed already. The canopy is often an important parameter for plants. Canopy closure, which describes that a gap between neighbored plants is closed, is crucial for weed control because weed plants growing between the plants competes often with the plants for nutrients and sunlight. In this way, the canopy closure can often hamper the growth of weed plants significantly.

LIST OF REFERENCE SIGNS

[0116] 1a data of a mobile platform [0117] 1.1 georeference data [0118] 1.2 image data [0119] 1.3 spectral data [0120] 2.1 assigning spatial coordinates to the image data [0121] 2.2 spatially correcting the image data [0122] 2a georeferenced image data [0123] 2b digital surface model [0124] 3.1 assigning spatial coordinates to spectral data [0125] 3.2 spatially correcting the spectral data [0126] 3.3 radiometric correcting the spectral data [0127] 3a georeferenced spectral data [0128] 4 field plan information [0129] 4a, 4b assigning of plot information according to geo coordinates [0130] 5 phenotyping analysis [0131] 20 geotiff [0132] 20.1 field sector [0133] 21 x-axis, easting [m] [0134] 22 y-axis, northing [m] [0135] 23 field piece [0136] 24 line of field pieces 23 [0137] 25 block [0138] 30.1 thermography image [0139] 30.2 RGB image [0140] 31 merging [0141] 32 segmentation [0142] 33 plant/maize plant [0143] 34 background [0144] 35 assigning leaves [0145] 40 field [0146] 41, 42 plant traits [0147] 50 soil [0148] 51 healthy leaves [0149] 52 leaf diseases [0150] 55 x-axis, wavelength in nm [0151] 53 y-axis, normalized intensity [0152] 60 contours [0153] 61 midpoint [0154] 62 biomass distribution [0155] 70 spectral reference scale, number of pixels [0156] 71 the upper part of the trial plot [0157] 72 lower part of the trial plot