TIME-OF-FLIGHT SENSING FOR HORTICULTURE

Abstract

The invention provides a sensing system (1000), e.g. for agricultural application, comprising a radiation generator (100), a sensing apparatus (200), and a control system (300) functionally coupled to the radiation generator (100) and the sensing apparatus (200), wherein the sensing system (1000) has one or more time-of-flight sensing modes of operation, wherein the generator (100) is configured to generate a pulse of radiation (111) in the one or more time-of-flight sensing modes of operation, and wherein the sensing apparatus (200) is configured to sense wavelength dependent spectral intensities of radiation received by the sensing apparatus (200) as a function of time in the one or more time-of-flight sensing modes, to provide a sensing system signal; wherein the sensing system signal is indicative of the wavelength dependent spectral intensity distribution of the received radiation as a function of time in the one or more time-of-flight sensing modes.

Claims

1. A sensing system comprising a radiation generator, a sensing apparatus, and a control system functionally coupled to the radiation generator and the sensing apparatus, wherein the sensing system has one or more time-of-flight sensing modes of operation, wherein the generator is configured to generate a pulse of radiation in the one or more time-of-flight sensing modes of operation, and wherein the sensing apparatus is configured to sense wavelength dependent spectral intensities of radiation received by the sensing apparatus as a function of time in the one or more time-of-flight sensing modes, to provide a sensing system signal; wherein the sensing system signal is indicative of the wavelength dependent spectral intensity distribution of the received radiation as a function of time in the one or more time-of-flight sensing modes; wherein the radiation generator and the sensing apparatus are configured movable relative to each other.

2. The sensing system according to claim 1, wherein the sensing system is functionally coupled to an agricultural device, wherein the agricultural device comprises a lighting device for illuminating a plant or plant part with a light recipe, wherein the control system controls the agricultural device to illuminate the plant or plant part with the light recipe in dependence of the sensing system signal.

3. The sensing system according to claim 1, wherein the generator is configured to generate in one or more of the one or more time-of-flight sensing modes of operation a plurality of pulses of radiation wherein two or more pulses of radiation differ in angle of incidence of the radiation.

4. The sensing system according to claim 1, wherein the generator is configured to generate in one or more of the one or more time-of-flight sensing modes of operation a plurality of pulses of radiation wherein two or more pulses of radiation differ in optical properties, wherein the optical properties are selected from the group consisting of polarization, and spectral intensity distribution.

5. The sensing system according to claim 2, wherein the radiation generator comprises two or more lasers configured to generate radiation having different spectral intensity distributions and wherein the generator is configured to generate in one or more of the one or more time-of-flight sensing modes of operation a plurality of pulses of radiation with the two or more lasers.

6. The sensing system according to claim 2, wherein the radiation generator is configured to generate radiation in one or more of the one or more time-of-flight sensing modes of operation having a wavelength selected from the wavelength ranges of 200-300 nm, 680-720 nm, 920-960 nm, 1080-1120 nm, 1340-1420 nm, and 1850-1890 nm.

7. The sensing system according to claim 1, wherein the sensing system includes one or more controllable sensing parameters, wherein the sensing system has an initial mode of operation wherein a value of the one or more controllable sensing parameters are defined in dependence of one or more of user input information, a sensor signal of a sensor, and radiation received in a preliminary time-of-flight sensing mode of operation, and wherein the sensing system is configured to execute one or more of one or more time-of-flight sensing modes of operation with the defined sensing parameters after executing the initial mode of operation.

8. The sensing system according to claim 7, wherein the controllable sensing parameters are selected from the group consisting of polarization of the radiation, spectral intensity distribution of the radiation, angle of incidence of the radiation, pulse modulation and/or pulse frequency, and polarization filter upstream of a detector of the sensing apparatus.

9. The sensing system according to claim 7, wherein the control system is configured to determine from the initial mode of operation at least two different types of radiation wherein a first type of radiation has a larger penetration depth in an plant object being sensed than a second type of radiation, and to execute the one or more of one or more time-of-flight sensing modes of operation with the at least two different types of radiation.

10. The sensing system according to claim 1, wherein the radiation has a beam cross-section, wherein the sensing apparatus has a field of view cross-section, wherein the radiation generator and the sensing apparatus have a predetermined configuration wherein within a predetermined distance from an entrance window of the sensing apparatus the beam cross-section and the field of view cross-section do not overlap, wherein the predetermined distance is selected from the range of 0-500 cm.

11. (canceled)

12. An agricultural facility comprising the sensing system according to claim 1, wherein one or more of the radiation generator and the sensing apparatus are configured movable, and wherein the control system is configured to control one or more of a position of the radiation generator and a position of the sensing apparatus.

13. The agricultural facility according to claim 12, wherein both the radiation generator and the sensing apparatus are configured movable relative to each other.

14. The agricultural facility according to claim 12, wherein the control system is configured to execute an action in dependence of the sensing system signal, wherein the action is selected from the group consisting of controlling growing conditions of a plant, controlling irradiation of a plant or plant part, controlling harvesting of a plant or a plant part, controlling treatment of a plant, and controlling pruning of a plant; and wherein the agricultural facility is selected from the group consisting a horticulture arrangement, a greenhouse, and an open field.

15. (canceled)

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0112] Embodiments of the invention will now be described, by way of example only, with reference to the accompanying schematic drawings in which corresponding reference symbols indicate corresponding parts, and in which:

[0113] FIGS. 1a-1b schematically depict some embodiments of a sensing system;

[0114] FIGS. 2a-2b schematically depict some aspects;

[0115] FIGS. 3a-3b schematically depict some embodiments; and

[0116] FIGS. 4a-4e schematically depict various further aspects.

[0117] The schematic drawings are not necessarily to scale.

DETAILED DESCRIPTION OF THE EMBODIMENTS

[0118] Possible robotized solutions for inspection and/or handling of plants and fruits may need to be able to correctly detect the plants/fruits. Amongst others, herein an (optimized) time-of-flight solution using active illumination with wavelengths which may be optimal for horticultural applications is proposed. By applying a wavelength tailored to the reflective/absorbing/luminescent properties of the plant or fruit, the accuracy and robustness of the range information may be improved. When applying two or more wavelengths, the difference in reflection of each wavelength can be used to classify the plant part or assess the condition and health of the plant (e.g. dehydration level or presence of diseases). It can be used for evaluation of the fruit status (e.g. ripeness, presence or concentration of certain components) to guarantee the quality of the fruit and automatically determine the best moment of harvesting. Also, a wavelength can be chosen such that it will penetrate through for example leaves and is reflected by a fruit. This enables the detection of fruits even in case of occlusion by leaves. Using an additional wavelength reflected by leaves will provide information how to approach the fruit with a robotized gripper. The provided functionality like seeing through leaves and multi-spectral analysis of plant/fruits is a very difficult, subjective and probably impossible task for (unskilled) workers.

[0119] Amongst others, active illumination for SNR improvement of range measurement may be provided. For instance, an active illumination with a wavelength which may be optimally reflected by the fruit/plant to be inspected may be used. For example, the reflectance of tomatoes appears to differ from leaves in the visible range but also in the SWIR range (1000-1900 nm). For instance, active illumination at 930 nm appears to be poorly reflected by tomatoes resulting in a lower SNR of the range signal, whereas for example using a 750 nm wavelength the SNR may be much better. Also, indirect light reflections scattering could have a negative effect on the range measurement. In case we are interested in range information of only tomatoes, a wavelength around the chlorophyll absorption spectrum of leaves and a relatively high reflectance for tomatoes can be selected (like e.g. 625 nm). The wavelengths could also be chosen such that the impact (disturbance) of sun light is reduced (see right picture for the spectrum).

[0120] The absorption spectra of tomatoes and tomato leaves are partially overlapping and therefore a single wavelength can be used to measure the range. However, it may also be useful to use different wavelength, wherein at least one wavelength is selected from a wavelength range wherein there is small or no overlap. In case of fruits and flowers with low reflectance in the overlapping part of the spectrum of leaves (e.g. blue flowers in the example below) the active wavelength used could be chosen between 450-500 nm.

[0121] Further, amongst others monochromatic/multi-spectral active illumination for spectral measurement with range and shape compensation. In further embodiments, active illumination may be used where the intensity of the active illumination may be optimized for the range/geometry and optical characteristics of the fruit/plant to be inspected. The reflection/absorption/transmission of the active illumination may depend on the distance to the plant, the orientation of the surface orientation for different wave lengths. The active illumination may use the information to adapt the intensity of the active (multi-spectral) illumination source to improve the signal quality.

[0122] Yet further, amongst others multi-spectral active illumination for seeing through leaves may be applied. When observing both fruits and leaves, their difference in spectral response can be exploited to see through leaves. For instance, such wavelength that (partially) penetrates through vegetation and reflects on fruits may be used, while measuring the distance. By having an additional wavelength reflected by leaves, an automated solution can use the measured 3D properties of both the fruits and leaves to automate the picking of fruits.

[0123] In embodiments, multi-spectral active illumination for plant growth and health monitoring may be applied. For instance, when using two or more specific wavelengths, the growth or health status of plants can be retrieved by taking the ratio of the spectral responses. For instance, the maturity of a flower or of a fruit may be determined by monitoring the reflectance within an entire first spectral range as reference while using the reflectance at a subset of wavelengths within (or even outside) such first range. Similarly, diseases can be detected from comparison of multiple responses.

[0124] Further, in embodiments multi-spectral active illumination for penetration measurement may be applied. The penetration of light in plants may depend on the wavelength used. Long wavelength red or near infrared (NIR) light may penetrate deeper in the leaf compared with blue light, which may be due to higher scattering on cell membranes and cell-to-cell interfaces. The measured distance for the different spectral components can be used to determine the penetration depth of the different wavelengths.

[0125] In embodiments, multi-spectral active illumination for fluorescence measurement may be applied. For instance, fluorescence reabsorption within a leaf may depend on the luminescence wavelength. Red fluorescence may have a larger probability of (at least partially) being reabsorbed by chlorophyll within the leaf compared with far-red fluorescence, due to the characteristics of the chlorophyll absorption spectra. Likewise, when a broad band radiation in the red is provided, this may have a larger probability of (at least partially) being reabsorbed by chlorophyll within the leaf compared with a broad band radiation in the far-red. The active illumination can be both used for range sensing and fluorescence measurement.

[0126] Yet further, in embodiments multi-spectral active illumination for scatter and BRDF measurement may be provided. For instance, 3D information may be used to reconstruct the surface and thus incident angle of the active light to measure the characteristics of the light reflected. The light scatter or the bidirectional reflectance distribution function (BRDF) can be used for monitoring plant development/health. The scatter effects could depend on the wavelength used.

[0127] In embodiments, multi-spectral active illumination for polarization measurement may be applied. For instance, a time-of-flight sensor with elements with different polarizations (see FIG. 3B) and non-polarized radiation may be applied. The effect of the plant on the polarization can be used for monitoring plant development (e.g. presence of cannabis flowers or ripening level of cannabis flowers based on e.g. detection and monitoring of amount and size of trichomes or terpene compounds in the trichomes). Another option is to use polarized light sources with sensing elements with(out) different polarization. However, also a combination of polarized radiation and a sensing apparatus which can measure different polarizations may be applied.

[0128] In yet further embodiments, multi-spectral active illumination for internal propagation/scatter measurement may be applied. For instance, the time-of-flight signal may be used to measure the propagation of light within the plant. By partially illuminating the plant with the active illumination the non-illuminated areas can be used as measurement location. One can measure the time for the light to enter the plant, propagate trough the plant (parts), and is next received by the sensor (see FIG. 4c (variant II)). The direct return reflection is measured by the sensing elements observing the areas illuminated by the active illumination, while the indirect reflection (that includes propagation through the plant) is measured via the sensing elements observing non-illuminated parts (see FIG. 4c (variant II)). The propagation of light through the plant also depends on the wavelength, providing additional information Also propagation time depends on internal-plant scatter effects, providing e.g. information of type and number of scatter locations (e.g. cell types, -size and -numbers, osmotic pressure in the cells, hydration level of the tissue, etc.).

[0129] With the aid of monochromatic and/or polychromatic active illumination range information can be extracted, which can be used to adapt/calibrate the spectral information. With the aid of monochromatic and/or polychromatic active illumination shape (3D conformation) information can be extracted, which can be used to adapt/calibrate the spectral information.

[0130] FIGS. 1a-1b schematically depict some embodiments of a sensing system 1000, such as here for agricultural application. Reference 1 refers to a very schematically depicted plant object, such as a plant, a tree, a bush, etc.

[0131] The sensing system comprises a radiation generator 100, a sensing apparatus 200 (functionally coupled to the generator 100), and a control system 300 functionally coupled to the radiation generator 100 and the sensing apparatus 200. Two or more of the radiation generator 100, sensing apparatus 200, and a control system 300 may be integrated in a single device. However, they may also be comprised in two or more different devices, see embodiments I and III, which show three separate devices, and embodiment II wherein the radiation generator 100 and sensing apparatus 200 are included in a single apparatus.

[0132] In embodiments, the radiation generator 100 may be configured to generate in one or more of the one or more time-of-flight sensing modes of operation a plurality of pulses of radiation 111, wherein in specific embodiments two or more pulses of radiation 111 differ in one or more of (a) optical properties, and (b) angle of incidence of the radiation 111 (see further also below).

[0133] The sensing system 1000 has one or more time-of-flight sensing modes of operation, wherein the generator 100 is configured to generate a pulse of radiation 111 in the one or more time-of-flight sensing modes of operation, and wherein the sensing apparatus 200 is configured to sense (wavelength dependent) spectral intensities, such as wavelength dependent spectral intensity distributions, of radiation received by the sensing apparatus 200 as a function of time in the one or more time-of-flight sensing modes, to provide a sensing system signal.

[0134] FIG. 1b schematically depicts embodiment wherein radiation may be measured under different angles (variant I) or may be generated under different angles (variant II); the latter may result in different angles of incidence. Of course, the variants I and II may also be combined.

[0135] The returning radiation, which may be sensed by the sensing apparatus 200, may be reflected by the plant object 1, and/or may be reflected or scattered after being transmitted through at least part of the plant object 1, and/or may be luminescence from the plant object 1 due to irradiation with the radiation 111.

[0136] FIG. 2a, variant I, very schematically depict different transmissions (transmission curves) of two different plant parts, such as e.g. leaves and stem or leaves and fruit. Radiations 1111 and 1112 and λ.sub.1 and λ.sub.2 indicate wavelengths of radiation which will have different impacts on the different plant parts. The first wavelength λ.sub.1 will be absorbed by a first plant part (solid curve) and essentially not by the second plant part (dashed curve); the second wavelength λ.sub.1 will be absorbed by the second plant part (dashed curve) and may essentially not by absorbed by the first plant part (solid curve). Variant II very schematically depicts luminescence under radiation with different wavelengths. Irradiation with λ.sub.a provides a clear luminescence band, whereas irradiation with λ.sub.b there is essentially no luminescence (the weak luminescence observed is also shifted relative to the luminescence under λ.sub.a irradiation). Note that there may also be plant parts that do essentially not luminesce at all.

[0137] FIG. 2b (top) schematically depicts a plurality of pulses of radiation 111. These pulses can all be the same.

[0138] Here, by way of example the pulses are essentially the same in pulse duration and distance between the pulses, but the pulses may differ in the spectral intensity distribution (and/or polarization). This is indicated with and λ.sub.1, λ.sub.2, and λ.sub.n which may indicate one of the afore-mentioned, or even yet further wavelengths differing from λ.sub.1 and λ.sub.2. Alternatively or additionally, pulses may also differ in polarization.

[0139] Further, alternatively or additionally pulses may differ in one or more of pulse duration, pulse intensity (pulse height), pulse repetition rate, etc.

[0140] In general, the pulses are relatively short and have short rise and decay times in view of the ToF detection. Pulses as short as about 1.25 ns full width half maximum (FWHM) may well be possible.

[0141] After the pulse or after each pulse, the radiation from the object may be detected. As indicated above, this may include reflected radiation, transmitted radiation, scattered radiation, luminescence. This radiation may be detected as function of time after the pulse. This sensing may be done for one or more, especially a plurality, of radiation wavelengths, and especially at at least two different times after the pulse.

[0142] FIG. 2b (lower part) schematically depict two embodiments or variants, wherein after each pulse at three different times the radiation is measured. In variant I, a spectral intensity distribution over a wavelength range is sensed; in variant II at two or more different wavelengths, here by way of example three different wavelengths, the intensity is measured. The spectral intensities may vary over time (after the pulse) and may also mutually vary over time (after the pulse). From the spectral intensities, as well as over the time dependence thereof, one or more of 3D conformation of a plant, the type of plant, different plant parts, the stage of growing, the presence of a disease, the health of the plant, etc., may be determined. Hence, based on the sensing system signal, one or more of 3D conformation of a plant, the type of plant, different plant parts, the stage of growing, the presence of a disease, the health of the plant, etc., may be determined.

[0143] A plurality of pulses of two or more different radiations will in general be generated consecutively, with (at least) sensing time in between, as will be known to a person skilled in the art. Two or more pulses may e.g. differ in wavelength and/or polarization.

[0144] FIG. 3a schematically depict some embodiments of the radiation generator 100, wherein the radiation generator comprises at least two sources of radiation, indicated with reference 110.

[0145] Variant I schematically depicts a cross-sectional view of an embodiment of the radiation generator 100, where two different sources of radiation 110, here laser sources 150 of radiation, each generating radiation 151, but having different spectral intensity distributions, such as a blue and a red laser, respectively. The different radiations are indicated with references 151′ and 151″ With optics, the beams can be combined to provide a single beam. The radiation 111 escaping from the radiation generator 100 may be indicated with references 111′ and 111″, respectively. Note that these radiations 111′ and 111″ may essentially be the same as the radiations 151′ and 151″, as the optics used may be chosen to have essentially no impact on the spectral intensity distribution. Hence, in embodiments the radiation generator 100 may comprise two or more lasers 150 configured to generate radiation 111 having different spectral intensity distributions, respectively, and wherein the generator 100 is configured to generate in one or more of the one or more time-of-flight sensing modes of operation a plurality of pulses of radiation 111 with the two or more lasers 150 (consecutively).

[0146] Variant II very schematically depicts a front view of an embodiment of the radiation generator 100, here comprising four different sources of radiation 110. These may be configured to generate the radiation 111, but having different spectral intensity distributions.

[0147] Variant III in FIG. 3a schematically depicts a similar radiation generator 100 as variant II, but now with optical filters 120 downstream of the sources of radiation 110. The optical filters may be used to change the spectral intensity distribution of the respective radiations of the respective sources of radiation 110. Alternatively or additionally, the optical filters may be used to impose a polarization to the respective radiations of the respective sources of radiation 110.

[0148] The sources of radiation 110 may provide in a (ToF) controlling mode radiation 111 simultaneously, especially when the sensing system may also detect different spectral intensities, such as wavelength dependent spectral intensity distributions, and/or polarizations. In an alternative (ToF) controlling mode the sources of radiation 110 may provide radiation 111 sequentially.

[0149] In variant IV, the source of radiation 110 may be configured to provide radiation having different wavelengths, especially broad band radiation, though e.g. a white multi-LED light source of radiation may also be possible. Alternatively or additionally, the source of radiation 110 may be configured to provide radiation having different polarizations (or the radiation is unpolarized). Downstream of the source of radiation an optical wheel with a plurality of optical filters 120 may be configured. The optical filters may be used to select a subset of the wavelengths and/or impose a specific polarization to the radiation 111.

[0150] Note that with sources of radiation 110 which are configured at different positions relative to each other (see variants II and III), it may possible to generate a beam with different optical paths (unlike variant I and IV).

[0151] Hence, as indicated above in embodiments the generator 100 may be configured to generate in one or more of the one or more time-of-flight sensing modes of operation a plurality of pulses of radiation 111 wherein two or more pulses of radiation 111 differ in one or more of (a) optical properties, and (b) angle of incidence of the radiation 111, and wherein the optical properties are selected from the group consisting of i polarization, and ii spectral intensity distribution (of the radiation 111).

[0152] Similar variants as schematically depicted above in relation to the radiation generator 100 may also be relevant for the sensing apparatus 200 as shown in FIG. 3b.

[0153] Variant I schematically depicts an embodiment of the sensing apparatus 200 comprises a plurality, here four, of sensing devices 210. These sensing device 210 may be configured to sense the same type of radiation (but inherently under (slightly) different angles), or may be configured to sense different types of radiation, in terms of spectral intensity distributions and/or polarization.

[0154] Variant II schematically depicts an embodiment of the sensing apparatus 200 with a plurality, here four, of sensing devices 210 which may e.g. be able to sense over a plurality of wavelengths radiation and/or essentially independent of polarization. However, optical filters 215 may be used to selected specific wavelengths and/or specific polarizations. Especially, the optical filters 215 are thus configured upstream of the sensing devices 210.

[0155] In variant III, an embodiments of the sensing apparatus 200 is schematically depicted with e.g. a single sensing device 210 which may e.g. be able to sense over a plurality of wavelengths radiation and/or essentially independent of polarization. However, optical filters 215, comprised by a filter wheel, may be used to selected specific wavelengths and/or specific polarizations. Especially, the optical filters 215 are thus configured upstream of the sensing devices 210.

[0156] Note that in FIG. 3a, variants III and IV, the small squares 110 are behind the plane of drawing and in FIG. 3b, variants II and III, the (hatched) small squares 210 are behind the plane of drawing.

[0157] FIG. 4a schematically depicts some possible stages and variants of the sensing method. Reference UI indicates user input, reference SI indicates a sensor input, and II indicates input via an iteration process. Reference C indicates a computing stage, where on the bases of one or more of user input UI, sensor input SI, and the iteration input II, optimized sensing parameters may be chosen. Then, the sensing stage (S) may start or commence, leading to a sensing system signal SSS. The sensing system signal SSS as such may be used, to derive or obtain information about the 3D conformation of a plant, the type of plant, different plant parts, the stage of growing, the presence of a disease, the health of the plant, etc. The sensing system signal SSS may also be used as input (trigger) of another apparatus. Here, AS may refer to e.g. an agricultural system which may include an apparatus, especially for executing an agricultural action. For instance, the action may be selected from the group consisting of controlling growing conditions of a plant, controlling irradiation of a plant or plant part, controlling harvesting of a plant or a plant part, controlling treatment of a plant, and controlling pruning of a plant

[0158] Hence, in embodiments the sensing system 1000 may include one or more controllable sensing parameters, wherein the sensing system 1000 has an initial mode of operation wherein a value of the one or more controllable sensing parameters are defined in dependence of one or more of (i) user input information, (ii) a sensor signal of a sensor 310 (see also FIG. 4b), and iii radiation received in a preliminary time-of-flight sensing mode of operation (of the sensing system 1000), and wherein the sensing system 1000 is configured to execute one or more of one or more time-of-flight sensing modes of operation with the defined sensing parameters after executing the initial mode of operation.

[0159] As indicated above, the controllable sensing parameters are selected from the group consisting of (i) polarization of the radiation 111, (ii) spectral intensity distribution of the radiation 111, (iii) angle of incidence of the radiation 111, (iv) pulse modulation and/or pulse frequency, and (v) polarization filter 215 upstream of a detector 210 of the sensing apparatus 200.

[0160] In embodiments, the control system 300 may be configured to determine from the initial mode of operation at least two different types of radiation 111 wherein a first type of radiation 1111 has a larger penetration depth in an plant object being sensed than a second type of radiation 1112, and to execute the one or more of one or more time-of-flight sensing modes of operation with the at least two different types of radiation 111 (see e.g. also FIG. 2a (variant II) wherein different optical properties for differ plant parts are schematically depicted).

[0161] FIG. 4b schematically depicts an embodiment of the sensing system 1000 further comprising a sensor 310. The sensor may especially comprise an optical sensor such as a camera. Also a plurality of (different types of) sensors may be applied. Further, FIG. 4b also very schematically depicts an embodiment of an agricultural facility 2000. The agricultural facility 2000, such as a greenhouse, may comprise the sensing system 1000. In specific embodiments, one or more of the radiation generator 100 and the sensing apparatus 200 are configured movable. Here, both may move, or may be moved, along rails. Especially, the control system 300 may be configured to control one or more of (i) a position of the radiation generator 100 and (ii) a position of the sensing apparatus 200. Optionally, also the sensor 310 may be configured movable. Alternatively or additionally, the sensor 310 may be comprised by one or more of the radiation generator 100 and the sensing apparatus 200. Note that the sensor 310 is different from the sensing devices.

[0162] As schematically depicted in FIG. 4b, in embodiments both the radiation generator 100 and the sensing apparatus 200 may be configured movable relative to each other.

[0163] As indicated above, the control system 300 is configured to execute an action in dependence of the sensing system signal, wherein the action is selected from the group consisting of controlling growing conditions of a plant, controlling irradiation of a plant or plant part, controlling harvesting of a plant or a plant part, controlling treatment of a plant, and controlling pruning of a plant; and wherein the agricultural facility 2000 is selected from the group consisting a horticulture arrangement, a greenhouse, and an open field.

[0164] Very schematically, an embodiment of a possible agricultural device 2100 is depicted, such as e.g. an irrigation and nutrient providing (spraying) system. The sensing system 1000 may be functionally coupled to such agricultural device 2100. The agricultural device 2100 is controllable in dependence of the sensing system signal. Of course, also other types of agricultural devices may alternatively or additionally be applied.

[0165] Hence, the invention also provides a plant monitor method, comprising executing a time-of-flight sensing mode of operation, wherein the method comprises subjecting one or more plants to a pulse of radiation 111, sensing radiation received by a sensing apparatus as function of time in the time-of-flight sensing mode of operation, and generating a sensing system signal, and wherein the method comprises executing the time-of-flight sensing mode of operation with the sensing system 1000 as described herein. Such plant monitor method may further comprise executing an action in dependence of the sensing system signal, wherein the action is selected from the group consisting of controlling growing conditions, controlling harvesting, controlling treatment, and controlling pruning.

[0166] FIG. 4c schematically depict an embodiment in variant I wherein the field of view (FoV) of the sensor system 200 overlaps at specific distances with the (beam of) radiation 111 of the radiation generator 100. Here, by way of example a focused beam of radiation 111 is depicted as a diverging sensing range. However, also the sensing apparatus may use optics such that e.g. a focal point of the radiation generator 100 and a focal point of the sensing apparatus 200 may overlap. The radiation 111 has a beam cross-section A1. The sensing apparatus 200 has a field of view cross-section A2 (in one or more of the one or more time-of-flight sensing modes of operation). Hence, at one or more distances these may at least partly overlap, which is the case in variant I. In variant I, distance dl indicates a distance from (an entrance window 205 of) the sensing apparatus 200 where the (beam of) radiation 111 of the radiation generator 100 and the field of view (FoV) of the sensor system 200 overlap. The distance d1 may be a range, below which there is no overlap, and above which, there is also no overlap. FIG. 4c, variant I, indicates the lowest value for d1.

[0167] In variant II the sensing system 1000 is used in such a way that the radiation generator 100 and the sensing apparatus 200 have a predetermined configuration wherein within a predetermined distance d2 from an entrance window 205 of the sensing apparatus 200 the beam cross-section A1 and the field of view cross-section A2 do not overlap, wherein the predetermined distance d2 is selected from the range of 0-500 cm. In such embodiment, also transmission and/or absorption in the plant object 1 of the radiation 111 may be sensed.

[0168] Of course, other embodiments than schematically depicted may be possible.

[0169] FIG. 4d schematically depict some phenomena that may occur upon irradiating a plant object 1 with radiation 111. Further, two or more of these may occur together. Variant I indicates transmission, variant II indicates reflection; variant III indicates scattering, and variant IV indicates luminescence. Note that perpendicular radiation may also lead to reflection and/or scattering. Further, irradiation under an angle may in embodiments also lead to (some) transmission and/or luminescence.

[0170] FIG. 4e schematically depicts two figures explaining some phenomena.

[0171] Assume that a plant object is irradiated with narrow beam radiation 111, indicated also with reference 111a. The radiation measured by the sensing apparatus may be indicated with radiation 111b, which may be lower in intensity due to one or more of absorption (see the absorption curve A) and scattering and/or reflection outside the field of view of the sensing system. Absorption may lead to loss of light (and/or to conversion; see also below).

[0172] Assume that a plant object is irradiated with broad band radiation 111, indicated also with reference 111c. The radiation measured by the sensing apparatus may be indicated with radiation 111d, which may be lower in intensity due to one or more of absorption (see the absorption curve A) and scattering and/or reflection outside the field of view of the sensing system. Absorption may lead to loss of light (or to conversion; see also below). Further, as the absorption is wavelength dependent, the sensed radiation 111d may have another wavelength dependence than the radiation 111c used for irradiation.

[0173] Assume that a plant object is irradiated with narrow beam radiation 111, indicated also with reference 111e in variant II. The radiation measured by the sensing apparatus may be indicated with radiation 111f, which may be lower in intensity due to one or more of absorption (see the absorption curve A) and scattering and/or reflection outside the field of view of the sensing system. Absorption may lead to loss of light and/or to conversion; here, also emission radiation 111g is schematically depicted. Hence, in embodiments irradiation with radiation 111 (here indicated as 111e) may lead to some reflection and/or scattering and/or transmission, and (some) luminescence (indicated with reference 1110.

[0174] Hence, in embodiments the invention may provide an optical range sensor with active illumination optimized to observe plants and fruits to extract 3D and spectral information simultaneously, wherein one or more of a (i) a single or multi-pixel (array/matrix) time-of-flight imager, (b) a laser scanner, and (c) a solid state LIDAR, may be applied. In embodiments, multi-spectral active illumination for time-of-flight ranging may be applied, wherein one or more of the following may be applied: (i) an optical sensor responsive to a wide spectrum in combination with an active light source with one or multiple monochromatic wavelengths that are activated sequentially, and (ii) two or more sensing elements with different optical filters or different spectral sensitivity to have different spectral response in combination with an active light source with a broad spectral range. Hence, also a combination may be applied. In embodiments monochromatic/multi-spectral active illumination using wavelengths matched to the reflective and/or transmissive optical properties of the fruit/plant may be applied, which may enable one or more following functionalities: (i) improvement of the signal quality of the range measurement, (ii) providing spectral information with range information, (iii) seeing through leaves functionality, (iv) classification between fruit/leaves based on spectral reflectance of single or multiple wavelengths, (v) assessment of plant development, plant health and detection of diseases based on spectral reflectance of single or multiple wavelengths, etc. In embodiments, monochromatic/multi-spectral active illumination using wavelengths matched to the reflective optical properties of the fruit/plant with 3D surface reconstruction may be applied, enabling one or more of the following functionalities: (i) scatter/bidirectional reflection distribution function measurement, (ii) reflectance properties measurement, (iii) polarization properties measurement, etc. In embodiments, monochromatic/multi-spectral active illumination using wavelengths matched to the transmissive optical properties of the fruit/plant with 3D surface reconstruction enabling the following functionalities: (i) scattering properties, (ii) penetration depth in material of specific wave lengths, etc. In embodiments, an algorithm may be provided and/or used, which may use the measured reflective and transmissive properties for specific wavelengths and reconstructed shape properties to extract one or more of the following information: (i) health status, (ii) growth/development status, (iii) absence/presence of diseases, (iv) ripeness, etc. Additionally or alternatively, the spectral and shape properties of the plant under inspection can be provided to a plant management system that can be used to create historical overview and trends and optimize the growth cycle and detect anomalies. Additionally or alternatively, the spectral and shape properties of the plant under inspection can be used to autonomously judge the quality of e.g. a fruit and guide a robot/drone in the actuation of picking such qualified fruit through the shielding leaf canopy. Amongst others, the invention may be applied for e.g. one or more of growth monitoring, quality monitoring, crop and fruit evaluation, property evaluation, fruit harvesting, making use of the lighting infrastructure instalment and functionality.

[0175] Range information (or distance information) can be used for example to compensate signal dampening depending on the measured distance. Similarly, the shape information derived from a pixelated or scanning ToF solution can be used to for example take into account the incident angle with respect to the surface of the active illumination.

[0176] The term “plurality” refers to two or more.

[0177] The terms “substantially” or “essentially” herein, and similar terms, will be understood by the person skilled in the art. The terms “substantially” or “essentially” may also include embodiments with “entirely”, “completely”, “all”, etc. Hence, in embodiments the adjective substantially or essentially may also be removed. Where applicable, the term “substantially” or the term “essentially” may also relate to 90% or higher, such as 95% or higher, especially 99% or higher, even more especially 99.5% or higher, including 100%.

[0178] The term “comprise” includes also embodiments wherein the term “comprises” means “consists of”.

[0179] The term “and/or” especially relates to one or more of the items mentioned before and after “and/or”. For instance, a phrase “item 1 and/or item 2” and similar phrases may relate to one or more of item 1 and item 2. The term “comprising” may in an embodiment refer to “consisting of” but may in another embodiment also refer to “containing at least the defined species and optionally one or more other species”.

[0180] Furthermore, the terms first, second, third and the like in the description and in the claims, are used for distinguishing between similar elements and not necessarily for describing a sequential or chronological order. It is to be understood that the terms so used are interchangeable under appropriate circumstances and that the embodiments of the invention described herein are capable of operation in other sequences than described or illustrated herein.

[0181] The devices, apparatus, or systems may herein amongst others be described during operation. As will be clear to the person skilled in the art, the invention is not limited to methods of operation, or devices, apparatus, or systems in operation.

[0182] It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design many alternative embodiments without departing from the scope of the appended claims.

[0183] In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim.

[0184] Use of the verb “to comprise” and its conjugations does not exclude the presence of elements or steps other than those stated in a claim. Unless the context clearly requires otherwise, throughout the description and the claims, the words “comprise”, “comprising”, and the like are to be construed in an inclusive sense as opposed to an exclusive or exhaustive sense; that is to say, in the sense of “including, but not limited to”.

[0185] The article “a” or “an” preceding an element does not exclude the presence of a plurality of such elements.

[0186] The invention may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In a device claim, or an apparatus claim, or a system claim, enumerating several means, several of these means may be embodied by one and the same item of hardware. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.

[0187] The invention also provides a control system that may control the device, apparatus, or system, or that may execute the herein described method or process. Yet further, the invention also provides a computer program product, when running on a computer which is functionally coupled to or comprised by the device, apparatus, or system, controls one or more controllable elements of such device, apparatus, or system.

[0188] The invention further applies to a device, apparatus, or system comprising one or more of the characterizing features described in the description and/or shown in the attached drawings. The invention further pertains to a method or process comprising one or more of the characterizing features described in the description and/or shown in the attached drawings.

[0189] The various aspects discussed in this patent can be combined in order to provide additional advantages. Further, the person skilled in the art will understand that embodiments can be combined, and that also more than two embodiments can be combined. Furthermore, some of the features can form the basis for one or more divisional applications.