METHOD OF CONTROLLING A ROBOTIC HARVESTING DEVICE

Abstract

A method of controlling a robotic harvesting device to pick a target piece of fruit from a plant includes determining a position of the target piece of fruit, determining a position of one or more obstacles relative to the target piece of fruit , and plotting a path for the robotic harvesting device to be moved along from a starting position to a picking position. The path is plotted such that the movement of the robotic harvesting device along the path serves to move at least some of the obstacles relative to the target piece of fruit so that the obstacles do not impede the picking of the target piece of fruit .

Claims

1. A method of controlling a robotic harvesting device to pick a target piece of fruit from a plant, the method comprising: determining a position of the target piece of fruit to be harvested from the plant; determining a position of one or more obstacles relative to the target piece of fruit; and plotting a path for the robotic harvesting device to be moved along from a starting position to a picking position, wherein the path is plotted such that the movement of the robotic harvesting device along the path serves to move at least some of the one or more obstacles relative to the target piece of fruit such that the one or more obstacles do not impede the picking of the target piece of fruit by the robotic harvesting device.

2.-4. (canceled)

5. The method as claimed in claim 1, wherein: said determining of a position of the target piece of fruit and/or a position of the one or more obstacles comprises capturing image data of the target piece of fruit and/or the one or more obstacles to generate an image or representation of the target piece of fruit and its surroundings; the image or representation of the target piece of fruit is divided into a plurality of sub-volumes corresponding to a plurality of locations around the target piece of fruit, the plurality of sub-volumes comprising a plurality of columns and/or a plurality of rows; and the plurality of rows defines one or more of: a sub-volume of the image or representation that is located above the target piece of fruit, a sub-volume of the image or representation that is located below the target piece of fruit and a sub-volume of the image or representation that is located around the target piece of fruit.

6. The method as claimed in claim 5, further comprising determining which of the plurality of sub-volumes contains an obstacle.

7. (canceled)

8. The method as claimed in claim 5, wherein each sub-volume of the plurality of sub-volumes is assigned one sub-vector of a plurality of sub-vectors.

9. The method as claimed in claim 8, wherein each sub-vector of the plurality of sub-vectors is directed from a center of the corresponding sub-volume to a center of the target piece of fruit.

10. The method as claimed in claim 1, wherein plotting the path for the robotic harvesting device to be moved along comprises determining a set of one or more path vectors.

11. The method as claimed in claim 10, wherein the one or more path vectors are determined using the determined position of the one or more obstacles.

12. The method as claimed in claim 11, wherein the directions of each path vector of the one or more path vectors is calculated from one or more of a plurality of subsets of sub-vectors that correspond to one or more of a plurality of subsets of sub-volumes.

13. The method as claimed in claim 12, wherein each subset of sub-volumes is determined using the determined positions of the one or more obstacles within the image or representation.

14. The method as claimed in claim 12, wherein each subset of sub-volumes comprises a plurality of adjacent sub-volumes all having the same characteristic representative of whether they are occupied or non-occupied by obstacles of the one or more obstacles.

15. The method as claimed in claim 14, wherein a subset of sub-volumes comprises a largest group of adjacent sub-volumes that all have the same characteristic representative of whether they are occupied or non-occupied by obstacles of the one or more obstacles.

16. The method as claimed in claim 1, wherein the path for the robotic harvesting device to be moved along comprises a plurality of sub-paths, optionally wherein each of the plurality of sub-paths is plotted substantially within a different region around the target piece of fruit.

17. The method as claimed in claim 16, wherein at least one sub-path of the plurality of sub-paths comprises a single linear movement or an oscillating movement of the robotic harvesting device within a region below the target piece of fruit.

18. The method as claimed in claim 16, wherein at least one sub-path of the plurality of sub-paths comprises a combined upwards and oscillating movement of the robotic harvesting device within a region proximal to the target piece of fruit.

19. The method as claimed in claim 16, wherein at least one sub-path of the plurality of sub-paths comprises a dragging movement of the robotic harvesting device that moves the target piece of fruit from a first position immediately below at least one of the one or more obstacles to a second, higher, position that is not immediately below at least one of the one or more obstacles.

20. (canceled)

21. The method as claimed in claim 1, further comprising identifying whether the one or more obstacles are movable or immovable.

22. The method as claimed in claim 1, further comprising moving the robotic harvesting device along the plotted path from the starting position to the picking position and controlling the robotic harvesting device to pick the target piece of fruit.

23. The method as claimed in claim 1, wherein the starting position is determined by determining the largest opening angle between two obstacles, having no obstacles therebetween, and positioning the starting position at the midpoint of the opening angle.

24. A non-transitory computer readable storage medium storing computer software code which when executing on a data processing system performs a method as claimed in claim 1.

25. An apparatus for picking a target piece of fruit from a plant, the apparatus comprising: a picking device for detaching the target piece of fruit from the plant; a maneuvering mechanism for moving the picking device along a path towards the target piece of fruit; one or more sensors for capturing data representative of positions of the target piece of fruit and one or more obstacles; and processing circuitry operable to: determine a position of the target piece of fruit to be harvested from the plant using the data captured by the one or more sensors; determine a position of the one or more obstacles relative to the target piece of fruit using the data captured by the one or more sensors; and plot the path for the picking device to be moved along from a starting position to a picking position by the maneuvering mechanism, wherein the path is plotted such that the movement of the picking device along the path serves to move at least some of the one or more obstacles relative to the target piece of fruit such that the one or more obstacles do not impede the picking of the target piece of fruit by the picking device.

Description

[0075] Certain preferred embodiments for the invention will now be described, by way of example only, with reference to the accompanying drawings in which:

[0076] FIG. 1 shows an apparatus for picking fruit in accordance with an embodiment of the present invention;

[0077] FIG. 2 shows a schematic of the data transfer within the apparatus shown in FIG. 1;

[0078] FIGS. 3a, 3b and 3c show segmented views of a cluster of strawberries, including a target strawberry to be picked by the apparatus of FIG. 1 in accordance with an embodiment of the present invention;

[0079] FIG. 4 shows a perspective view of a cluster of strawberries in which the precise position of each object has been determined;

[0080] FIGS. 5a and 5b show the apparatus shown in FIG. 1 moving an obstacle in accordance with an embodiment of the present invention;

[0081] FIG. 6 shows a plan view of the cluster of strawberries shown in FIG. 4;

[0082] FIGS. 7a and 7b show plan views of the movement of a device according to a horizontal multi-push operation in accordance with an embodiment of the present invention;

[0083] FIGS. 8a, 8b, 8c and 8d show the apparatus shown in FIG. 1 moving two further obstacles in accordance with an embodiment of the present invention;

[0084] FIGS. 9a, 9b, 9c and 9d show the apparatus shown in FIG. 1 moving a further obstacle in accordance with an embodiment of the present invention; and

[0085] FIG. 10 shows a flowchart illustrating the method steps of an exemplary embodiment of the present invention;

[0086] FIG. 11 shows a front view of the movement of a device according to an upwards multi-push operation in accordance with an embodiment of the present invention.

[0087] Robotic harvesting devices that are moved towards an individual, target piece of fruit in order to perform picking operations are moved along a determined path. The method by which this path is determined affects the efficiency and accuracy of the harvesting process and may help to reduce the risk of damaging both the target piece of fruit and other pieces of fruit on the plant. The operation of a robotic harvesting device in accordance with the present invention will now be described.

[0088] FIG. 1 shows an apparatus 1 for picking a target piece of fruit in accordance with an embodiment of the present invention.

[0089] The apparatus 1 comprises a strawberry picking device 22 mounted on a robotic arm 30 (not shown in FIG. 1). The strawberry picking device 22 comprises a container 28 which is provided with an aperture 25 for receiving a target strawberry 4. The device 22 further comprises a number of locating members 24, hingedly mounted around the aperture 25 such that they may be pivoted to open and close the aperture 25.

[0090] Prior to picking a target (e.g. ripe) strawberry 4, the locating members 24 are pivoted outwards in order to open the aperture 25. As a result of the movement of the device 22 along a path (determined in accordance with an embodiment of the present invention, as will be explained), the device 22 is moved to receive a target strawberry 4 within the aperture 25. Following initiation of the picking process, the locating members 24 are configured to close around the stem 5 of the target strawberry 4.

[0091] A cutting mechanism 26, mounted on an inside surface of one of the locating members 24 is arranged to cut the stem 5 of the target strawberry 4, allowing the target strawberry 4 to be collected in the container 28.

[0092] FIG. 2 shows a schematic of the components of, and the data transfer within, the apparatus 1 shown in FIG. 1. The apparatus 1 comprises a robotic arm 30, comprising a processor 32, a motor mechanism 36 (e.g, of one or more motors) and a sensor 34 (comprising a visible light camera and a depth sensor).

[0093] In alternative arrangements, the sensor 34 may be mounted on the device 22 or on a separate movable platform connected thereto.

[0094] The visible light camera and the depth sensor are configured to capture image data and distance data of a cluster of strawberries. The processor 32 is configured to receive the captured data from the visible light camera and the depth sensor, to identify a target (e.g. ripe) strawberry 4 to pick, and to determine a path for the strawberry picking device 22 to follow, according to an embodiment of the present invention, to enable the strawberry picking device 22 to pick the target strawberry 4. Path instructions, once the path has been determined, are sent from the processor 32 to the motor mechanism 36, which manoeuvres the strawberry picking device 22 accordingly.

[0095] The process of determining a path along which the strawberry picking device 22 is to be moved, in order to pick a target strawberry 4, will now be described with reference to FIGS. 1-11. FIGS. 3a, 3b and 3c show segmented views of a cluster of strawberries 6, including a target strawberry 4 to be picked by the apparatus 1 of FIG. 1, in accordance with an embodiment of the present invention.

[0096] To determine the path along which the strawberry picking device 22 is to be moved, in order to pick the target strawberry 4, the processor 32 is configured to process the captured data from the visible light camera and the depth sensor to produce a 3D image 2 of the target strawberry 4 and its surroundings. FIG. 3a shows an exemplary 3D image 2 of a target strawberry 4 that is to be picked from within a cluster of other strawberries 6.

[0097] Once the target (e.g. ripe) strawberry 4 has been identified, the 3D image 2 is processed by the processor 32 to determine a Region of Interest (ROI) 16, comprising a cuboid volume around the target strawberry 4. The ROI 16 is dimensioned according to user-defined parameters, e.g. such that the ROI 16 encompasses all the strawberries in the cluster 6. The ROI 16 is centred on the target strawberry 4. Objects outside the ROI 16 are ignored.

[0098] The processor 32 is configured to divide the 3D image 2 into four horizontal layers: a top layer 8, an upper-central layer 10, a lower-central layer 12 and a bottom layer 14. FIG. 3b shows a front elevation of the 3D image 2. The top layer 8 extends along the z-axis from the highest point of the target strawberry 4 towards the top of the ROI 16. The upper-central layer 10 extends from the vertical midpoint of the target strawberry 4 to the highest point of the target strawberry 4. The lower-central layer 12 extends from the vertical midpoint of the target strawberry 4 to the lowest point of the target strawberry 4. The bottom layer 14 extends from the lowest point of the target strawberry 4 to the bottom of the ROI 16.

[0099] In alternative embodiments where the precise positions of both the target piece of fruit 4 and the one or more obstacles 6 are known, the one or more obstacles 6 may be marked with bounding boxes 19, which define the outer edges of each obstacle 6 and the target piece of fruit 4, as shown in FIG. 4.

[0100] However, in accordance with the above-described embodiment of the invention where only the general positions (for example, 3D points) of the one or more obstacles are known, FIG. 3c shows a plan view of the ROI 16. Each layer 8, 10, 12, 14 of the ROI 16 is further segmented into nine blocks 18. On each layer, the blocks 18 are arranged in a 3×3 grid that has its centre at the horizontal midpoint of the target strawberry 4 such that the central block C.sub.C encompasses the position of the target strawberry 4 in the xy plane.

[0101] Each block 18 is assigned a vector 20 representative of the direction from the block 18 to the central block C.sub.C. The direction of the vector 20 is determined by the position of the block 18 so that all vectors 20 are directed from the centre of the corresponding block 18 towards the centre of the central block C.sub.C.

[0102] If an obstacle 6 is detected within a block 18, the block 18 is labeled as being occupied. If no obstacle 6 is detected within a block 18, the block 18 is labeled as being unoccupied.

[0103] The picking device 22 is instructed by the processor 32 to operate in four distinct stages. During the first stage, the device 22 moves obstacles 6 within the bottom layer 14 of the ROI 16. During the second stage, the device 22 moves obstacles 6 within the lower-central layer 12 and the upper central layer 10. During the third stage, the device 22 moves the target 4 into a picking position. During the fourth stage, the device 22 moves upwards to pick the target 4.

[0104] FIGS. 5a and 5b show a strawberry picking device 22 operating in accordance with the first stage of operation.

[0105] In FIG. 5a, the device 22 is in (e.g. has been moved to) a starting position close to a target strawberry 4 and an obstacle strawberry 6, which is located in the bottom layer 14 of the ROI 16. Obstacles 6 in other layers of the ROI 16 have been removed from FIGS. 5a and 5b to improve clarity.

[0106] As shown in FIG. 5a, the obstacle 6 is located directly below the target 4. Therefore, if the device 22 were to simply approach the target 4 from below, without moving the obstacle 6 out of its path, the obstacle 6 may be captured by the device 22. This is undesirable, for example, when the obstacle 6 is an unripe strawberry.

[0107] In FIG. 5a, the obstacle 6 is identified as being within the central block C.sub.C of the bottom layer 14 of the ROI 16. The processor 32 subsequently determines the largest group of unoccupied blocks 18 that are adjacent to each other within the bottom layer 14.

[0108] If the number of blocks 18 within the largest group of adjacent unoccupied blocks 18 is equal to or greater than a predetermined threshold (in this exemplary embodiment, four or more blocks 18), the processor 32 determines that a single-push operation of the device 22 is appropriate to move the obstacle(s) out of the way. The direction of the single-push operation for the device 22 is calculated based on the positions of the occupied blocks 18 according to the following equation:

[00001]$\begin{array}{cc}{D}_{\mathrm{single}}=-{.Math.}_i^n{X}_i& \left(1\right)\end{array}$

[0109] In Equation 1, X.sub.i is the vector of the i.sup.th occupied block 18 within the largest group of adjacent occupied blocks 18 and n is the total number of blocks 18 within the largest group of adjacent occupied blocks 18. Thus it will be appreciated that the direction is chosen such that the movement of the device 22 serves to move the obstacles 6 away from the target fruit 4 prior to picking. If only the central block C.sub.C is occupied, as shown in FIGS. 5a and 5b, then D.sub.single=0. In this situation, the direction in which the device 22 must move in order to push the obstacle 6 is instead determined by calculating the shortest path from the current location of the device 22 to the centre of the bottom layer 14, i.e. the centre of the central block C.sub.C.

[0110] In the scenario shown in FIG. 5a, as there is only one obstacle 6 located within the bottom layer 14, the processor 32 determines that a single-push operation is appropriate. FIG. 5a shows the device 22 in a starting position that it has been moved to by the robotic arm 30. The starting position is determined according to the size of the blocks 18 (which is determined from the size of the target fruit 4) and the size of the device 22 such that edge of the device 22 that is nearest the target fruit 4 is coincident with the edge of the ROI 16. In other embodiments, the starting position may be adjusted according to the characteristics of the crop to be harvested.

[0111] Alternatively, the starting position may be determined by calculating the largest angle α of a region around the target 4 that is free of obstacles 6. For example, the starting position may lie a predetermined distance from the target 4 at the midpoint of α. FIG. 6 shows an exemplary plan view of the arrangement shown in FIG. 4, in accordance with an embodiment of the invention in which the precise positions of both the target piece of fruit 4 and the one or more obstacles 6 are known. Bounding boxes 19 marking the positions of each of the obstacles 6 have been determined and the largest angle α of a region around the target 4 that is free of obstacles 6 is subsequently calculated.

[0112] FIG. 5b shows the location of the device 22 once the single-push operation has been completed. During this operation the locating members 24 of the device 22 are moved to come into contact with and push the obstacle 6. As can be seen, the movement of the device 22 has served to relocate the obstacle 6 such that it will no longer impede the upwards picking motion of the device 22 towards the target strawberry 4.

[0113] If the largest group of adjacent unoccupied blocks 18 comprises fewer than the threshold number of blocks 18 (e.g. four blocks 18), the processor 32 determines that a horizontal multi-push operation of the device 22 is appropriate. The overall direction of the horizontal multi-push operation for the device 22 is calculated based on the positions of the unoccupied blocks 18 according to the following equation:

[00002]$\begin{array}{cc}{D}_{\mathrm{hori\_multi}}=\left({.Math.}_j^m{X}_j\right)& \left(2\right)\end{array}$

[0114] In Equation 2, X.sub.j is the vector of the j.sup.th unoccupied block 18 within the largest group of adjacent unoccupied blocks 18 and m is the total number of blocks 18 within the largest group of adjacent unoccupied blocks 18.

[0115] During a horizontal multi-push operation, the device 22 moves according to a zig-zag motion in the xy plane, wherein the resultant vector of the zig-zag motion is equal to D.sub.hori_multi and the amplitude and frequency of the zig-zag motion are determined according to the specific picking scenario. For example, the values may be determined according to the peduncle length or the damping ratio of the fruit. Such data may be collected and analysed in order to determine the most effective zig-zag motion for different scenarios.

[0116] FIGS. 7a and 7b show plan views of the bottom layer 14 of the ROI 16, segmented into blocks 18 as described above with reference to FIG. 3c, and illustrate the movement of the device 22 according to the horizontal multi-push operation described above.

[0117] In FIG. 7a, the position of the target fruit 4 in the xy plane is denoted by a dotted circle in block C.sub.C. Blocks 18 containing obstacles 6 are labeled with vectors 20 with filled lines. As can be seen, FIGS. 7a and 7b depict a different arrangement to that shown in FIGS. 5a and 5b. In this arrangement, obstacles 6 are located in all of the blocks 18 apart from C.sub.F. Consequently, the processor 32 determines that a horizontal multi-push operation appropriate.

[0118] FIG. 7b shows the path 21 of the device 22 following the horizontal multi-push movement. The positions of the device 22 in the xy plane at various points are denoted by dotted circles. As can be seen, the device 22 moves in a zig-zag motion from outside the ROI 16 (at the bottom of FIG. 7b) to the central block C.sub.C, at which point it is located directly below the target fruit 4. This movement causes the device 22 to push aside obstacles 6 within the bottom layer 14 so that the obstacles 6 are no longer in a position to impede the subsequent picking process.

[0119] In a scenario where all of the blocks 18, excluding the central block C.sub.C, are occupied, resulting in a value of D.sub.hori_multi=0, the direction to move the device 22 is instead determined by calculating the shortest path from the current location of the device 22 to the centre of the bottom layer 14, i.e. the centre of the central block C.sub.C. The movement of the device 22 in this direction is performed according to the zig-zag motion described above.

[0120] FIGS. 8a-d show a strawberry picking device 22 operating in accordance with the second stage of operation.

[0121] FIGS. 8a-d show the strawberry picking device 22 shown in FIGS. 5a and 5b, having completed the bottom layer 14 operations discussed above. Thus, in FIG. 8a, the device 22 is in the position shown in FIG. 5b. However, to improve clarity, only obstacles 6 within the lower-central layer 12 and the upper-central layer 10 are shown.

[0122] During the second stage of operation, the device 22 uses an upwards multi-push operation.

[0123] The upwards multi-push operation comprises the movement of the device 22 in a principally vertical direction towards the target fruit 4 and a side-to-side movement to clear obstacles 6.

[0124] In some embodiments, such as when the precise coordinates of the one or more obstacles 6 are known (for example, as shown in FIG. 4), the device 22 may be moved towards each obstacle 6 within the central layers 12, 10 in turn, from the lowest obstacle 6 to the highest obstacle 6, finishing at the centre of the central block C.sub.C of the upper-central layer 10. In this case, the frequency and amplitude of the upwards multi-push operation are determined by the precise locations of the obstacles 6. A movement of the device 22 in accordance with this embodiment is shown in FIG. 11, which shows the path 41 of the device 22. With this movement, the device 22 moves between each of the obstacles 6 whilst simultaneously moving upwards to enclose the target fruit 4.

[0125] If only the general location of the one or more obstacles 6 is known, the processor 32 determines the largest group of adjacent occupied blocks 18 within the lower-central layer 12 and upper-central layer 10. The initial direction of the upwards multi-push operation in the xy plane is calculated based on the positions of the occupied blocks 18 according the following equation:

[00003]$\begin{array}{cc}{D}_{\mathrm{up\_multi}}=-{.Math.}_i^n{X}_i& \left(3\right)\end{array}$

[0126] In Equation 3, X.sub.i is the vector of the i.sup.th occupied block 18 within the largest group of adjacent occupied blocks 18 and n is the total number of blocks 18 within the largest group of adjacent occupied blocks 18.

[0127] The device 22 moves in the initial direction D.sub.up_multi whilst moving vertically in the z direction towards the target 4. FIG. 8b shows the position of the device 22 at this intermediate point. After a first predetermined time, the device 22 continues to move vertically, but in the opposite direction −D.sub.up_multi in the xy plane. FIG. 8c shows the position of the device 22 at this later point. After a second predetermined time, the device 22 continues to move vertically but in the initial direction D.sub.up_multi in the xy plane. This zig-zag motion is repeated until the target 4 is located within the locating members 24 of the device 22. This position of the device 22 is shown in FIG. 8d.

[0128] The first predetermined time and the second predetermined time (which determine the frequency and amplitude of the zig-zag motion) are determined according to the specific picking scenario. For example, the values may be determined according to the peduncle length or the damping ratio of the fruit. Such data may be collected and analysed in order to determine the most effective zig-zag motion for different scenarios.

[0129] FIGS. 9a-d show a strawberry picking device 22 operating in accordance with the third and fourth stage of operation.

[0130] FIGS. 9a-d show the strawberry picking device 22 shown in FIGS. 8a-d, having completed the lower-central layer 12 and upper-central layer 10 operations discussed above. Thus, in FIG. 9a, the device 22 is in the position shown in FIG. 8d. However, to improve clarity, only obstacles in the upper layer 8 are shown.

[0131] During the third stage of the method, the device 22 first determines whether there are obstacles in the central block C.sub.C of the upper layer 8. If the central block C.sub.C of the upper layer 8 is unoccupied, the device 22 may proceed directly to the fourth stage of the method and move upwards to pick the target strawberry 4. However, if the central block C.sub.C of the upper layer 8 is occupied by an obstacle 6, as shown in FIG. 9a, the device 22 may risk swallowing the obstacle 6 were it to be moved upwards to capture the target strawberry 4. To avoid this, the device 22 uses a drag operation to move the target 4 further away from the obstacle 6.

[0132] The processor 32 determines the xy direction of the drag operation by identifying the region of the upper layer 8 that contains the maximum number of connected (adjacent) unoccupied blocks 18. Thus, the xy direction of the drag operation is given by the following equation:

[00004]$\begin{array}{cc}{D}_{\mathrm{drag}}=-{.Math.}_j^m{X}_j& \left(4\right)\end{array}$

[0133] In Equation 4, X.sub.j is the vector of the j.sup.th unoccupied block 18 within the largest group of adjacent unoccupied blocks 18 and m is the total number of blocks 18 within the largest group of adjacent unoccupied blocks 18.

[0134] The device 22 moves from the position shown in FIG. 9a in the xy direction D.sub.drag combined with an upwards movement to the position shown in FIG. 9b. As can be seen, this movement causes the target strawberry 4 to be moved into a region of the ROI 16 that is unoccupied. As shown in FIG. 9c, the device 22 may subsequently move upwards further and horizontally back to the central position of the ROI 16, thus pushing the obstacle 6 out of the central block C.sub.C of the upper layer 8 of the ROI 16.

[0135] FIG. 9d shows the device 22 during the fourth stage of operation, in which the device 22 moves upwards to fully capture the target strawberry 4. In this position, the device 22 may begin the cutting operation to detach the target strawberry from its stem 5.

[0136] FIG. 10 shows a flowchart that illustrates the steps of an exemplary path-planning process in which only the general positions of the one or more obstacles 6 may be determined, in accordance an embodiment of the present invention. During the first step 100 of the method, a 3D image 2 of the target fruit 4 and its surroundings is obtained, an ROI 16 is determined and subsequently segmented into layers 8, 10, 12, 14 and blocks 18, as described above with reference to FIGS. 3a-c. The identified target fruit 4 is positioned at the centre of the ROI 16.

[0137] In step 110, if one or more obstacles 6 are identified in the bottom layer 14 of the ROI 16, the method proceeds with the first stage of the process, as described above with reference to FIGS. 5a and 5b.

[0138] During the first stage of the process, the largest group of adjacent unoccupied blocks 18 within the bottom layer 14 is determined (step 112). If the number of blocks 18 within this group is greater than or equal to the threshold (e.g. four) (step 113), the largest group of adjacent occupied blocks 18 within the bottom layer 14 is determined (step 114). The vectors 20 corresponding to the blocks 18 within this group are subsequently used to calculate the direction of a single-push operation and the starting position (step 116), as discussed above.

[0139] If the number of blocks 18 within the largest group of adjacent unoccupied blocks 18 within the bottom layer 14 is less than the threshold (e.g. four) (step 113), the vectors 20 corresponding to the blocks 18 within this group are subsequently used to calculate the direction, frequency and amplitude of a horizontal multi-push operation and the starting position (step 115), as discussed above.

[0140] If the calculation of the single-push operation or the horizontal-push operation results in a zero vector (steps 117, 119 respectively), or if no obstacles are located within the bottom layer (step 110), the path is instead determined by calculating the shortest path from the current location of the device 22 to the centre of the central block C.sub.C of the ROI 16 (step 118).

[0141] In step 120, if one or more obstacles 6 are identified in the lower-central layer 12 or the upper-central layer 10 of the ROI 16, the operation proceeds with the second stage of the process, as described above with reference to FIGS. 8a-d.

[0142] During the second stage of the process, the largest group of adjacent occupied blocks 18 within the lower-central 12 and upper-central 10 layers is determined (step 122). The vectors 20 corresponding to the blocks 18 within this group are subsequently used to calculate the direction and first and second time periods for an upwards multi-push operation (step 124), as discussed above.

[0143] If no obstacles 6 are identified in the lower-central layer 12 or the upper-central layer 10 of the ROI 16 (step 120), the second stage of the process is omitted.

[0144] In step 130, if one or more obstacles 6 are identified in the central block C.sub.C of the upper layer 8 of the ROI 16, the operation proceeds with the third stage of the process, as described above with reference to FIGS. 9a-c.

[0145] During the third stage of the process, the largest group of adjacent unoccupied blocks 18 within the upper layer 8 is determined (step 132). The vectors 20 corresponding to the blocks 18 within this group are subsequently used to calculate the direction of a drag operation (step 134), as discussed above.

[0146] If no obstacles 6 are identified in the upper layer 8 of the ROI 16 (step 130), the third stage of the process is omitted.

[0147] In step 140, an upwards movement of the device 22 is determined so that the target fruit 4 may be fully located within the locating members 24 and the cutting operation can be performed by separating the target fruit 4 from the stem 5. FIG. 9d shows the position of the device 22 having completed this movement.

[0148] In step 150, the path-planning algorithm is concluded and the device 22 may be moved along the determined path. Although, in this embodiment, the path is determined before the device 22 beings to move, in alternative embodiments, the path-planning process and the movement of the device 22 may be conducted simultaneously. This may be useful in a closed-loop vision guided manipulation system, in which the positions of the obstacles 6 and the position of the target 4 are updated continuously, thus allowing the parameters of the movement operations to be continuously recalculated to improve accuracy.

[0149] It can be seen from the above that, in at least preferred embodiments, the invention provides a method for determining a path for a robotic harvesting device, based on the position of obstacles, that, when traveled, serves to move said obstacles out of the way of the robotic harvesting device. By considering the positions of individual obstacles and determining a path according to how they may be repositioned to avoid accidental damage and/or harvesting, the accuracy and efficiency of the robotic harvesting process may be significantly improved.

[0150] Although the embodiments illustrated in FIGS. 1-11 have been shown in use with a strawberry picking device in particular, those skilled in the art will recognise that the method of the present invention may be used with a variety of different crops and/or a variety of different harvesting devices.