TOOL-POSITIONING SYSTEM AND METHOD, ROTARY MILKING PLATFORM, COMPUTER PROGRAM AND NON-VOLATILE DATA CARRIER
20210392842 · 2021-12-23
Inventors
Cpc classification
International classification
A01J5/007
HUMAN NECESSITIES
Abstract
In picking up tools in an automatic milking arrangement, the positions of the tools are determined by registering, via a camera at an origin location, three-dimensional image data representing the tools whose positions are to be determined. Using an algorithm involving matching the image data against reference data, tool candidates are identified in the three-dimensional image data. A respective position is calculated for the tools based on the origin location and data expressing respective distances from the origin location to each of the identified tool candidates. It is presumed that the tools have predefined locations relative to one another. Therefore, any second tool candidate is disregarded, which is detected at a separation distance from a first tool candidate when the separation distance exceeds a first threshold distance in relation to the predefined relative locations.
Claims
1. A system for determining the positions of tools in an automatic milking arrangement and picking up the tools, the system comprising: a camera (130) configured to, from an origin location (P.sub.C), register three-dimensional image data (D.sub.img3D) of at least two tools (141, 142, 143, 144) whose respective positions are to be determined; a control unit (120) configured to: cause the camera (130) to obtain three-dimensional image data (D.sub.img3D) representing the at least two tools, identify tool candidates (T.sub.C1, T.sub.C2, T.sub.C3, T.sub.C4) in the three-dimensional image data (D.sub.img3D) using an algorithm involving matching the three-dimensional image data (D.sub.img3D) against reference data, and calculate a respective position (P.sub.T1, P.sub.T2, P.sub.T3, P.sub.T4) for the at least two tools (141, 142, 143, 144) based on the origin location (P.sub.C(x.sub.C, y.sub.C, z.sub.C)) and data expressing respective distances (d.sub.1(x.sub.1, y.sub.1, z.sub.1)) from the origin location to each of the tool candidates (T.sub.C1, T.sub.C2, T.sub.C3, T.sub.C4) identified, wherein, the at least two tools (141, 142, 143, 144) have predefined locations (Δd) relative to one another, and the control unit (120) is further configured to disregard any second tool candidate (T.sub.C2) detected at a separation distance (d.sub.s) from a first tool candidate (T.sub.C1) when the separation distance (d.sub.s) exceeds a first threshold distance (d.sub.th1) in relation to the predefined relative locations (Δd); and a tool-pickup system configured to use the respective positions (P.sub.T1, P.sub.T2, P.sub.T3, P.sub.T4) for the at least two tools (141, 142, 143, 144), determined by the control unit (120), in picking up at least one of the at least two tools (141, 142, 143 and 144).
2. The system according to claim 1, wherein the at least two tools comprises four tools (141, 142, 143, 144) arranged relative to one another in a predefined pattern, and the control unit (120) is configured to use information about the predefined pattern to confirm and/or disregard at least one of the tool candidates (T.sub.C1, T.sub.C2, T.sub.C3, T.sub.C4).
3. The system according to claim 1, wherein the at least two tools comprises four tools (141, 142, 143, 144) arranged along a line (L) and the control unit (120) is configured to disregard any tool candidate (T.sub.C3) detected at an outlier distance (d.sub.o) exceeding a second threshold distance (d.sub.th2) from an estimated line (L.sub.e) interconnecting at least two other tool candidates (T.sub.C1, T.sub.C2, T.sub.C4) in a set of tool candidates for said tools.
4. The system according to claim 3, wherein the four tools (141, 142, 143, 144) are arranged with an equal distance (Δd) between each neighboring tool of said tools in said line (L), and the control unit (120) is configured to disregard a tool candidate (T.sub.C2) detected at such a position that said tool candidate (T.sub.C2) results in that a difference between a first inter-distance (Δd.sub.1) and a second inter-distance (Δd.sub.2) exceeds a third threshold distance, where the first inter-distance (Δd.sub.1) is an interspace between a primary pair of neighboring tool candidates (T.sub.C1, T.sub.C2) including said tool candidate (T.sub.C2) and a first tool candidate (T.sub.C1) and the second inter-distance is an interspace between a secondary pair of neighboring tool candidates including said tool candidate (T.sub.C2) and a second tool candidate (T.sub.C3).
5. The system according to claim 1, wherein, the respective position (P.sub.T1, P.sub.T2, P.sub.T3, P.sub.T4) for each of the at least two tools (141, 142, 143, 144) is expressed in terms of the space coordinates for a particular point on an object depicted in the three-dimensional image data (D.sub.img3D).
6. The system according to claim 1, wherein, when a processing time after having obtained the three-dimensional image data (D.sub.img3D) in the control unit (120), less than a predefined number of tool candidates have been identified, the control unit (120) is configured to reposition the camera (130) to a new origin location (P.sub.C) from which the at least two tools (141, 142, 143, 144) are visible in the camera (130), cause the camera (130) to obtain updated three-dimensional image data (D.sub.img3D) representing the at least two tools, which updated three-dimensional image data (D.sub.img3D) have been registered from the new origin location (P.sub.C), calculate a respective position (P.sub.T1, P.sub.T2, P.sub.T3, P.sub.T4) for the at least two tools (141, 142, 143, 144) based on the new origin location (P.sub.C) and data expressing respective distances from the new origin location (P.sub.C) to each of the identified tool candidates (T.sub.C1, T.sub.C2, T.sub.C3, T.sub.C4).
7. The system according to claim 1, further comprising a memory unit (160), wherein the control unit (120) is configured to store the respective positions (P.sub.T1, P.sub.T2, P.sub.T3, P.sub.T4) for the at least two tools (141, 142, 143, 144) in the memory unit (160) from which the stored respective positions (P.sub.T1, P.sub.T2, P.sub.T3, P.sub.T4) are retrievable by the tool-pickup system in connection with picking up the at least one of the at least two tools (141, 142, 143, 144) for attachment to an animal.
8. The system according to claim 7, further comprising a tool rack (150), wherein the at least two tools (141, 142, 143, 144) are placed in the tool rack (150), wherein the system further comprises a grip device (115) arranged on a robotic arm (110), and after that the respective positions (P.sub.T1, P.sub.T2, P.sub.T3, P.sub.T4) have been stored in the memory unit (160), the control unit (120) is further configured to: retrieve the stored respective positions (P.sub.T1, P.sub.T2, P.sub.T3, P.sub.T4) from the memory unit (160), and control the robotic arm (110) and the grip device (115) to pick up at least one of the at least two tools (141, 142, 143, 144) from the tool rack (150).
9. The system according to claim 8, wherein the camera (130) is arranged on the robotic arm (110).
10. A rotary milking platform (500) having a plurality of milking stalls (520.sub.i) each of which comprises at least two tools (141, 142, 143, 144), and the rotary milking platform (500) comprises a system for determining the positions of tools according to claim 1, which system is arranged to calculate a respective position (P.sub.T1, P.sub.T2, P.sub.T3, P.sub.T4) for the at least two tools (141, 142, 143, 144) in each of said milking stalls (520.sub.i).
11. A method for determining the positions of tools in an automatic milking arrangement and picking up the tools, the method comprising: registering, via a camera (130) at an origin location (P.sub.C), three-dimensional image data (D.sub.img3D) representing at least two tools (141, 142, 143, 144) whose positions are to be determined; identifying tool candidates (T.sub.C1, T.sub.C2, T.sub.C3, T.sub.C4) in the three-dimensional image data (D.sub.img3D) using an algorithm involving matching the three-dimensional image data (D.sub.img3D) against reference data; calculating a respective position (P.sub.T1, P.sub.T2, P.sub.T3, P.sub.T4) for the at least two tools (141, 142, 143, 144) based on the origin location (P.sub.C(x.sub.C, y.sub.C, z.sub.C)) and data expressing respective distances (d.sub.1(x.sub.1, y.sub.1, z.sub.1)) from the origin location to each of the tool candidates (T.sub.C1, T.sub.C2, T.sub.C3, T.sub.C4) identified, wherein the at least two tools (141, 142, 143, 144) have predefined locations (Δd) relative to one another; disregarding any second tool candidate (T.sub.C2) detected at a separation distance (d.sub.s) from a first tool candidate (T.sub.C1) when the separation distance (d.sub.s) exceeds a first threshold distance (d.sub.th1) in relation to the predefined relative locations (Δd); and having a tool-pickup system use the calculated respective positions (P.sub.T1, P.sub.T2, P.sub.T3, P.sub.T4) for the at least two tools (141, 142, 143, 144) to pick up at least one of the at least two tools (141, 142, 143 and 144).
12. The method according to claim 11, wherein the at least two tools comprise four tools (141, 142, 143, 144) arranged relative to one another in a predefined pattern, and the method comprises: using information about the predefined pattern to confirm and/or disregard at least one of the tool candidates (T.sub.C1, T.sub.C2, T.sub.C3, T.sub.C4).
13. The method according to claim 11, wherein the at least two tools comprise four tools (141, 142, 143, 144) arranged along a line (L) and the method comprises: disregarding any tool candidate (T.sub.C3) detected at an outlier distance (d.sub.o) exceeding a second threshold distance (d.sub.th2) from an estimated line (L.sub.e) interconnecting at least two other tool candidates (T.sub.C1, T.sub.C2, T.sub.C4) in a set of tool candidates for said tools.
14. The method according to claim 13, wherein the four tools (141, 142, 143, 144) are arranged with an equal distance (Δd) between each neighboring tool of said tools in said line (L), and the method comprises: disregarding a tool candidate (T.sub.C2) detected at such a position that said tool candidate (T.sub.C2) results in that a difference between a first inter-distance (Δd.sub.1) and a second inter-distance (Δd.sub.2) exceeds a third threshold distance, where the first inter-distance (Δd.sub.1) is an interspace between a primary pair of neighboring tool candidates (T.sub.C1, T.sub.C2) including said tool candidate (T.sub.C2) and a first tool candidate (T.sub.C1) and the second inter-distance is an interspace between a secondary pair of neighboring tool candidates including said tool candidate (T.sub.C2) and a second tool candidate (T.sub.C3).
15. The method according to claim 11, wherein, the respective position (P.sub.T1, P.sub.T2, P.sub.T3, P.sub.T4) for each of the at least two tools (141, 142, 143, 144) is expressed in terms of the space coordinates for a particular point on an object depicted in the three-dimensional image data (D.sub.img3D).
16. The method according to claim 11, wherein, when a processing time after having obtained the three-dimensional image data (D.sub.img3D), less than a predefined number of tool candidates have been identified, the method comprises: repositioning the camera (130) to a new origin location (P.sub.C) from which the at least two tools (141, 142, 143, 144) are visible in the camera (110), causing the camera (110) to obtain updated three-dimensional image data (D.sub.img3D) representing the at least two tools, which updated three-dimensional image data (D.sub.img3D) have been registered from the new origin location (P.sub.C), and calculating a respective position (P.sub.T1, P.sub.T2, P.sub.T3, P.sub.T4) for the at least two tools (141, 142, 143, 144) based on the new origin location (P.sub.C) and data expressing respective distances from the new origin location (P.sub.C) to each of the identified tool candidates (T.sub.C1, T.sub.C2, T.sub.C3, T.sub.C4).
17. The method according to claim 11, comprising: storing the respective positions (P.sub.T1, P.sub.T2, P.sub.T3, P.sub.T4) for the at least two tools (141, 142, 143, 144) in a memory unit (160) from which the stored respective positions (P.sub.T1, P.sub.T2, P.sub.T3, P.sub.T4) are retrievable by the tool-pickup system in connection with picking up the at least one of the at least two tools (141, 142, 143, 144) for attachment to an animal.
18. The method according to claim 17, wherein the at least two tools (141, 142, 143, 144) are placed in a tool rack (150), the tool-pickup system comprises a grip device (115) arranged on a robotic arm (110), and after that the respective positions (P.sub.T1, P.sub.T2, P.sub.T3, P.sub.T4) have been stored in the memory unit (160), the method further comprises: retrieving the stored respective positions from the memory unit (160), and controlling the robotic arm (110) and the grip device (115) to pick up the at least one of the at least two tools (141, 142, 143, 144) from the tool rack (150).
19. A non-transitory data carrier (126) containing a computer program (127) loadable into a processing unit (125), the computer program (127) comprising software, when executed by the processing unit (125), causes the processing unit to perform, the method according to claim 11.
20. (canceled)
21. The system according to claim 1, further comprising a memory unit (160), wherein the tool-pickup system comprises a grip device (115) arranged on a robotic arm (110), and wherein the control unit (120) is further configured to store the respective positions (P.sub.T1, P.sub.T2, P.sub.T3, P.sub.T4) for the at least two tools (141, 142, 143, 144) in the memory unit (160) from which the stored respective positions (P.sub.T1, P.sub.T2, P.sub.T3, P.sub.T4) are retrievable by the tool-pickup system for use in controlling the robotic arm (110) and grip device (115) to have the grip device (115) pick up the at least one of the at least two tools (141, 142, 143, 144) for attachment to an animal.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
[0022] The invention is now to be explained more closely by means of preferred embodiments, which are disclosed as examples, and with reference to the attached drawings.
[0023]
[0024]
[0025]
[0026]
DETAILED DESCRIPTION
[0027] In
[0028] The system includes a camera 130 and a control unit 120. The camera 130 is configured to register three-dimensional image data D.sub.img3D of the tools 141, 142, 143 and 144 whose respective positions are to be determined. Preferably, the camera 130 is a time-of-flight camera (ToF camera), i.e. a range imaging camera system that resolves distance based on the known speed of light. According to the invention, however, the camera 130 may be any alternative imaging system capable of determining the respective distances to the objects being imaged, for example a 2D camera emitting structured light or a combined light detection and ranging, LIDAR, camera system. Moreover, the three-dimensional image data D.sub.img3D may be dynamic. This means that the three-dimensional image data D.sub.img3D can be represented by a video sequence and/or be built up from multiple still images registered from one or more origin locations P.sub.C in one or more angles.
[0029] Thus, the camera 130 is positioned at an origin location P.sub.C, and may either be arranged on a robotic arm 110, as illustrated in
[0030] The control unit 120 is configured to cause the camera 130 to obtain three-dimensional image data D.sub.img3D representing the tools 141, 142, 143 and 144, and forward the three-dimensional image data D.sub.img3D to the control unit 120. The control unit 120 is then configured to identify tool candidates T.sub.C1, T.sub.C2, T.sub.C3 and T.sub.C4 in the three-dimensional image data D.sub.img3D using an algorithm involving matching the image data D.sub.img3D against reference data. For example, the reference data may comprise one or more characteristic patterns of the tool and/or a typical tool outline. The control unit 120 is configured to calculate a respective position P.sub.T1, P.sub.T2, P.sub.T3 and P.sub.T4 for each of the tools 141, 142, 143 and 144 based on the origin location P.sub.C(x.sub.C, y.sub.C, z.sub.C) and data expressing respective distances from the origin location P.sub.C(x.sub.C, y.sub.C, z.sub.C) to each of the identified tool candidates T.sub.C1, T.sub.C2, T.sub.C3 and T.sub.C4.
[0031] To improve the efficiency of the positioning process, the following strategy is applied according to the invention. Since it is safe to assume that each of the tools 141, 142, 143 and 144 is stored in a dedicated space, such as at a given position in a rack 150, it can be presumed that the tools 141, 142, 143 and 144 have predefined locations relative to one another. In
[0032] Referring now to
[0033] Provided that there are three or more tools, say four tools 141, 142, 143 and 144, these tools may be arranged relative to one another in a predefined pattern, for instance in the corners of a square, along an arc or along a line L as shown in
[0034]
[0035] Moreover, if there is a plurality of tools 141, 142, 143 and 144, the control unit 120 may draw further conclusions based on the predefined pattern in which the tools are arranged relative to one another.
[0036] Here, four tools 141, 142, 143 and 144 are arranged in a line L (see
[0037] Analogously, the predefined pattern in which the tools are organized may also be used by the control unit 120 to confirm a tool candidate. I.e. if, for example, a second tool candidate is found at the expected distance Δd from a first tool candidate, the position for the second tool candidate can be confirmed.
[0038] To facilitate making efficient use of the calculated tool positions [P] when operating the automatic milking arrangement, according to one embodiment of the invention, the system includes a memory unit 160, e.g. a storage medium in the form of a Flash memory or a Read Only Memory (ROM). The control unit 120 is further configured to store the respective positions [P] for the at least two tools 141, 142, 143 and 144 in the memory unit 160. The stored respective positions [P] are retrievable from the memory unit 160 by a tool-pickup system in connection with picking up at least one of the at least two tools 141, 142, 143 and 144, typically for attachment to an animal.
[0039] According to one embodiment of the invention, the at least two tools 141, 142, 143 and 144 are placed in a tool rack 150, as illustrated in
[0040] Preferably, if a robotic arm 110 is included in the system, the camera 130 is arranged on the robotic arm 110. Namely, this highly facilitates implementing the above above-described procedure.
[0041] It is generally advantageous if the control unit 120 and the camera 130 are configured to effect the above-described procedure in an automatic manner by executing a computer program 127. Therefore, the control unit 120 may include a memory unit 126, i.e. non-volatile data carrier, storing the computer program 127, which, in turn, contains software for making processing circuitry in the form of at least one processor in the central control unit 120 execute the above-described actions when the computer program 127 is run on the at least one processor.
[0042]
[0043] Preferably, the rotary milking platform 500 is stepwise rotated pass the system 510 for calculating the respective tool positions for all the milking stalls 520.sub.i on the platform 500 in a consecutive manner.
[0044] In order to sum up, and with reference to the flow diagram in
[0045] In a first step 610, three-dimensional image data are registered via a camera at a known origin location. The three-dimensional image data represent at least two tools of the milking arrangement whose respective positions are to be determined.
[0046] In a subsequent step 620, tool candidates are identified in the three-dimensional image data using an algorithm that involves matching the image data against reference data.
[0047] Thereafter, in step 630, a respective position is calculated for the at least two tools. The position calculations are based on the known origin location of the camera and data expressing respective distances from the origin location to each of the identified tool candidates. The at least two tools have predefined locations relative to one another. Any second tool candidate is disregarded, which is detected at a separation distance from a first tool candidate, if the separation distance exceeds a first threshold distance in relation to the predefined relative locations.
[0048] Subsequently, the procedure ends.
[0049] All of the process steps, as well as any sub-sequence of steps, described with reference to
[0050] The term “comprises/comprising” when used in this specification is taken to specify the presence of stated features, integers, steps or components. However, the term does not preclude the presence or addition of one or more additional features, integers, steps or components or groups thereof.
[0051] The invention is not restricted to the described embodiments in the figures, but may be varied freely within the scope of the claims.