Teat treatment method and apparatus

Abstract

Automatic teat treatment method and apparatus for treatment operations on teats of an animal using a robot apparatus. The method includes initiating automatic detection of spatial teat positions and registering the spatial positions of the detected teats; and deriving a dedicated treating action effective to move the treating device through a dedicated path defined by the registered spatial teat positions and carrying out treatment by moving the treating device through the dedicated path. When not all of the spatial positions are detected, then the method registers the spatial position of each detected teat and derives an adapted treating action effective to move the treating device through an adapted path defined by the registered spatial teat positions and by approximated spatial locations. When no teat spatial positions have been detected, the method derives a default treating action effective to move the treating device through a default treating path.

Claims

1. An automatic teat treatment method for carrying out a treatment operation on the teats of an individual animal, said method being performed using a robot apparatus with a control system and an arm, the arm of the robot being configured to carry a teat treating device, said method comprising the steps of: using the control system, automatically establishing a start time t0 for said treatment operation; establishing a presence of an animal at a treatment location; using the robot, initiating automatic detection of spatial teat positions of all said teats of said individual animal at said treatment location; (i) when all said spatial positions of all said teats have been detected before a predefined time td after said start time t0 has passed, then registering the spatial positions of all of the detected teats before said predefined time td has passed, and deriving, by using of said control system, a dedicated treating action for said treating device, effective to move said treating device through a dedicated path defined by said registered spatial teat positions of said individual animal, and initiating and carrying out said dedicated treating action of all said teats by movement of said treating device through said dedicated path and thereafter ending said treatment operation; (ii) when said predefined time td has passed after at least one said spatial positions and before all said spatial positions of all the teats have been detected, then immediately halting any further detection of said spatial teat positions and registering the spatial position of each detected teat, and deriving, by using said control system, an adapted treating action for said teat treating device and said individual animal, effective to move said treating device through an adapted path defined by said registered spatial teat positions, said adapted path being additionally defined, for undetected teat positions, by approximated spatial locations, and initiating and carrying out said adapted treating action of all said teats by movement of said treating device through said adapted path and thereafter ending said treatment operation; and (iii) when said predefined time td has passed before any said spatial position of the teats has been detected, then immediately halting any further detection of said spatial teat positions and deriving a default treating action for said treating device and said individual animal, effective to move said treating device through a default treating path, and initiating and carrying out said default treating action of all said teats by movement of said treating device through said default treating path and thereafter ending said treatment operation, wherein said robot apparatus is a pre-treatment robot apparatus or a post-treatment robot apparatus, said treatment operation is a pre-treatment operation or a post-treatment operation, and said treatment location is a pre-treatment location or a post-treatment location.

2. The automatic teat-treatment method according to claim 1, wherein said approximated spatial locations are computed by said control system at or immediately after said predefined time td is passed.

3. The automatic teat-treatment method according to claim 1, wherein said approximated spatial locations are computed by estimating a respective position of all undetected teats using data comprising the detected spatial positions of the detected teats.

4. The automatic teat-treatment method according to any claim 1, wherein said approximated spatial locations are determined by normalizing acquired spatial position data from an imaging apparatus and using look-up tables comprising normalized teat position data stored in a memory associated with said control system.

5. The automatic teat-treatment method according to any claim 1, wherein, for one said undetected teat, said approximated spatial location is derived by assigning said one undetected teat to a position corresponding to a mirror image position of an opposite teat.

6. The automatic teat treatment method according to claim 1, wherein said method further includes, prior to said step of initiating automatic detection of said spatial teat positions, a further step of determining a spatial reference point R in relation to said animal at said treatment location.

7. The automatic teat-treatment method according to claim 6, wherein said step of determining a spatial reference point R in relation to said animal at said treatment location further includes the steps of detecting, using an imaging apparatus and an image processing apparatus, a spatial position of at least one hind leg of the animal and of an udder of the animal and deriving therefrom said reference point R.

8. The automatic teat-treatment method according to claim 6, further comprising moving said imaging apparatus to said reference point R.

9. The automatic teat-treatment method according to claim 1, wherein said spatial positions of said teats are detected, derived, or approximated without reference to historic data for the individual animal.

10. The automatic teat-treatment method according to any claim 1, wherein said method is carried out using said robot apparatus in association with a rotary platform.

11. The automatic teat treatment method according to claim 1, wherein said robot apparatus is a pre-treatment robot apparatus, said treatment operation is a pre-treatment operation, and said treatment location is a pre-treatment location.

12. The automatic teat treatment method according to claim 1, wherein said robot apparatus is a post-treatment robot apparatus, said treatment operation is a post-treatment operation, and said treatment location is a post-treatment location.

13. An automatic teat treatment apparatus for carrying out a treatment operation on teats of an individual animal having plural teats, said apparatus comprising: a robot with a robot arm and an associated control system, said robot arm configured for carrying a teat-treating device; an imaging apparatus; and an image processing apparatus, wherein, said control system, in operation, automatically establishes a start time t0 for said treatment operation, and establishes a presence of the animal at a treatment location, and controls the steps of: said imaging apparatus and said image processing apparatus initiating automatic detection of spatial teat positions of all said teats of said animal at said treatment location; determining, during said detection of said spatial teat positions, whether a predefined time td after said start time t0 has passed, and wherein (i) when all said spatial positions of all said teats have been detected before a predefined time td after said start time t0 has passed, then registering the spatial positions of all of the detected teats before said predefined time td has passed, and deriving, by using of said control system, a dedicated treating action for said treating device, effective to move said treating device through a dedicated path defined by said registered spatial teat positions of said individual animal, and initiating and carrying out said dedicated treating action of all said teats by movement of said treating device through said dedicated path and thereafter ending said treatment operation, (ii) when said predefined time td has passed after at least one said spatial positions and before all said spatial positions of all the teats have been detected, then immediately halting any further detection of said spatial teat positions and registering the spatial position of each detected teat, and deriving, by using said control system, an adapted treating action for said teat treating device and said individual animal, effective to move said treating device through an adapted path defined by said registered spatial teat positions, said adapted path being additionally defined, for undetected teat positions, by approximated spatial locations, and initiating and carrying out said adapted treating action of all said teats by movement of said treating device through said adapted path and thereafter ending said treatment operation, and (iii) when said predefined time td has passed before any said spatial position of the teats has been detected, then immediately halting any further detection of said spatial teat positions and deriving a default treating action for said treating device and said individual animal, effective to move said treating device through a default treating path, and initiating and carrying out said default treating action of all said teats by movement of said treating device through said default treating path and thereafter ending said treatment operation, wherein said robot is a pre-treatment robot or a post-treatment robot, said treatment operation is a pre-treatment operation or a post-treatment operation, and said treatment location is a pre-treatment location or a post-treatment location.

14. The automatic teat-treatment apparatus according to claim 13, wherein said robot is associated with a rotary platform and detects passage of successive animal stalls on the rotary platform.

15. The automatic teat-treatment apparatus according to claim 13, wherein said robot is a stand-alone robot.

16. The automatic teat-treatment apparatus according to any claim 13, wherein said imaging apparatus is provided on said robot arm.

17. The automatic teat-treatment apparatus according to claim 13, wherein said robot is a pre-treatment robot, said treatment operation is a pre-treatment operation, and said treatment location is a pre-treatment location.

18. The automatic teat-treatment apparatus according to claim 13, wherein said robot is a post-treatment robot, said treatment operation is a post-treatment operation, and said treatment location is a post-treatment location.

Description

(1) Additional aspects of the invention will be described with reference to illustrations which show non-limiting examples of certain features and embodiments.

(2) FIG. 1 shows a flowchart setting out some method steps according to one implementation of the invention

(3) FIG. 2 shows a schematic illustration of a view of a moving animal platform for illustrating a possible implementation of the invention

(4) FIG. 3 shows a schematic illustration of a robot capable of implementing aspects of the present invention

(5) FIG. 4 shows a flowchart setting out some method steps according to a further possible implementation of the invention

(6) In FIG. 3, there is shown, schematically, a rotary platform 18 although the invention is not limited to embodiments at a platform and may include embodiments at a conventional fixed milking stall (not shown). The platform is shown in the present example for illustrative purposes only. A teat-treatment apparatus 1 is shown schematically by way of example in the form of an articulated robot having a fixed, stationary base 9 and movable arms 10. The basic robot construction may be of a known type and may comprise its own movement control system module 8. Associated with the robot is a control system 5 for the teat treatment apparatus and connected by a cable 6 or wirelessly to the robot. This control system 5 may be directly or indirectly associated with other control elements such as milking installation control elements (not shown) or sensors such as a platform movement sensor, illustrated in FIG. 3 in the form of an encoder 20 associated with an encoder wheel 21. Also the encoder is connected to the robot or to the control system 5 by means of a cable 22 or other wireless connector. Other sensors may be provided such as other physical sensors for detecting animal stalls or animals at the vicinity of the platform 18. The direction of movement of the encoder wheel 21 and of the platform 18 are indicated by dotted arrows in FIG. 3. The working area of the robot (i.e. its working range) may encompass an area corresponding to a sector of the platform 18 in the robot's vicinity, which may be considered to be a pre- or post-treatment location for the purposes of the present disclosure and which may by way of example extend across the size equivalent to approximately 2.5 milking stalls. Towards an extremity of the robot arm, there may be provided an end-effector shown in FIG. 3 schematically as a teat treating device 12 having an image capture device 15 and a treatment fluid applicator 4, which may be one or more spray nozzles, or a dipping cup or another type of treatment medium applicator such as a brush or set of brushes. As can be inferred from the illustration of FIG. 3, the encoder provides an ongoing means of reference for the control system 5 as to the rotational movement speed and distance of the platform and allows a determination to be made of the platform's angular position at any time. This also allows the control system to determine the exact location of each of the stalls on the platform 18 at any time and it further allows the control system 5 to determine which, if any, stall is momentarily at the robot's treatment location, i.e. within working range of the robot. The robot in FIG. 3 is shown in a partially deployed position, between a retracted [starting] position and a fully deployed, operational position in which the end effector would be above the platform 18 in the vicinity of an animal's udder.

(7) In FIG. 2 there is illustrated a partial schematic view of a platform 18 as seen from its periphery nearby a position 7 of a treatment apparatus 1. Three animal stalls 24, 25 and 26 are visible in the partial view, each separated by a respective bail 3 and each occupied by an animal 27, 28 or 29. Also shown is a series of points O.sub.1, O.sub.2, O.sub.3 corresponding to successive points in time t.sub.0 as the platform 18 moves them past the robot position 7; a series of points S.sub.1, S.sub.2, S.sub.3 corresponding to successive points in time t.sub.s; a series of points D.sub.1, D.sub.2, D.sub.3 corresponding to successive points in time t.sub.d; a series of points E.sub.1, E.sub.2, E.sub.3 corresponding to successive points in time t.sub.e etc. A reference location R is indicated under the udder of animal 28 which occupies stall 25. The positions O.sub.1, O.sub.2, O.sub.3 may not be visible on the platform 18 although they will be reference positions corresponding to particular platform periphery locations which will be registered by the apparatus control system 5 as the platform moves past the robot location 7. The direction of movement of the platform is indicated by the arrow M. From the approximate positions of the markings O.sub.1, O.sub.2, O.sub.3, E.sub.1, E.sub.2, E.sub.3 etc, as indicated in FIG. 2, it can be seen that if the platform moves at a more or less constant speed, then the minimum time available for the treating action corresponds to the time taken to move a relevant stall 24, 25, 26 through the distance between D.sub.1 and E.sub.1 (or D.sub.2 and E.sub.2 etc.) past a robot position 7.

(8) The flowchart in FIG. 1 shows an illustrative method sequence according to the present invention. A discussion of the flowchart will be given in the context of the illustrations at FIGS. 2 and 3. When an encoder 20 generates a movement signal which leads to a position determination either in the encoder 20 itself or in the control system 5 indicating that a platform is in an angular position which brings a milking stall 24, 25 or 26 to a starting position for a treatment, then a t.sub.0 signal is generated within the control system 5. Then actual time (current time) t.sub.a is considered to be a starting time t.sub.0. This point in time may in particular coincide with a leading bail 3, as seen in the direction of platform movement, reaching a position at the working vicinity of the treatment apparatus 1 shown by way of example at position 7 in FIG. 2. At this point, the imaging apparatus 15 which may be a TOF camera or other 3D camera, would, from its retracted position 7 of the robot, be actively gathering image information for analysis by an associated image processor. From the information, a determination is made as to whether or not an animal 27, 28, 29 is present at a relevant stall in the vicinity of the robot (i.e. at a treatment location). As an alternative, according to the invention, any suitable known sensor device may be employed for ascertaining an animal's presence after time t.sub.0 is recognised by the control system 5. If an animal is found, the control system 5 makes a determination of the animal's position using any relevant information from the sensors which are used. Provided a point in time t.sub.d has not been exceeded, the end effector 12 of the robot arm, carrying a treating device 4 is then moved to a working position in the vicinity of the animal's udder from which position additional visual information is gathered in the form of image information using imaging device 15, allowing for a determination of the spatial positions of the animal's teats, by means of associated image processing means. When all teat positions have been determined, and provided the latest point in time t.sub.d for a completed detection phase has not passed, then a dedicated treatment action is derived which action takes into account all the detected teat positions and is then carried out. Accordingly, the end effector 12 is moved though a treatment path allowing the treating device 4 to treat each one of the teats before the robot arm is retracted to its starting position. Although not illustrated in the flowchart, in case an end time point t.sub.e is reached before the treating action is completed, then it is interrupted and the robot is moved to its starting position out of the way of the animal 27, 28 or 29 and the stall 24, 25 or 26 which are being moved along by the platform. It may be noted that in the context of a moving platform 18, the time available between various points in time t.sub.0, t.sub.d etc. may vary, depending on the speed of progress of the platform.

(9) It should be noted that the robot control system 8 may ensure that during image gathering operations by the imaging device 15, when in an operational deployed position in the vicinity of an animal's udder, the end effector 12 is moved along in synchronisation with the platform. Thus, the end effector 12, during its image gathering operation, is effectively motionless in relation to the platform stall.

(10) As can be seen from the flowchart in FIG. 1, if, during a detection phase in which udder images are gathered, from a deployed position of the robot arm end effector 12, and analysed, the point in time t.sub.d is reached (i.e. actual time t.sub.a equals or is after t.sub.d) before all four teat positions are detected but after at least one teat position is found, then the teat position determination operation is interrupted. Instead of searching for, detecting and recording/registering the remaining undetected teat positions, an adapted treating action is derived on the basis of that or those teat position/s which has/have been detected and on the basis of the best estimate by the control system of where the undetected teats are likely to be, i.e. on the basis of one or more approximated teat positions. Again, provided the point in time t.sub.d is not exceeded, then the adapted treating action is carried out, treating all teats, before the robot arm is retracted to its starting position. Approximated positions are used for remaining unknown teat positions and these may be derived in a number of ways, in particular, using information from detected teats and knowing that teats are generally arranged in a regular or more or less predictable arrangement depending on the animal species or breed in question.

(11) If, during a detection phase in which udder images are gathered, from a deployed position of the robot arm end effector 12, and analysed, the point in time t.sub.d is reached before any teat position is detected, then the teat position determination operation is interrupted. Instead of detecting the undetected teat positions, a default treating action is derived on the basis of estimated positions of all four teats. This may be carried out using basic data gained from the location of an animal's udder or from an approximate determination of the animal's position. Provided the point in time t.sub.d is not exceeded, then the default treating action is carried out, treating all teats, before the robot arm is retracted to its starting position. Approximated positions are used for unknown teat positions and these may be derived in a number of ways, in particular, using information from a detected user position or from detected position information of any other body parts of the animal in question. Teats are generally arranged in a regular or more or less predictable arrangement in respect of the animal species or breed in question.

(12) It will be understood that the most efficient and effective treating action is the dedicated treating action, while the adapted treating action offers the best possible treatment in case at least one, but not all teat positions are detected. The default treating action is likely to take longest and to use most treatment medium such as treatment fluid. Nevertheless, the default and adapted treating actions provide backup treating actions which are likely to be required only in a minority of instances. An advantage of the present invention is that generally speaking, a dedicated treating action can be implemented, saving on treating medium and ensuring an excellent treatment. In case the dedicated action cannot be performed, then instead of there being no treatment or a robot error, possibly stopping further progress of a platform or further operations at a stationary milking stall, the adapted treating action is implemented offering almost all the advantages of the dedicated action, or, a default action is implemented, at least ensuring that the relevant animal is treated and avoiding interrupting production.

(13) In FIG. 4, there is illustrated a flowchart of a method including some optional aspects of the invention. After detection of an animal at a treatment location, a determination of a reference location R is made, nearby the udder of the relevant animal. As discussed previously in this disclosure, the reference location R corresponds to a co-ordinate position in relation to the treatment location which puts the end effector 12 in the operational vicinity of the animal's udder for the purpose of both image gathering for teat position detection and to the purpose of providing a starting position for the treating action of the treating device 4 of the end effector (i.e. the treating device of the treatment apparatus 1). After the location R is detected, the imaging device 15 and its associated image processor attempt to determine a so-called entry window to the reference location R by calculating the space available around a path between the robot 7 and the reference location R. If there is calculated to be enough space to allow passage of the end effector 12 to the reference location R without contacting the animal, and provided time t.sub.s, the latest point in time for completion of searching is not passed, then the robot is controlled to move the end effector 12 to that location R for the next stages of the method, as previously described, to take place. A robot arm end effector 12 may be held at position R, effectively motionless in relation to a moving platform stall, while the platform continues to move.

(14) In some embodiments, the reference position R may be determined after a determination of access via an entry “window” has been made. In still further embodiments, an additional step (not represented) of analysing image information to assess whether a milking cluster is or is not attached to the animal's udder may be carried out. If a cluster is determined to be attached to the animal's udder, then in particular, no pre- or post-treatment may be carried out. The animal in question might travel around the rotary platform (or “rotary”) once again. In such a case, also no further pre-treatment would be carried out at a pre-treatment location owing to a cluster being detected on the animal's teats also at a pre-treatment location. A further example of specific and optional aspects of the operation of a method and apparatus according to the invention may be as follows:

(15) The teat treating system may be controlled by a number of interacting programs. One optional spray operation may include some or all of the following steps: 1. A robot manager program monitors the rotary platform movement. At one specified location per stall, a robot manager program of the treatment apparatus may send a start command to a robot controller program. 2. A robot controller program uses the information from a vision sensor (TOF camera) to determine whether an animal is present in the stall, whether a milking cluster is detached, and whether there is enough room for the robot arm to enter between the rear legs of the animal or—as the case may be—from the animal's side. 3. If an animal is present and other conditions are fulfilled, the robot may wait until the cow is directly in front of the robot at position 7. If not, operation may be aborted and a robot controller program may wait for the next start command (the next stall) indicating another point in time t.sub.0. 4. When the animal is in front of the robot opposite position 7, the robot starts following the rotary platform movement, enters—possibly between the rear legs, or from the side—and uses the vision sensor to identify the teats. The position to where the robot enters is determined by the location of the udder. If the rotary starts reversing, the robot follows, keeping in the same position relative the moving platform. 5. After identifying the at least some or all of the teat positions, a treatment path is derived and the robot moves to the relevant determined or calculated teat positions one by one and sprays them individually, in accordance with a dedicated, adapted or default treatment path. 6. After all teats are sprayed, the robot exits from underneath the cow, moving out from the rotating platform. The robot returns to the start position and may wait for the next start command from the robot manager program.

(16) There may also be a stop command sent at time t.sub.e by a robot manager program to abort all operations if the robot is still under the cow and is approaching the physical limit of its working range. If the robot by some reason is unable to complete the exit action in time, the platform may be stopped by triggering an emergency stop.

(17) The platform motion may be measured and monitored with a standard encoder, possibly mounted on the inside or outside of the rotary platform in line with the working range (or working area) for the robot and relayed to the control system 5 of the treatment apparatus 1. The robot may in some embodiments use a separate encoder connected directly to a robot controller 8. Optionally, both encoders may be mounted on the same encoder wheel axis.

(18) Next to the platform encoder 20, there may be provided a synchronization sensor which is triggered once per revolution. The synchronization signal may ensure that the apparatus control 5 system at every moment knows which stall is in front of the robot, and also exactly the relation between the robot and platform coordinate systems.

(19) The vision guided robot motion may be divided into two subtasks; (i) locate animal and udder, and (ii) find individual teats. This image processing may operate on one set of images at a time, each set from one time sample. The identification and decision making using information from also previous time samples may be performed by robot controller program.

(20) According to embodiments of the invention, the pre- or post-treatment apparatus may comprise a spray robot, having a treating device 4 in the form of one or more spray nozzles. The robot may be based on a standard 6-axis industrial robot mounted to the floor on the outside of a parallel type rotary platform. Modifications from a standard industrial robot may mainly consist of coating/sealing certain moving parts and interlaces as well as a communication interface and end-effector 12. The end effector may have a TOF camera pointing towards the centre of the rotary. The camera (and associated image processor) is used for identifying cow presence in the stall, it may additionally check for presence of a cluster or no cluster on an udder of a detected animal. If there is no cluster found, the camera and image processor may find a suitable opening between rear legs or under the animal from a side position and later finding the teats to be treated. Camera cleaning may be built in the end effector and may be performed automatically by a cycle of water and air sprayed at a camera housing viewing glass shield.

(21) The end effector may in particular have three spray nozzles pointing upwards, to ensure enough spray capacity to coat the relevant teat and also to create a drip on the teat tip, while ensuring adequate coverage of the teat target area. The system may be controlled by a touch-screen graphical user interface placed outside the robot cell and allowing interactive control by an operator. A display screen may for example present the current status of the system and the success/failure of treatment operations.

(22) Embodiments illustrated and described herein are non-limiting. Further and equivalent features within the scope of the claims will be apparent to one skilled in the art.