Robot System and Calibration Method
20250326124 · 2025-10-23
Assignee
Inventors
Cpc classification
G05B2219/39057
PHYSICS
B25J13/089
PERFORMING OPERATIONS; TRANSPORTING
International classification
Abstract
A robot system includes a robot, a controller, and a sensor. The robot 1) comprises links connected to one another by joints. The sensor comprises an optical sensor portion arranged to sense displacement of one of the links and an evaluation portion for deriving velocity information from image data output by the sensor portion. The controller calculates an expected position of a reference point on one of the links based on rotation angles associated with the joints and on a vector of intrinsic parameters of the robot, controls movement of the reference point along a predetermined path so that the expected position varies with a predetermined velocity, compares a velocity measured by the sensor during movement to the predetermined velocity, and minimizes a deviation between the measured and predetermined velocities by varying the vector of intrinsic parameters.
Claims
1. A robot system, comprising: an articulated robot comprising a plurality of links connected to one another by rotary joints; an optical flow sensor comprising an optical sensor portion arranged to sense displacement of one of the plurality of links, and an evaluation portion for deriving velocity information from image data output by the sensor portion; and a controller configured to: calculate an expected position of a reference point on one of the plurality of links based on rotation angles associated to the rotary joints and on a vector of intrinsic parameters of the articulated robot; control movement of the reference point along a predetermined path so that the expected position varies with a predetermined velocity; compare a velocity measured by the optical flow sensor during the movement to the predetermined velocity; and minimize a deviation between the measured and predetermined velocities by varying the vector of intrinsic parameters.
2. The robot system of claim 1, wherein the optical sensor portion is arranged to sense displacement of the one of the plurality of links by being mounted on the particular link and being disposed to sense a stationary surface.
3. The robot system of claim 1, wherein the optical sensor portion is a camera, and the movement of the reference point is parallel to a surface that is being observed by the camera during said movement.
4. The robot system of claim 3, wherein during the movement an optical axis of the camera is controlled to be perpendicular to the surface.
5. The robot system of claim 3, wherein the surface is a floor or a wall of a room containing the articulated robot, or a surface on which the robot is mounted.
6. The robot system of claim 1, wherein in a coordinate system in which the sensor portion is stationary, an end effector of the robot is also stationary, optionally wherein the end effector and the sensor portion are attached to a distalmost one of said links.
7. The robot system of claim 1, wherein the predetermined path comprises at least one of a straight line, a circle, a circular arc, a pair of lines that meet under a predetermined angle, a rectangle, a polygon.
8. The robot system of claim 1, wherein at least one of the intrinsic parameters relates to an offset between a target rotation angle and an actual rotation angle of a joint.
9. A method for calibrating an articulated robot comprising a plurality of links connected to one another by rotary joints, the method comprising: a) controlling movement of a reference point on one of the plurality of links along a predetermined path so that an expected position of the reference point calculated based on rotation angles associated to the joints and on a vector of intrinsic parameters of the robot varies with a predetermined velocity; b) simultaneously with step a), measuring the velocity of the reference point using an optical flow sensor; c) comparing the velocity measured in step b) to said predetermined velocity; and d) minimizing a deviation between the measured and predetermined velocities by varying the vector of intrinsic parameters.
10. The method of claim 9, wherein step d) comprises repeating steps a) to c) after having made a variation to the vector of intrinsic parameters.
11. The method of claim 9, wherein the amount and the direction of the predetermined velocity is constant along at least part of the predetermined path.
12. The method of claim 11, wherein step c) comprises recording fluctuations of the amount and of the direction of the velocity measured in step b).
13. The method of claim 9, further comprising: e) simulating at least one data series of velocities expected along said path when the intrinsic parameter vector of the robot differs from a nominal parameter vector by a predetermined deviation vector; and f) choosing the variation made in step d) based on a comparison of velocities measured in step b) with said at least one simulated data series.
14. The method of claim 9, wherein in step b), measurement of the velocity is further based on IMU data.
15. The method of claim 9, wherein the method is computer implemented in a set of instructions that are stored on tangible computer media and are executed by computer to carry out steps of the method.
Description
BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWING(S)
[0007]
[0008]
[0009]
DETAILED DESCRIPTION OF THE INVENTION
[0010]
[0011] Motors 92, 93, 94 associated to joints 5.sub.2, 5.sub.3, 5.sub.4 and reduction gears 10.sub.2, 10.sub.3, 10.sub.4 connecting these motors to links 4.sub.2, 4.sub.2 and 4.sub.3, respectively, are shown in
[0012] The reduction gear 10.sub.j is a harmonic gear, shown in the cross section in
Since the flex spline is 18 rotating in a direction opposite that of the wave generator 16, points of the two currently in contact will meet again after the wave generator 16 has rotated by 2i.sub.f/i.sub.c, slightly less than a complete turn.
[0013] The link 4.sub.6 at the distal end of the robot arm 1 carries an optical sensor portion 20 of an optical flow sensor 22. A signal processor 21 for extracting the optical flow from the output of the optical sensor portion 20 can be integrated in the optical sensor portion 20 itself, or controller 2. As shown here, the optical sensor portion 20 can be attached to a flange of link 4.sub.6 which, in normal operation of the robot arm 1, would carry an end effector. The optical sensor portion 20 can be of a type commonly used in optical computer mice, designed to detect a displacement of the link 4.sub.6 while being in sliding contact with the support 7. Alternatively, the end effector can be attached to the flange and hold the optical sensor portion 20.
[0014] According to a still further alternative, the optical sensor portion 20 might comprise an electronic camera, outputting a series of frames, and the signal processor for extracting the optical flow from the frames is implemented in controller 2. The camera can be attached to the link 4.sub.6 as described above, or a dedicated connector for the camera could be provided on the link 4.sub.6 in addition to the flange, or on link 4.sub.5, so that in normal operation the camera can be attached to the robot arm 1 concurrently with the end effector and can be used to monitor operation of the end effector.
[0015] In a particular embodiment, the camera is part of a smartphone temporarily attached to the robot arm 1, e.g. being held by the end effector. In that case, the signal processor 21 for extracting the optical flow can be the processor which is conventionally built-into the smartphone, and which can be adapted to extract the optical flow by appropriate software loaded into a memory of the smartphone. Since many conventional smartphones come with an IMU, the software can be designed to consider output from the IMU when extracting the optical flow.
[0016] The support 7 can be provided with a grid or pattern 23, in order to facilitate quantitative detection of a displacement of the link 4.sub.6 by the flow sensor 22.
[0017] To facilitate quantitative detection of a displacement, it is also helpful for the camera to be held with its optical axis perpendicular to the surface of the support 7 observed by it, so that the size of a feature on that surface will not vary while in images from the camera while the camera is moved parallel to the surface.
[0018] Controller 2 comprises a command generator 8 which receives a predetermined trajectory , defined by a plurality of points in three-dimensional space and associated instants in time at which a reference point of the link 4.sub.6 or of the optical sensor portion 20 mounted to it is to reach each of these points. Based on a model of the robot arm 1, the command generator determines a series of rotation angles q.sub.mot,j to be assumed by the motors 9.sub.j of the joints 5.sub.j in order to place the reference point at points of the predetermined trajectory and of motor speeds {dot over (q)}.sub.mot,j needed in order to have the reference point move along the trajectory at a constant speed.
[0019] To calculate the position of the reference point in three-dimensional space from motor angles q.sub.mot,j, the relation between each motor angle and the associated joint angle and the length of each link must be known. Ideally, the relation between motor and joint angles would simply be q.sub.arm,j=R.sub.jq.sub.mot,j, and the length of each link should be equal to a rated value. When imperfections of the reduction gear 10.sub.j are taken into account, the relation can be written as
wherein .sub.0 is an offset between an angle measured by resolver 8; and the actual rotation angle of link 4.sub.j, A.sub.i,j and .sub.i,j are weighting coefficients and phases of harmonics caused by imperfections such as eccentricities of gearwheels in the reduction gear 10.sub.j, and .sub.0+
is referred to as the kinematic error correction q.sub.ke,j of joint 4.sub.j. Based on the relation above, the model allows to calculate the position of the reference point as a function not only of the motor angles q.sub.mot,j, but also of the intrinsic parameters A.sub.i,j, .sub.i,j, .sub.0 that are characteristic for this particular robot arm 1.
[0020] It should be kept in mind that the above equation is an approximation, the precision of which varies depending on the number of summands M considered. In a simple but practically relevant case, M can be zero, i.e. the equation reduces to
[0021] This case will not be specifically described below, since it can be readily derived from any of the following mathematical expressions by setting M=0, or A.sub.i,j, .sub.i,j=0.
[0022] Trajectory generator 11 is adapted to output to joint controllers 12.sub.j associated to each joint 5.sub.j the series of position and speed commands q.sub.mot,j {dot over (q)}.sub.mot,j with an appropriate timing, as would be required for having a reference point follow the predetermined trajectory assuming that the resolver offset .sub.0 is zero, and that the transmission ratio of each reduction gear 10.sub.j is the rated ratio and independent of the orientation of its associated motor output shaft 17.sub.j i.e. that all coefficients .sub.0, A.sub.i,j are zero.
[0023] Each joint controller 12.sub.j further has associated to it a kinematic error calculator 13.sub.j for calculating the kinematic error correction q.sub.ke,j and its derivative {dot over (q)}.sub.ke,j and adders 14.sub.j, 15.sub.j for superimposing the kinematic error correction onto the commands q.sub.mot,j, {dot over (q)}.sub.mot,j output by the trajectory generator 11, and providing corrected commands {circumflex over (q)}.sub.mot,j=q.sub.mot,j+q.sub.ke,j, {circumflex over ({dot over (q)})}.sub.mot,j={dot over (q)}.sub.mot,j+q.sub.ke,j as will be explained in detail below.
[0024] It can be assumed that prior to calibration of the robot arm 1 by the method of the present invention, the kinematic error calculators 13.sub.j apply kinematic error corrections q.sub.ke, {dot over (q)}.sub.ke=0 for all joints.
[0025] In a first step of the method, the command generator calculates a series of commands {q.sub.mot,j, {dot over (q)}.sub.mot,j}.sub.0 which would cause the reference point to follow the predetermined trajectory by moving along a predetermined path 24 at a constant speed under the assumption that the intrinsic parameters A.sub.i,j, .sub.i,j, .sub.0 are all zero, and dimensions of the links equal their respective rated values.
[0026] While trajectory generator 11 outputs this series of commands {q.sub.mot,j, {dot over (q)}.sub.mot,j}.sub.0, causing the reference point to move approximately along the path 24, the optical flow sensor 22 records fluctuations of the velocity vector.
[0027] In an elementary embodiment, the same series of commands {q.sub.mot,j, {dot over (q)}.sub.mot,j}.sub.0 is output repeatedly, with different vectors of values of the intrinsic parameters being provided to the kinematic error calculators 13.sub.j, finally retaining the vector that minimized variation of the optical flow. Such an approach can be viable where the number of parameters to be optimized is small, when fluctuations of the transmission ratio are assumed to vanish for all joints, i.e. .sub.0, A.sub.i,j=0.
[0028] In a more systematic embodiment, controller 2 first calculates based on a path integral of these fluctuations a trajectory that was actually followed by the reference point.
[0029] When, at a given point in time moving along the trajectory , commands are such that the reference point should be located at a point r in space, but the actual position resulting from execution of the commands is r+, then it should be possible to place the reference point closer to r by issuing commands that would place it at r, instead. Based on the original predetermined trajectory defining points r and the one actually followed and defining points r+, the controller 7 then calculates a corrected trajectory * formed of points r and a series of corrected commands associated to the corrected trajectory *. Then it determines a vector of intrinsic parameters (A.sub.i,j, .sub.i,j, .sub.0) so that when the trajectory generator outputs the original series of commands {q.sub.mot,j, {dot over (q)}.sub.mot,j}.sub.0, kinematic error calculators 13.sub.j will add such a correction q.sub.ke, {dot over (q)}.sub.ke that commands {circumflex over (q)}.sub.mot,j=q.sub.mot,j+q.sub.ke, {circumflex over ({dot over (q)})}.sub.mot,j={dot over (q)}.sub.mot,j+q.sub.ke received by the joint controllers 12.sub.j approximate those associated to the corrected trajectory *.
[0030] Based on plain and simple guessing or on some more elaborate, conventional optimization algorithm, the controller determines a vector of values of the intrinsic parameters .sub.0, and, if appropriate, A.sub.i,j, .sub.i,j, such that an improved trajectory predicted by the model based on this vector of values is closer to the corrected trajectory * than the original predetermined trajectory was. Different metrics can be used for quantifying closeness, e.g. an integral, taken along one of the trajectories *, , of the scalar distance between points associated to a same instant in time, of the two trajectories, or the maximum of the scalar distance over the two trajectories. This vector of values k is then input to the kinematic error calculators 13.sub.j, enabling these, while the trajectory generator 11 outputs the same series of commands {q.sub.mot,j, {dot over (q)}.sub.mot,j} as before a second time, to calculate, for each value of q.sub.mot,j, {dot over (q)}.sub.mot,j output by the trajectory generator, the appropriate kinematic error q.sub.ke,j. {dot over (q)}.sub.ke,j as a function of q.sub.mot,j so that the joint controllers 12.sub.j receive corrected commands {circumflex over (q)}.sub.mot,j=q.sub.mot,j+q.sub.ke, {circumflex over ({dot over (q)})}.sub.mot,j={dot over (q)}.sub.mot,j+{dot over (q)}.sub.ke. In this way, a vector of intrinsic parameters reasonably close to the true values can be obtained based on a single displacement of the reference point along the predeterminer trajectory.
[0031] If desired, intrinsic parameters still closer to the true values can be obtained by having the command generator calculates new series of commands {q.sub.mot,j, {dot over (q)}.sub.mot,j}.sub. based on the vector of intrinsic parameters obtained previously, i.e. comprising those commands that would be required to have the reference point follow the predetermined trajectory if the intrinsic parameter values used in the model for determining the commands had been , not 0. Using this new series of commands {q.sub.mot,j, {dot over (q)}.sub.mot,j }.sub. in place of {q.sub.mot,j, {dot over (q)}.sub.mot,j}.sub.0 the above procedure can be repeated as often as needed to reduce the fluctuations of the velocity vector below a given limit.
[0032] The predetermined trajectory can be chosen so that in order to displace the reference point along its path, the robot arm 1 must move in all its degrees of freedom. In that case, the intrinsic parameters can be obtained concurrently for all joints 5.sub.j resp. all reduction gears 10.sub.j of the robot arm. However, the large number of parameters under consideration tends to make calculations lengthy and to increase the number of iterations needed to bring the fluctuations of the velocity vector below a given limit. In the case shown in
[0033] Once intrinsic parameters have thus been determined for joints 5.sub.2, 5.sub.3 and 5.sub.5, these can be regarded as known and no longer subject to optimization when intrinsic parameters for the remaining links are determined using a different predetermined path that requires a movement of one or more of the yet unexamined joints and of one or more of joints 5.sub.2, 5.sub.3 and 5.sub.5. For example, a second predetermined path might be a path 25, perpendicular to path 24, on horizontal support 7. For the reference point to follow this path, joints 5.sub.1, 5.sub.6 must rotate in addition to joints 5.sub.2, 5.sub.3 and 5.sub.5 so that, when following path 25 with intrinsic parameters for joints 5.sub.2, 5.sub.3 and 5.sub.5 being known from having carried out the above method for path 24, parameters for joints 5.sub.1, 5.sub.6 are quite straightforward to find in spite of the large number of joints involved in the movement.
[0034] As an alternative, paths 26, 27 on a vertical surface, e.g. on a workshop wall next to the robot 1, might be used.
[0035] While conventional camera-based calibration methods require that relative positions of a reference point of the robot, a reference point of the working environment and of the camera be known with high precision for a reference pose of the robot, and inaccuracies in the measurement of each can affect the quality of calibration, there is no such requirement for the robot system of the present invention, so that a precise calibration can be achieved without much preparatory work.
[0036] The optical sensor portion could be stationary and be arranged to sense a displacement of the reference point by having the latter move through its detection range. Preferably, however, the optical sensor portion is arranged to sense displacement of the reference point by being mounted on a same link with said reference point and sensing a stationary surface, since then a single optical sensor portion can be sufficient for carrying out a calibration spanning the entire displacement range of the link, even if the detection range of the optical sensor portion is smaller than the displacement range.
[0037] Different types of optical flow sensor can be used for the invention. In a basic embodiment, an optical flow sensor of the type used in computer mice could be employed, and the movement of the sensor portion of the optical flow sensor might be in sliding contact with a surface of the robot's working environment. In a preferred embodiment, the optical flow sensor comprises a camera, and the evaluation portion is adapted to extract an optical flow from image data of the camera, since the camera might also be used for closed loop control of the robot, for example when the reference point must be positioned with respect to a workpiece whose position is variable.
[0038] Extracting the optical flow from image data is facilitated when the movement of the reference point is parallel to a surface that is being observed by the camera during said movement.
[0039] The surface observed can be any flat surface in the working environment of the robot, such as a floor or a wall of a room containing the robot, or a tabletop supporting the robot. Preferably the robot is mounted on said surface, since thus the orientation of the surface in a coordinate system in which a base of the robot is stationary is known, and the predetermined path can easily be controlled to be parallel to the surface.
[0040] In a coordinate system in which the sensor portion is stationary, the reference point should preferably be stationary, too. This can be achieved by attaching both to a same link, typically a distalmost link, of the robot.
[0041] Irregularities of the optical flow are the easier to detect, the more regular the predetermined path is. Preferably, therefore, the predetermined comprises at least one of a straight line, a circle or a circular arc. To allow calibration of various degrees of freedom of the robot, more than one of the above-defined paths can be combined, thus forming a composite path comprising e.g. a pair lines that meet under a predetermined angle, or several pairs forming a rectangle or a polygon.
[0042] At least one of the intrinsic parameters to be optimized in the calibration should relate to the length of one of said links (or, to be precise, the distance between axes of joints at the proximal and distal ends of said link), to an offset between a target rotation angle and an actual rotation angle of a joint, or to the transmission ratio of a reduction gear associated to one of said joints. Where the intrinsic parameter relates to the transmission ratio, it may define Fourier coefficients and phases of harmonic components of the rotation speed of an output shaft of the reduction gear when the input shaft is driven at a constant rotation speed.
[0043] The deviation to be minimized should be a difference amount integrated along said predetermined path, as such a deviation between an expected position or speed and a position or speed deduced from the optical flow will vanish when calibration is perfect.
[0044] The invention further provides a method for calibrating an articulated robot comprising a plurality of links connected to one another by rotary joints, the method comprising the steps of: [0045] a) controlling movement of a reference point on one of said links along a predetermined path so that an expected position of the reference point calculated based on target rotation angles associated to the joints and on a vector of intrinsic parameters of the robot varies with a predetermined velocity, [0046] b) simultaneously with step a), measuring the velocity of the reference point using an optical flow sensor, [0047] c) comparing the velocity measured in step b) to said predetermined velocity; [0048] and [0049] d) minimizing a deviation between the measured and predetermined velocities by varying the vector of intrinsic parameters.
[0050] Step d) may comprise repeating steps a) to c) after having made a variation to the vector of intrinsic parameters, to determine the effect of the variation.
[0051] The amount and, optionally, the direction of the predetermined velocity can be held constant along at least part of the predetermined path, so that any variation of the amount or direction of the velocity derived from the optical flow can be regarded as a deviation which is to be minimized.
[0052] In order to reduce the time needed for the minimization, the method can comprise the steps of: simulating at least one data series of velocities expected along said path when the intrinsic parameter vector of the robot differs from a nominal parameter vector by a predetermined deviation vector, choosing the variation made in step d) based on a comparison of velocities measured in step b) with said at least one simulated data series. The more closely one of these data series resembles the result of measuring the velocity in step b), the more safely it can be assumed correcting the vector of intrinsic parameters by said predetermined deviation vector will reduce the deviation between measured and predetermined velocities.
[0053] The measurement of velocity in step b) can be facilitated by also taking into account output from an inertial measurement unit (IMU) which is moved jointly with at least the optical sensor portion of the flow sensor. Since many mobile phones come equipped with both a camera and an IMU, using such a mobile phone by e.g. temporarily attaching it to the robot arm can be a very convenient and economic way to carry out the calibration.
[0054] A further object of the invention is a computer program product comprising instructions which, when executed by a processor, cause the processor to operate as the controller in the robot system described above or to carry out the above method.
[0055] Further features and advantages of the invention will become apparent from the subsequent description of embodiments, referring to the appended drawings.
[0056] All references, including publications, patent applications, and patents, cited herein are hereby incorporated by reference to the same extent as if each reference were individually and specifically indicated to be incorporated by reference and were set forth in its entirety herein.
[0057] The use of the terms a and an and the and at least one and similar referents in the context of describing the invention (especially in the context of the following claims) are to be construed to cover both the singular and the plural, unless otherwise indicated herein or clearly contradicted by context. The use of the term at least one followed by a list of one or more items (for example, at least one of A and B) is to be construed to mean one item selected from the listed items (A or B) or any combination of two or more of the listed items (A and B), unless otherwise indicated herein or clearly contradicted by context. The terms comprising, having, including, and containing are to be construed as open-ended terms (i.e., meaning including, but not limited to,) unless otherwise noted. Recitation of ranges of values herein are merely intended to serve as a shorthand method of referring individually to each separate value falling within the range, unless otherwise indicated herein, and each separate value is incorporated into the specification as if it were individually recited herein. All methods described herein can be performed in any suitable order unless otherwise indicated herein or otherwise clearly contradicted by context. The use of any and all examples, or exemplary language (e.g., such as) provided herein, is intended merely to better illuminate the invention and does not pose a limitation on the scope of the invention unless otherwise claimed. No language in the specification should be construed as indicating any non-claimed element as essential to the practice of the invention.
[0058] Preferred embodiments of this invention are described herein, including the best mode known to the inventors for carrying out the invention. Variations of those preferred embodiments may become apparent to those of ordinary skill in the art upon reading the foregoing description. The inventors expect skilled artisans to employ such variations as appropriate, and the inventors intend for the invention to be practiced otherwise than as specifically described herein. Accordingly, this invention includes all modifications and equivalents of the subject matter recited in the claims appended hereto as permitted by applicable law. Moreover, any combination of the above-described elements in all possible variations thereof is encompassed by the invention unless otherwise indicated herein or otherwise clearly contradicted by context.