DYNAMIC MOTION PLANNING SYSTEM
20230158670 · 2023-05-25
Inventors
- Hsien-Chung Lin (Fremont, CA, US)
- Chiara Talignani Landi (Reggio Emilia, IT)
- Chi-Keng Tsai (Bloomfield Hills, MI, US)
- Tetsuaki Kato (Fremont, CA, US)
Cpc classification
B25J9/1676
PERFORMING OPERATIONS; TRANSPORTING
B25J9/1666
PERFORMING OPERATIONS; TRANSPORTING
B25J13/089
PERFORMING OPERATIONS; TRANSPORTING
B25J9/1653
PERFORMING OPERATIONS; TRANSPORTING
G05B2219/40477
PHYSICS
International classification
Abstract
A method and system for dynamic collision avoidance motion planning for industrial robots. An obstacle avoidance motion optimization routine receives a planned path and obstacle detection data as inputs, and computes a commanded robot path which avoids any detected obstacles. Robot joint motions to follow the tool center point path are used by a robot controller to command robot motion. The planning and optimization calculations are performed in a feedback loop which is decoupled from the controller feedback loop which computes robot commands based on actual robot position. The two feedback loops perform planning, command and control calculations in real time, including responding to dynamic obstacles which may be present in the robot workspace. The optimization calculations include a safety function which efficiently incorporates both relative position and relative velocity of the obstacles with respect to the robot.
Claims
1. A dynamic motion planning system for an industrial robot, said system comprising: a perception module including at least one sensor or camera configured to detect obstacles in a workspace of the robot; a planning module executing on a first computing device having a processor and memory, said planning module being configured to compute a design robot motion based on a target location of a tool center point on a tool attached to the robot; a motion optimization module executing on the first computing device and receiving obstacle data from the perception module, said motion optimization module being configured to compute a commanded robot motion based on the design robot motion and the obstacle data, where the commanded robot motion is provided as feedback to the planning module for computing the design robot motion for a next control cycle; and a robot control module executing on the first computing device or a second computing device, said robot control module computing robot control commands based on the commanded robot motion, providing the robot control commands to the robot, and receiving actual robot state data as feedback from the robot.
2. The system according to claim 1 wherein the first computing device is a robot controller which executes all of the planning module, the motion optimization module and the robot control module, and provides the robot control commands to the robot.
3. The system according to claim 1 wherein the first computing device executes the planning module and the motion optimization module, and the second computing device is a robot controller in communication with the first computing device, where the robot controller executes the robot control module and provides the robot control commands to the robot.
4. The system according to claim 1 wherein the commanded robot motion deviates from the design robot motion as necessary to provide a clearance space between the robot and any obstacles present in the obstacle data.
5. The system according to claim 1 wherein the motion optimization module computes the commanded robot motion using an iterative optimization computation having an objective function and one or more inequality constraints.
6. The system according to claim 5 wherein the objective function minimizes deviation from the design robot motion, and the inequality constraints include maintaining robot joint positions within predefined joint position ranges, maintaining robot joint velocities below predefined joint velocity limits and maintaining robot joint accelerations below predefined joint acceleration limits.
7. The system according to claim 5 wherein the inequality constraints include a safety constraint computed from a safety function, where the safety function is determined based on a robot-obstacle minimum distance and a robot-obstacle relative velocity.
8. The system according to claim 7 wherein the safety constraint is that the rate of change of the safety function must be greater than or equal to the negative of the safety function multiplied by a coefficient.
9. The system according to claim 7 wherein, when the robot-obstacle relative velocity is greater than zero, the safety function is equal to the robot-obstacle minimum distance, and when the robot-obstacle relative velocity is less than or equal to zero, the safety function is equal to the robot-obstacle minimum distance minus an additional safety margin distance which is a function of the robot-obstacle relative velocity.
10. The system according to claim 1 wherein the design robot motion is a tool center point acceleration in Cartesian space, and the commanded robot motion includes robot joint rotational accelerations for all joints in the robot.
11. A dynamic motion planning system for an industrial robot, said system executing on one or more computing devices and comprising: a planning module configured to compute a design robot motion based on a target location of a robot tool center point; a motion optimization module configured to compute a commanded robot motion based on the design robot motion and obstacle data received from sensors, where the commanded robot motion is provided as feedback to the planning module for computing the design robot motion for a next control cycle; and a robot control module computing robot joint control commands based on the commanded robot motion, providing the robot joint control commands to the robot, and receiving actual robot joint state data as feedback from the robot.
12. The system according to claim 11 wherein the motion optimization module computes the commanded robot motion using an iterative optimization computation having an objective function which minimizes deviation from the design robot motion and a safety constraint determined based on a robot-obstacle minimum distance and a robot-obstacle relative velocity, where the safety constraint is more relaxed when the robot-obstacle relative velocity is greater than zero than when the relative velocity is less than or equal to zero.
13. A method for dynamic motion planning for an industrial robot, said method executed on at least one computing device and comprising: computing a design robot motion based on a target location of a robot tool center point; computing a commanded robot motion based on the design robot motion and obstacle data received from sensors, where the commanded robot motion is provided as feedback for computing the design robot motion for a next control cycle; computing robot joint control commands based on the commanded robot motion; providing the robot joint control commands to the robot; and receiving actual robot joint state data as feedback from the robot.
14. The method according to claim 13 wherein the at least one computing device is a robot controller which executes all of computing a design robot motion, computing a commanded robot motion, computing robot joint control commands, and providing the robot joint control commands to the robot.
15. The method according to claim 13 wherein a first computing device executes computing the design robot motion and computing the commanded robot motion, and a robot controller executes computing the robot joint control commands, and providing the robot joint control commands to the robot.
16. The method according to claim 13 wherein the commanded robot motion is computed using an iterative optimization computation having an objective function and a plurality of inequality constraints, where the objective function minimizes deviation from the design robot motion, and the inequality constraints include maintaining robot joint positions within predefined joint position ranges, maintaining robot joint velocities below predefined joint velocity limits and maintaining robot joint accelerations below predefined joint acceleration limits.
17. The method according to claim 16 wherein the inequality constraints also include a safety constraint computed from a safety function, where the safety function is determined based on a robot-obstacle minimum distance and a robot-obstacle relative velocity.
18. The method according to claim 17 wherein the safety constraint is that the rate of change of the safety function must be greater than or equal to the negative of the safety function multiplied by a coefficient.
19. The method according to claim 17 wherein, when the robot-obstacle relative velocity is greater than zero, the safety function is equal to the robot-obstacle minimum distance, and when the robot-obstacle relative velocity is less than or equal to zero, the safety function is equal to the robot-obstacle minimum distance minus the relative velocity squared divided by two times a maximum robot acceleration.
20. The method according to claim 13 wherein the design robot motion is a tool center point acceleration in Cartesian space, and the commanded robot motion includes robot joint rotational accelerations for all joints in the robot.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
[0008]
[0009]
[0010]
[0011]
[0012]
[0013]
DETAILED DESCRIPTION OF THE EMBODIMENTS
[0014] The following discussion of the embodiments of the disclosure directed to a dynamic motion planning system is merely exemplary in nature, and is in no way intended to limit the disclosed devices and techniques or their applications or uses.
[0015] It is well known to use industrial robots for a variety of manufacturing, assembly and material movement operations. In many robot workspace environments, obstacles may be present and may at times be in the path of the robot's motion. That is, without adaptive motion planning, some part of the robot may collide with or be near to some part of an obstacle when the robot moves from its current position to a destination position. The obstacles may be static structures such as machines, fixtures and tables, or the obstacles may be dynamic (moving) objects such as people, forklifts and other machines.
[0016] Techniques have been developed in the art for computing robot motions such that the tool follows a path to the destination position while avoiding collision of the robot with any obstacle. However, these systems exhibit a variety of shortcomings—including numerical instability issues, insensitivity to static obstacles, computational complexity leading to slowness in motion planning calculations, and inability to consider robot parts (i.e., robot arms) other than the end-of-arm tool in the robot-obstacle collision avoidance calculations.
[0017]
[0018] The safety filter module 120 also receives obstacle data input from a perception module 130. The perception module 130 includes one or more cameras or sensors configured to provide data about obstacles which may exist in the robot workspace. The obstacle data typically includes a minimum robot-obstacle distance, and may also include other data about the position (including spatial shape) and velocity of any obstacles. Based on the obstacle data, the safety filter module 120 computes modified motion commands u.sub.mod and provides the modified motion commands to a robot system 140. If no obstacles exist in the robot workspace, the modified motion commands u.sub.mod will be the same as the design motion commands u.sub.des.
[0019] The robot system 140 includes the robot, which responds to the modified motion commands u.sub.mod by physically moving. The actual robot state vector X represents the robot position and velocity at the current control cycle time step, either in Cartesian space or in joint space. The actual robot state vector X is provided on a feedback loop 150 to both the controller 110 and the safety filter module 120, where it is used in feedback control calculations of new values of the design motion commands u.sub.des and the modified motion commands u.sub.mod, respectively, for the next control cycle time step.
[0020] The feedback control arrangement of the prior art system shown in
[0021] The dynamic motion planning system of the present disclosure overcomes the shortcomings of prior art systems by decoupling motion planning and obstacle avoidance calculations from the feedback control loop of the robot and its controller. The presently disclosed system also uses a simplified but effective safety function formulation which considers both the position and velocity of any obstacles relative to the robot when computing robot motion commands.
[0022]
[0023] The dynamic motion optimization module 220 also receives obstacle data input from a perception module 230. The perception module 230 includes one or more cameras or sensors configured to provide data about obstacles which may exist in the robot workspace. As discussed above relative to
[0024] The dynamic motion optimization module 220 performs an optimization computation which minimizes tracking deviation from the planned robot motion u.sub.des while including robot mechanical limitations and a collision avoidance safety function as constraints. This optimization computation results in a commanded robot motion q.sub.cmd. The commanded robot motion q.sub.cmd is the robot motion in joint space which will take the robot tool to the target location while avoiding any obstacles in the robot workspace. The optimization computation is discussed in detail below. A feedback loop 240 provides the commanded robot motion q.sub.cmd from the dynamic motion optimization module 220 back to the planner module 210. The planner module 210 and the dynamic motion optimization module 220 repeat the calculations described above at each control cycle.
[0025] The dynamic motion optimization module 220 also provides the commanded robot motion q.sub.cmd to a robot controller 250. The robot controller 250 provides robot control commands to a robot 260, and receives actual robot joint positions q.sub.act on a feedback loop 270. The robot controller 250 updates the robot control commands at each control cycle based on the actual robot joint positions q.sub.act and the commanded robot motion q.sub.cmd.
[0026] The dynamic motion planning system of
[0027] The actual hardware implementation of the dynamic motion planning system of
[0028] The dynamic motion optimization module 220 of the presently disclosed system also uses a simplified but effective safety function formulation which considers both the position and velocity of any obstacles relative to the robot when computing the commanded robot motion q.sub.cmd. This safety function formulation and its use in the motion optimization computation are discussed below.
[0029]
[0030] In
[0031] In
where h(X) is the safety function (used in an inequality constraint in the optimization computation), d is the distance from the robot 300 to the obstacle 320, v.sub.rel is the relative velocity, and a.sub.max is the maximum allowable acceleration of the robot based on mechanical limitations.
[0032] The safety function formulation depicted in
[0033]
[0034] At box 406, robot motion optimization computations are performed based on the planned robot motion and the obstacle data. The output of the robot motion optimization computations is the commanded robot motion q.sub.cmd discussed above with respect to
[0035] At box 408, a robot controller provides the commanded robot motion to a robot. The robot controller may perform computations or transformations in order to provide suitable robot joint motion commands to the robot. At box 410, the robot actually moves based on the joint motion commands from the controller. The robot and controller operate as a closed loop feedback control system, where the actual robot state q.sub.act (joint positions and velocities) is fed back to the controller for computation of updated joint commands. The robot and controller operate on a control cycle having a designated time period (i.e., a certain number of milliseconds).
[0036] In the box 406, the motion optimization problem can be formulated as:
argmin∥u.sub.des−{dot over (J)}{dot over (q)}−{dot over (J)}{umlaut over (q)}∥ (1)
Such that;
∥{dot over (q)}∥≤{dot over (q)}.sub.max (2)
∥{umlaut over (q)}∥≤{umlaut over (q)}.sub.max (3)
{dot over (h)}(X)≥−γh(X) (4)
where Equation (1) is the optimization objective function (tracking deviation from the planned motion u.sub.des) to be minimized, and Equations (2)-(4) are inequality constraints which must be met during the iterative optimization computation. In Equations (1)-(3), {dot over (q)} and {umlaut over (q)} are the joint velocities and accelerations, respectively, for all joints in the robot, J is the Jacobian (a derivative of robot configuration) and {dot over (J)} is the derivative of the Jacobian, and {dot over (q)}.sub.max and {umlaut over (q)}.sub.max are predefined maximum joint velocities and accelerations based on robot mechanical limitations. An additional inequality constraint may be added (not shown above) which requires that the joint positions q remain within predefined joint position ranges.
[0037] For the obstacle avoidance constraint, the goal is to keep the safety function h(X)≥0, as shown in
[0038] Upon convergence, the optimization computations yield the commanded robot motion {umlaut over (q)}.sub.cmd which represents the robot motion having the minimum tracking deviation while satisfying the inequality constraints. If an obstacle approaches the robot extremely rapidly, the optimization computations of Equations (1)-(4) become infeasible; in this case, the system will slow down the motion of the robot and, if the optimization computation remains infeasible, stop the robot.
[0039] In
[0040] The dynamic motion planning techniques of
[0041]
[0042] Two different scenarios are depicted in
[0043] In a second scenario, an obstacle moves through the workspace 500, tracing a series of points along an obstacle path 530. In the real laboratory test, the obstacle was a small object held in a person's hand, where the person stepped toward and extended her arm toward the robot, causing the object to trace the obstacle path 530. The obstacle moves along the obstacle path 530 during the time that the robot tool is to be moved from the start point 510 to the target point 512. Using the dynamic path planning techniques of the present disclosure, the robot moves the tool center point from the start point 510 to the target point 512 along an obstacle avoidance path 540. The obstacle avoidance path 540 deviates significantly from the reference path 520 in order to provide safe clearance from the obstacle moving along the obstacle path 530. It is noteworthy that the most pronounced movement of the obstacle avoidance path 540 away from the reference path 520 is at the beginning of the path 540. This is because, at that time, the obstacle is moving toward the robot tool, causing the safety function h(X) to be reduced to compensate for the approach velocity, as discussed with respect to
[0044]
[0045] Multiple scenarios are depicted in
[0046] In other scenarios, a fixed spherical obstacle 630 was placed in the workspace 600 in a location obstructing the nominal path 620. A buffer zone 640 defines a safe distance margin around the obstacle 630, where the robot and the tool center point should remain outside of the buffer zone 640. Four different simulations were run with the obstacle 630, using the same start point 610 and target point 612, where the robot's programmed maximum tool center point velocity was varied from a slowest maximum speed of 850 mm/sec to a fastest maximum speed of 1800 mm/sec. The dynamic path planning techniques of the present disclosure were used to compute the obstacle avoidance paths shown in
[0047] For the slowest robot tool center point speed of 850 mm/sec, the robot moves the tool center point from the start point 610 to the target point 612 along an obstacle avoidance path 652. It can be observed that the obstacle avoidance path 652 deviates only enough from the nominal path 620 to cause the tool center point to remain slightly outside the buffer zone 640. For the somewhat faster robot tool center point speed of 1000 mm/sec, the robot moves the tool center point from the start point 610 to the target point 612 along an obstacle avoidance path 654. The obstacle avoidance path 654 deviates more from the nominal path 620 (than the path 652 does) to cause the tool center point to remain further outside the buffer zone 640. This is the expected behavior, because of the relative velocity subtractive term in the safety function h(X) (shown in
[0048] For the still faster robot tool center point speed of 1500 mm/sec, the robot moves the tool center point from the start point 610 to the target point 612 along an obstacle avoidance path 656. The obstacle avoidance path 656 deviates even more from the nominal path 620 (than the path 654 does) to cause the tool center point to remain even further outside the buffer zone 640. Finally, the fastest robot tool center point speed of 1800 mm/sec results in an obstacle avoidance path 658, which deviates furthest from the nominal path 620.
[0049] Using the dynamic path planning techniques of the present disclosure, the obstacle avoidance path deviates most from the nominal path 620 for the fastest tool speed (the path 658), and deviates least for the slowest tool speed (the path 652). These simulations confirm the behavior that is expected from the safety function and motion optimization computations described above.
[0050] In addition, the commanded motion computation time for each control cycle (the time to provide an output from the dynamic motion optimization module 220) was measured to have an average of 0.38 milliseconds (ms) for the disclosed techniques depicted in
[0051] Throughout the preceding discussion, various computers and controllers are described and implied. It is to be understood that the software applications and modules of these computer and controllers are executed on one or more computing devices having a processor and a memory module. In particular, this includes a processor in each of the robot controller 250 and the optional separate computer (if used) of
[0052] As outlined above, the disclosed techniques for dynamic motion planning to avoid obstacles in a robot workspace provide significant advantages over prior art methods. The disclosed techniques decouple obstacle avoidance motion optimization from the robot-controller feedback loop, thereby avoiding the feedback delay problems of prior art systems. In addition, the disclosed safety function used in motion optimization computations compensates for robot-obstacle relative velocity in a way that is both effective and easily corn puted.
[0053] While a number of exemplary aspects and embodiments of the dynamic motion planning system have been discussed above, those of skill in the art will recognize modifications, permutations, additions and sub-combinations thereof. It is therefore intended that the following appended claims and claims hereafter introduced are interpreted to include all such modifications, permutations, additions and sub-combinations as are within their true spirit and scope.