Robotic imaging system with velocity-based collision avoidance mode
12447621 ยท 2025-10-21
Assignee
Inventors
Cpc classification
B25J9/1682
PERFORMING OPERATIONS; TRANSPORTING
B25J9/1676
PERFORMING OPERATIONS; TRANSPORTING
B25J9/1666
PERFORMING OPERATIONS; TRANSPORTING
International classification
Abstract
A robotic imaging system includes a camera configured to obtain one or more images of a target site. A robotic arm is operatively connected to the camera, the robotic arm being adapted to selectively move the camera in a movement sequence. The robotic imaging system includes a sensor configured to detect and transmit sensor data related to a respective position and/or a respective speed of the camera. A controller is configured to receive the sensor data, the controller having a processor and tangible, non-transitory memory on which instructions are recorded. The controller is adapted to selectively execute a collision avoidance mode, which includes determining a trajectory scaling factor for the camera. The trajectory scaling factor is applied to modulate the respective speed when the camera and/or the robotic arm are in a predefined buffer zone.
Claims
1. A robotic imaging system comprising: a camera configured to record one or more images of a target site; a robotic arm operatively connected to the camera, the robotic arm being adapted to selectively move the camera in a movement sequence, the robotic arm including one or more joints; a sensor configured to detect and transmit sensor data related to a respective position and/or a respective speed of the camera; a controller configured to receive the sensor data, the controller having a processor and tangible, non-transitory memory on which instructions are recorded; an orbital scan mode executable by the controller to enable the robotic arm to sweep an orbital trajectory at least partially circumferentially around the target site; wherein the controller is adapted to selectively execute a collision avoidance mode based on the sensor data, including: determining a trajectory scaling factor for the camera; applying the trajectory scaling factor to modulate the respective speed when at least one of the camera and the robotic arm is in a predefined buffer zone; and generating an adjusted orbital trajectory based in part on the trajectory scaling factor; wherein the controller is adapted to generate the adjusted orbital trajectory from the orbital trajectory via a limiting feedback term and a resetting feedback term; and wherein the limiting feedback term causes a cycle radius in the adjusted orbital trajectory to decrease while near a joint limit, and the resetting feedback term causes the cycle radius in the adjusted orbital trajectory to reset back when the joint limit has been cleared.
2. The robotic imaging system of claim 1, wherein: the camera is a stereoscopic camera configured to record a left image and a right image for producing at least one stereoscopic image of the target site.
3. The robotic imaging system of claim 1, wherein: the predefined buffer zone is within a delta value of at least one keep-out zone, application of the trajectory scaling factor pushing the camera away from the at least one keep-out zone.
4. The robotic imaging system of claim 1, further comprising: a head unit for housing the camera; a coupling plate mechanically coupling the head unit to the robotic arm, the head unit being operatively connected to a cart; and wherein the controller is adapted to calculate the trajectory scaling factor for a set of checkpoints located on the head unit, the robotic arm and/or the coupling plate, the set of checkpoints each having a respective position along a first direction and a respective velocity along the first direction.
5. The robotic imaging system of claim 1, wherein: the controller is adapted to initialize the trajectory scaling factor to a normalized value when the robotic arm and/or the camera is outside of the predefined buffer zone, the trajectory scaling factor being based on multiple limit calculations; and the controller is adapted to obtain a minimum value of a plurality of local scales respectively obtained from the multiple limit calculations, the trajectory scaling factor being chosen as a lower one of the normalized value and the minimum value of the plurality of local scales.
6. The robotic imaging system of claim 5, wherein the multiple limit calculations each employ respective linear functions raised to a predetermined scale power between 1 and 2, inclusive.
7. The robotic imaging system of claim 1, wherein: the robotic arm includes a plurality of joints defining respective joint angles therebetween; the trajectory scaling factor is based on multiple limit calculations, including a joint avoidance calculation; and the controller is adapted to execute the joint avoidance calculation by checking a distance and speed of the respective joint angles of the robotic arm against respective fixed joint angle limits.
8. The robotic imaging system of claim 1, further comprising: a cart operatively connected to the camera; wherein the trajectory scaling factor is based on multiple limit calculations, including a cart avoidance calculation; and wherein the controller is adapted to execute the cart avoidance calculation for a set of checkpoints located on the camera, including checking the respective speed and distance of the set of checkpoints against a surface of the cart.
9. The robotic imaging system of claim 8, wherein the surface of the cart is modelled as a sphere.
10. The robotic imaging system of claim 1, wherein: the trajectory scaling factor is based on multiple limit calculations, including a boundary plane avoidance calculation; and the controller is adapted to execute the boundary plane avoidance calculation for a set of checkpoints located on the camera, including checking the respective speed and distance of a set of checkpoints against at least one predefined boundary plane.
11. The robotic imaging system of claim 1 wherein the adjusted orbital trajectory is defined by a spherical coordinate axis with a first spherical angle and a second spherical angle.
12. The robotic imaging system of claim 1, wherein the target site includes an or a serrata of an eye.
13. The robotic imaging system of claim 11, further comprising: a low-pass filter selectively executable by the controller to smooth changes in the second spherical angle in each cycle; and a saturation function selectively executable by the controller, the saturation function limiting a magnitude of the second spherical angle to be within 0 and 90 degrees in each cycle, inclusive.
14. The robotic imaging system of claim 11, wherein: the controller is adapted to change a view angle of the orbital trajectory by keeping the first spherical angle constant while iterating the second spherical angle until a desired viewing angle is reached; and the controller is adapted to selectively command the orbital trajectory by iterating the first spherical angle between a predefined starting angle and a predefined ending angle while the second spherical angle is at the desired viewing angle.
15. The robotic imaging system of claim 11, wherein the limiting feedback term is based on the trajectory scaling factor, a time derivative of the trajectory scaling factor, a first proportional gain constant and a first derivative gain constant.
16. The robotic imaging system of claim 15, wherein: the limiting feedback term (R.sub.limit) is obtained as: R.sub.limit=[K.sub.p1(1.0SF)K.sub.d1({dot over (S)}F)], such that SF denotes the trajectory scaling factor, {dot over (S)}F denotes the time derivative of the trajectory scaling factor, K.sub.p1 denotes the first proportional gain constant and K.sub.d1 denotes the first derivative gain constant.
17. The robotic imaging system of claim 11, wherein the resetting feedback term is based on a cycle radius corresponding to the second spherical angle, a time derivative of the cycle radius, a nominal radius corresponding to a desired viewing angle, a second proportional gain constant and a second derivative gain constant.
18. The robotic imaging system of claim 17, wherein: the resetting feedback term (R.sub.reset) is obtained as: R.sub.reset=[K.sub.p2(R.sub.0R)K.sub.d2({dot over (R)})], such that R denotes the cycle radius, R.sub.0 denotes the nominal radius, {dot over (R)} denotes the time derivative of the cycle radius, K.sub.p2 denotes the second proportional gain constant and K.sub.d2 denotes the second derivative gain constant.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
(1)
(2)
(3)
(4)
(5)
(6)
(7)
(8)
(9) Representative embodiments of this disclosure are shown by way of non-limiting example in the drawings and are described in additional detail below. It should be understood, however, that the novel aspects of this disclosure are not limited to the particular forms illustrated in the above-enumerated drawings. Rather, the disclosure is to cover modifications, equivalents, combinations, sub-combinations, permutations, groupings, and alternatives falling within the scope of this disclosure as encompassed, for instance, by the appended claims.
DETAILED DESCRIPTION
(10) Referring to the drawings, wherein like reference numbers refer to like components,
(11) Referring to
(12) Referring to
(13) Referring to
(14) The robotic arm 24 includes one or more joints, such as first joint 30 and second joint 32, configured to provide further degrees of positioning and/or orientation of the head unit 18. Referring to
(15) Referring to
(16) The head unit 18 may be connected to a cart 34 having at least one display medium (which may be a monitor, terminal or other form of two-dimensional visualization), such as first and second displays 36 and 38 shown in
(17) Referring to
(18) The camera 12 is configured to acquire images of the target site 16, which may be presented in different forms, including but not limited to, captured still images, real-time images and/or digital video signals. Real-time as used herein generally refers to the updating of information at the same rate as data is received. More specifically, real-time means that the image data is acquired, processed, and transmitted at a high enough data rate and a low enough delay that when the data is displayed, objects move smoothly without user-noticeable judder or latency. Typically, this occurs when new images are acquired, processed, and transmitted at a rate of at least about 30 frames per second (fps) and displayed at about 60 fps and when the combined processing of the video signal has no more than about 1/30.sup.th second of delay.
(19) As described below, the collision avoidance mode 14 is velocity-based. The controller C is adapted to compute a single multiplier (referred to herein as trajectory scaling factor) and applying it to the velocity commands (to the robotic arm 24) to slow down robot speeds to a stop as specific limits are approached. The trajectory scaling factor is utilized to avoid self-collisions of the various components, for example, collisions of the camera 12 with the robotic arm 24, collisions of the camera 12 with a storage unit (e.g., cart 34) and collisions of the first joint 30 with the second joint 32.
(20) At each control update of the robotic arm 24, the desired velocity is checked and if it is determined the robotic arm 24 is approaching a limit, a scale-down occurs. The limits can be defined in joint space, Cartesian space or another reference frame. If the robotic arm 24 is not approaching a limit, the trajectory scaling factor is set to 1.0, allowing a quick reversal from a limit. The collision avoidance mode 14 provides relatively smooth motion when entering a region to be avoided. There is no need to clear limits in order to exit the restricted region.
(21) The controller C is adapted to calculate the trajectory scaling factor for one or more checkpoints 200 located on the head unit 18, the robotic arm 24, and other places, as shown in
(22) The collision avoidance mode 14 of
(23) Referring now to
(24) Method 100 begins with block 102 of
(25) Advancing to block 106 in
(26) Proceeding to block 108 in
(27) Advancing to block 110 in
(28) In the embodiment described herein, the method 100 uses three different types of limit calculations: a joint avoidance calculation, a cart avoidance calculation and a boundary plane avoidance calculation. However, it is understood that other types of limit calculations may be employed. With respect to the joint avoidance calculation, the controller C executes it by checking the distance and speed of the respective joint angles of the robotic arm 24 against respective fixed joint angle limits. With respect to the cart avoidance calculation, for a set of checkpoints 200, the controller C checks the respective distance (e.g., radial distance r) and respective speed ({dot over (r)}) of the checkpoints 200 against a surface of the cart 34. The surface of the cart 34 may be modelled as a sphere. With respect to the boundary plane avoidance calculation, the controller C is adapted to check the respective distance and respective speed of the checkpoints 200 against at least one predefined boundary plane 50 (see
(29) Each of the limit calculations has a set of tunable parameters that govern how quickly the robotic arm 24 decelerates to zero as the respective limit is approached. The set of tunable parameters may include which of the axes to limit and the origin of the limit. Because the joint links are able to move position and change orientation, the checkpoints 200 may collide with them from many different angles. To prevent this, the X-axis, Y-axis, and Z-axis must be selectively constrained. In other words, the tunable parameters have an enabled axis member that sets a respective multiplier for whether the X, Y and Z axes are active. For link boundaries, the origin is presumed to be where the reference frame for a link boundary exists. The set of tunable parameters may include a trigger distance (D.sub.T) and a stop distance (D.sub.S). The trigger distance 306 (D.sub.T) is the distance that the checkpoints 200 should be from a boundary origin (e.g., origin O in
(30) A graphical representation of how the local scale is chosen for each of the limit calculations is shown in
(31)
(32) The local scale (S.sup.V) is obtained as the scale factor S raised to a power, referred to herein as scale power V. The scale power is one of the tunable factors governing the process. The higher the value of the scale power V, the more aggressively the collision avoidance mode 14 will scale the speed of the robotic arm 24. The higher the selected value of the scale power, the more aggressively the collision avoidance mode 14 will scale the speed of the robotic arm 24.
(33) Further, per block 110 of
(34) Proceeding to block 112 of
(35) Orbital Trajectory Correction with Collision Avoidance Mode
(36) Referring now to
(37) Due to workspace and other limitations, the robotic imaging system 10 may be unable to complete a full rotation of the orbital scan. These limitations are typically from joint limits. Many common joint limits are fully elbow extension or compression. Current orbit algorithms may get stuck in joint limits and be unable to complete the designated rotation trajectory, requiring manual user intervention to exit the limit and re-position the camera 12.
(38) The orbital trajectory 410 may be defined in terms of a spherical coordinate system having a first spherical angle (U) and a second spherical angle (V), shown in
(39) Next, the orbital trajectory 410 is achieved by holding the second sphere angle (V) constant at the desired viewing angle, while iterating movement along the first sphere angle (U) of the virtual sphere 500, an example of which is shown in
(40) The controller C may generate the adjusted orbital trajectory 420 by modifying the orbital trajectory 410 by a limiting feedback term and a resetting feedback term [R=R.sub.limit+R.sub.reset]. The limiting feedback term (R.sub.limit) may be obtained as: R.sub.limit=[K.sub.p1(1.0SF)K.sub.d1({dot over (S)}F)], such that SF denotes the trajectory scaling factor, SF denotes a time derivative of the trajectory scaling factor, K.sub.p1 denotes a first proportional gain constant and K.sub.d1 denotes a first derivative gain constant. The resetting feedback term (R.sub.reset) may be obtained as: R.sub.reset=[K.sub.p2(R.sub.0R)K.sub.d2({dot over (R)})], such that R denotes the cycle radius, R.sub.0 denotes a nominal radius, R denotes the time derivative of the cycle radius, K.sub.p2 denotes a second proportional gain constant and K.sub.d2 denotes a second derivative gain constant. The first proportional gain constant, first derivative gain constant, second proportional gain constant and second derivative gain constant may be individually tuned and chosen via a calibration process.
(41) When the trajectory scaling factor declines due to proximity to a joint limit, the limiting feedback term reduces the cycle radius (as shown at region 422 of
(42) In summary, when performing an orbital trajectory in a scenario where robot joint limits prevent a full rotation of the first spherical angle (U) at the desired viewing angle, the collision avoidance mode 14 may dynamically pull in the cycle radius (as shown at region 422 of
(43) The controller C of
(44) The network 64 may be a serial communication bus in the form of a local area network. The local area network may include, but is not limited to, a Controller Area Network (CAN), a Controller Area Network with Flexible Data Rate (CAN-FD), Ethernet, blue tooth, WIFI and other forms of data. The network 64 may be a Wireless Local Area Network (LAN) which links multiple devices using a wireless distribution method, a Wireless Metropolitan Area Network (MAN) which connects several wireless LANs or a Wireless Wide Area Network (WAN) which covers large areas such as neighboring towns and cities. Other types of connections may be employed.
(45) The controller C of
(46) Look-up tables, databases, data repositories or other data stores described herein may include various kinds of mechanisms for storing, accessing, and retrieving various kinds of data, including a hierarchical database, a set of files in a file system, an application database in a proprietary format, a relational database management system (RDBMS), etc. Each such data store may be included within a computing device employing a computer operating system such as one of those mentioned above and may be accessed via a network in one or more of a variety of manners. A file system may be accessible from a computer operating system and may include files stored in various formats. An RDBMS may employ the Structured Query Language (SQL) in addition to a language for creating, storing, editing, and executing stored procedures, such as the PL/SQL language mentioned above.
(47) The flowcharts presented herein illustrate an architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It will also be noted that each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations, may be implemented by specific purpose hardware-based devices that perform the specified functions or acts, or combinations of specific purpose hardware and computer instructions. These computer program instructions may also be stored in a computer-readable medium that can direct a controller or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable medium produce an article of manufacture including instructions to implement the function/act specified in the flowchart and/or block diagram blocks.
(48) The numerical values of parameters (e.g., of quantities or conditions) in this specification, including the appended claims, are to be understood as being modified in each respective instance by the term about whether or not about actually appears before the numerical value. About indicates that the stated numerical value allows some slight imprecision (with some approach to exactness in the value; about or reasonably close to the value; nearly). If the imprecision provided by about is not otherwise understood in the art with this ordinary meaning, then about as used herein indicates at least variations that may arise from ordinary methods of measuring and using such parameters. In addition, disclosure of ranges includes disclosure of each value and further divided ranges within the entire range. Each value within a range and the endpoints of a range are hereby disclosed as separate embodiments.
(49) The detailed description and the drawings or FIGS. are supportive and descriptive of the disclosure, but the scope of the disclosure is defined solely by the claims. While some of the best modes and other embodiments for carrying out the claimed disclosure have been described in detail, various alternative designs and embodiments exist for practicing the disclosure defined in the appended claims. Furthermore, the embodiments shown in the drawings or the characteristics of various embodiments mentioned in the present description are not necessarily to be understood as embodiments independent of each other. Rather, it is possible that each of the characteristics described in one of the examples of an embodiment can be combined with one or a plurality of other desired characteristics from other embodiments, resulting in other embodiments not described in words or by reference to the drawings. Accordingly, such other embodiments fall within the framework of the scope of the appended claims.