A robotic system may comprise a first robotic arm operatively coupleable to a first tool. The first tool has a first working end. The system may also comprise an image capture device, a display, and a processor. The processor may be configured to cause an image of a work site, which was captured by the image capture device from a perspective of an image reference frame, to be displayed on the display. The image of the work site includes an image of the first working end of the first tool. The processor may also determine a position of the first working end of the first tool in the image of the work site and render a tool information overlay at the position of the first working end of the first tool in the image of the work site. The tool information overlay visually indicates state information for the first tool. The processor may also change the tool information overlay while the first tool is in a first operational state by changing a brightness of the tool information overlay.

An offline programming apparatus includes a first part which causes an operator to designate contact points at which a contact sensor arranged at a distal end of a robot contacts a workpiece on a 3D model of the workpiece, a second part which automatically adds detection starting points in operations of the robot which approaches each of the contact points in directions of movement along coordinate axes of a reference coordinate system with reference to which the operation of the robot to detect the position of the workpiece is performed, at positions where no interference occurs between the robot and the workpiece in a virtual space, and a third part which automatically adds an interference avoidance point in the virtual space in order to avoid the interference if the robot and workpiece interfere with each other on a movement path of the robot which moves between the detection starting points.

A method and apparatus for generating machining code of workpieces from a paper engineering drawing are provided. The method includes processing the paper engineering drawing to be a binary image; extracting dimension features and shape features of the workpieces from the binary image; and generating the machining codes of the workpieces based on the extracted dimension features and shape features of the workpieces. The machining codes indicate the dimension and shape of the workpieces. Machining codes of workpieces are generated from a paper engineering drawing directly without manual involvement.

An operator telerobotically controls tools to perform a procedure on an object at a work site while viewing real-time images of the work site on a display. Tool information is provided in the operator's current gaze area on the display by rendering the tool information over the tool so as not to obscure objects being worked on at the time by the tool nor to require eyes of the user to refocus when looking at the tool information and the image of the tool on a stereo viewer.

An operator telerobotically controls tools to perform a procedure on an object at a work site while viewing real-time images of the work site on a display. Tool information is provided in the operator's current gaze area on the display by rendering the tool information over the tool so as not to obscure objects being worked on at the time by the tool nor to require eyes of the user to refocus when looking at the tool information and the image of the tool on a stereo viewer.

A synthetic representation of a robot tool for display on a user interface of a robotic system. The synthetic representation may be used to show the position of a view volume of an image capture device with respect to the robot. The synthetic representation may also be used to find a tool that is outside of the field of view, to display range of motion limits for a tool, to remotely communicate information about the robot, and to detect collisions.

A synthetic representation of a robot tool for display on a user interface of a robotic system. The synthetic representation may be used to show the position of a view volume of an image capture device with respect to the robot. The synthetic representation may also be used to find a tool that is outside of the field of view, to display range of motion limits for a tool, to remotely communicate information about the robot, and to detect collisions.

An offline programming apparatus includes a first part which causes an operator to designate contact points at which a contact sensor arranged at a distal end of a robot contacts a workpiece on a 3D model of the workpiece, a second part which automatically adds detection starting points in operations of the robot which approaches each of the contact points in directions of movement along coordinate axes of a reference coordinate system with reference to which the operation of the robot to detect the position of the workpiece is performed, at positions where no interference occurs between the robot and the workpiece in a virtual space, and a third part which automatically adds an interference avoidance point in the virtual space in order to avoid the interference if the robot and workpiece interfere with each other on a movement path of the robot which moves between the detection starting points.

A system comprises: a robotic arm operatively coupleable to a tool comprising a working end; and an input device communicatively coupled to the robotic arm. The input device is manipulatable by an operator. The system further comprises a processor configured to cause an image of a work site, captured by an image capture device from a perspective of an image reference frame, to be displayed on a display. The image of the work site includes an image of the working end of the tool. The processor is further configured to determine a position of the working end of the tool in the image of the work site and render a tool information overlay at the position of the working end of the tool in the image of the work site. The tool information overlay visually indicates an identity of the input device.

An operator telerobotically controls tools to perform a procedure on an object at a work site while viewing real-time images of the work site on a display. Tool information is provided in the operator's current gaze area on the display by rendering the tool information over the tool so as not to obscure objects being worked on at the time by the tool nor to require eyes of the user to refocus when looking at the tool information and the image of the tool on a stereo viewer.