A control console includes a first input control operable by an operator and outside of a field of view of the operator, a display within the field of view of the operator, and a first image capture device. The first image capture device acquires a first image of a physical environment surrounding the first input control and at least part of the first input control. The first image is output on the display. In some embodiments, a second image of a work site may also be displayed, so that the operator can locate and operate the first input control while continuously viewing the second image of the work site.

A surgical method is provided for use with a teleoperated surgical system (surgical system), the method comprising: recording surgical instrument kinematic information indicative of surgical instrument motion produced within the surgical system during the occurrence of the surgical procedure; determining respective kinematic signatures associated with respective surgical instrument motions; producing an information structure in a computer readable storage device that associates respective kinematic signatures with respective control signals; comparing, during a performance of the surgical procedure surgical instrument kinematic information during the performance with at least one respective kinematic signature; launching, during a performance of the surgical procedure an associated respective control signal in response to a match between surgical instrument kinematics during the performance and a respective kinematic signature.

A teleoperated medical device includes a sensor that senses information that represents a position of a control input out of an operator's field of view and displays the information to the operator. The displayed information may be combined with an image of a surgical site, so that the operator can locate and operate the control input while continuously viewing the image of the surgical site.

A robot system includes robot bodies, operation devices each configured to accept operation and generate operational information for causing the robot body to operate, motion controllers configured to control operation of the corresponding robot body in response to the operational information, operation target selectors configured to receive an operation for selecting any of the robot bodies and request a permission to operate the selected robot body based on the operational information from the corresponding operation device, and an operation permitting device having a determinator configured to receive the permission request from the operation target selector and determine whether a permission is to be granted for the permission request. When the permission request is received, and the operation of the robot body selected by the operation target selector based on the operational information from a different operation device is permitted, the determinator prohibits the permission to the permission request.

Techniques for input control visualization include a control console. The control console includes a first input control operable by an operator and outside of a field of view of the operator, a display within the field of view of the operator, and one or more sensors. The one or more sensors acquire information about the first input control and a physical environment surrounding the first input control. A representation of the first input control and the physical environment is generated from the information. The representation is displayed on the display.

A display system for performing a predetermined operation with respect to a moving body is disclosed. The display system includes an operation reception unit configured to receive a switching operation to switch an operation mode between a manual operation mode and an autonomous movement mode, the manual operation mode being selected for moving the moving body by manual operation and the autonomous movement mode being selected for moving the moving body by autonomous movement; and a display controller configured to display notification information representing accuracy of the autonomous movement.

This robot system comprises: a display control unit which controls an image to be displayed in a display region of a display unit; and a reference direction setting unit which sets a reference direction for determining the orientation of an image. On the basis of the axial positions of respective joint shafts of the robot and the reference direction, the display control unit controls the orientation of an image shown on the display unit so as to perform display that represents a prescribed postural relation with respect to the reference direction.

A robot system according to the present disclosure includes a robot installed in a work area, a manipulator configured to be gripped by an operator and manipulate the robot, a sensor disposed at a manipulation area and configured to wirelessly detect positional information and posture information on the manipulator, and a control device which calculates a locus of the manipulator based on the positional information and the posture information on the manipulator detected by the sensor, and operates the robot on real time.

A surgical robotic system includes a robotic arm having at least one surgical instrument and a control device coupled to the robotic arm and configured to control the robotic arm, the control device including one or more components. The surgical robotic system also includes a power supply having: a tower power supply chassis configured to supply first direct current to the robotic arm; a power distribution unit configured to supply second direct current to the one or more components; a first uninterruptable power supply device coupled to the tower power supply chassis and configured to receive a first alternating current from a first alternating current input; and a second uninterruptable power supply device coupled to the power distribution unit and configured to receive a second alternating current from a second alternating current input.

A method includes receiving sensor data for an environment about the robot. The sensor data is captured by one or more sensors of the robot. The method includes detecting one or more objects in the environment using the received sensor data. For each detected object, the method includes authoring an interaction behavior indicating a behavior that the robot is capable of performing with respect to the corresponding detected object. The method also includes augmenting a localization map of the environment to reflect the respective interaction behavior of each detected object.