A robot controller that controls a robot by automatically obtaining a controller capable of suitably controlling a wide range of robots. An image is acquired from an image capturing apparatus that photographs an environment including the robot. The robot is driven based on an output result obtained by inputting the image to a neural network. The neural network is updated according to a reward generated in a case where a plurality of virtual images photographed while changing an environmental condition of a virtual environment generated by virtualizing the environment and a state of a virtual robot are input to the neural network, and a policy of the virtual robot, which is output from the neural network, satisfies a predetermined condition.

Methods of positioning a surgical instrument can involve advancing a first medical instrument to a treatment site of a patient, the first medical instrument comprising a camera, recording a target position associated with a target anatomical feature at the treatment site, generating a first image of the treatment site using the camera of the first medical instrument, identifying the target anatomical feature in the first image using a pretrained neural network, and adjusting the target position based at least in part on a position of the identified target anatomical feature in the first image.

Methods and apparatus related to receiving a request that includes robot instructions and/or environmental parameters, operating each of a plurality of robots based on the robot instructions and/or in an environment configured based on the environmental parameters, and storing data generated by the robots during the operating. In some implementations, at least part of the stored data that is generated by the robots is provided in response to the request and/or additional data that is generated based on the stored data is provided in response to the request.

Mitigating the reality gap through utilization of technique(s) that enable compliant robotic control and/or compliant robotic contact to be simulated effectively by a robotic simulator. The technique(s) can include, for example: (1) utilizing a compliant end effector model in simulated episodes of the robotic simulator; (2) using, during the simulated episodes, a soft constraint for a contact constraint of a simulated contact model of the robotic simulator; and/or (3) using proportional derivative (PD) control in generating joint control forces, for simulated joints of the simulated robot, during the simulated episodes. Implementations additionally or alternatively relate to determining parameter(s), for use in one or more of the techniques that enable effective simulation of compliant robotic control and/or compliant robotic contact.

This invention provides a robot control device for controlling a robot configured to perform a predetermined operation, where the robot control device comprises an acquisition unit configured to acquire a plurality of images captured by a plurality of image capturing devices including a first image capturing device and a second image capturing device different from the first image capturing device; and a specification unit configured to use the plurality of captured images acquired by the acquisition unit as inputs to a neural network, and configured to specify a control instruction for the robot based on an output as a result from the neural network.

The present disclosure generally relates to a robotic control system and method that utilizes active perception to gather the relevant information related to a robot, a robotic environment, and objects within the environment, and allows the robot to focus computational resources where needed, such as for manipulating an object. The present disclosure also enables viewing and analyzing objects from different distances and viewpoints, providing a rich visual experience from which the robot can learn abstract representations of the environment. Inspired by the primate visual-motor system, the present disclosure leverages the benefits of active perception to accomplish manipulation tasks using human-like hand-eye coordination.

This control device for controlling the motion of a robot comprises a first processing part and a command part. The first processing part sets a first state of the robot and a second state to which the robot transitions from the first state as inputs, and sets at least one basic motion selected from a plurality of basic motions the robot is instructed to perform for transitioning from the first state to the second state and the order in which the basic motions are to be performed as outputs. Prescribed operating parameters are set for each of the basic motions. The command part executes motion commands for the robot on the basis of the output from the first processing part.

A vibration suppression device acquires a teaching position, computes a speed plan based on the acquired teaching position and a first acceleration/deceleration parameter, computes data related to deflection occurring during an acceleration/deceleration operation of a robot based on the teaching position and the speed plan, and acquires data indicating a posture at the teaching position. Further, a machine learning unit of the vibration suppression device estimates an acceleration/deceleration parameter with respect to the data related to the deflection and the data related to the posture using the data related to the deflection and the data related to the posture as input data.

A computer system trains a neural network to predict, for each pixel in an input image, the position that a robot's end effector would reach if a grasp (poke) were attempted at that position. Training data consists of images and end effector positions recorded while a robot attempts grasps in a pick-and-place environment. For an automated grasping policy, the approach is self-supervised, as end effector position labels may be recovered through forward kinematics, without human annotation. Although gathering such physical interaction data is expensive, it is necessary for training and routine operation of state of the art manipulation systems. Therefore, the system comes for free while collecting data for other tasks (e.g., grasping, pushing, placing). The system achieves significantly lower root mean squared error than traditional structured light sensors and other self-supervised deep learning methods on difficult, industry-scale jumbled bin datasets.

A machine learning device acquires from a numerical controller information relating to machining when the machining is performed, and further acquires an actual delay time due to servo control and due to machine movement which are caused in the machining when the machining is performed. Then, the device performs supervised learning using the acquired machining-related information as input data, and using the acquired actual delay time due to servo control and due to machine movement as supervised data, and constructs a learning model, thereby predicting the machine delay time caused in a machine with high precision.