An information processing device includes at least one memory, and at least one processor configured to perform, based on a state of a virtual world and a predetermined environment variable, a simulation with respect to the state of the virtual world, the state of the virtual world being based on an observation result of a real world, and the simulation being differentiable, and update the predetermined environment variable so that a result of the simulation approaches a changed state of the virtual world, the changed state being based on an observation result of the real world that is observed after the real world has changed.

A method for determining a control position of a robot includes (a) acquiring N real reference positions in a real space and N control reference positions in a robot control coordinate system, (b) setting M figures having vertices in a plurality of real reference positions of the N real reference positions within the real space, and obtaining a transform function expressing a correspondence relationship between a real position and a control position within each figure, (c) receiving input of a target position of the control point in the real space, (d) selecting an object figure for calculation of a control position for the target position from the M figures, and (e) calculating a target control position for the target position using the transform function with respect to the object figure.

A method of calibrating a system including a camera, the method including detecting a robot navigating within an environment modeled as a geo-polygon space, including a transit of the robot through a scene of the environment captured by the camera, mapping a plurality of points occupied by the robot in images of the scene to the geo-polygon space, recording data about the mapping, and configuring at least one alert using the data recorded about the mapping, the alert executed by the computing system and configured to be triggered by an object transiting the scene.

A system has a virtual-world (VW) controller and a physical-world (PW) controller. The pairing of a PW element with a VW element establishes them as corresponding physical and virtual twins. The VW controller and/or the PW controller receives measurements from one or more sensors characterizing aspects of the physical world, the VW controller generates the virtual twin, and the VW controller and/or the PW controller generates commands for one or more actuators affecting aspects of the physical world. To coordinate the corresponding virtual and physical twins, (i) the VW controller controls the virtual twin based on the physical twin or (ii) the PW controller controls the physical twin based on the virtual twin. Depending on the operating mode, one of the VW and PW controllers is a master controller, and the other is a slave controller, where the virtual and physical twins are both controlled based on one of VW or PW forces.

A control system includes: a controller configured to operate one or more robots in a real space based on an operation program; and circuitry configured to: operate one or more virtual robots based on the operation program in a virtual space, the one or more virtual robots corresponding to the one or more robots respectively; cause the controller to suspend an operation based on the operation program by the one or more robots; simulate a suspended state of the real space after suspension of the operation by the one or more robots, in the virtual space; and resume at least a part of the operation by the one or more virtual robots based on the operation program, in the virtual space in which the suspended state of the real space has been simulated.

A method for generating a control program for a laboratory automation device includes receiving configuration data of the laboratory automation device, generating a three-dimensional model of the components of the laboratory automation device from the configuration data, the three-dimensional model additionally including a virtual pipette; displaying the three-dimensional model with a virtual reality headset; receiving movement data of a motion sensing controller controlled by a user wearing the virtual reality headset, the movement data indicating a three-dimensional movement of the motion sensing controller in space; determining a movement of the virtual pipette in the three-dimensional model from the movement data and updating the three-dimensional model according to the movement of the virtual pipette; and generating a control program for the laboratory automation device from the movement data.

A computer-implemented method includes operating a controllable physical device to perform a task. The method also includes miming forward simulations of the task by a physics engine based on one or more physics parameters. The physics engine communicates with a parameter data layer where each of the one or more physics parameters is modeled with a probability distribution. For each forward simulation run, a tuple of parameter values is sampled from the probability distribution of the one or more physics parameters and fed to the physics engine. The method includes obtaining an observation pertaining to the task from the physical environment and a corresponding forward simulation outcome associated with each sampled tuple of parameter values. The method then includes updating the probability distribution of the one or more physics parameters in the parameter data layer based on the observation from the physical environment and the corresponding forward simulation outcomes.

A method for generating a control program (54) for a laboratory automation device (12) comprises: receiving configuration data (46) of the laboratory automation device (12), the configuration data (46) encoding positions of components (22) in the laboratory automation device (12); generating a three-dimensional model (58) of the components (22) of the laboratory automation device (12) from the configuration data (46), the three-dimensional model (22) additionally including a virtual pipette (60); displaying the three-dimensional model (58) with a virtual reality headset (14); receiving movement data (50) of a motion sensing controller (16) controlled by a user wearing the virtual reality headset (14), the movement data (50) indicating a three-dimensional movement of the motion sensing controller (16) in space; determining a movement of the virtual pipette (60) from the movement data (50) in the three-dimensional model (58) and updating the three-dimensional model (58) according to the movement of the virtual pipette (60); and generating a control program (54) for the laboratory automation device (12) from the movement data (50), wherein the control program (54) is adapted for moving a pipetting arm (30) with a pipette (32) of the laboratory automation device (12) with respect to the components (22) accordingly to the movement of the virtual pipette (60) in the three-dimensional model (58).

A system has a virtual-world (VW) controller and a physical-world (PW) controller. The pairing of a PW element with a VW element establishes them as corresponding physical and virtual twins. The VW controller and/or the PW controller receives measurements from one or more sensors characterizing aspects of the physical world, the VW controller generates the virtual twin, and the VW controller and/or the PW controller generates commands for one or more actuators affecting aspects of the physical world. To coordinate the corresponding virtual and physical twins, (i) the VW controller controls the virtual twin based on the physical twin or (ii) the PW controller controls the physical twin based on the virtual twin. Depending on the operating mode, one of the VW and PW controllers is a master controller, and the other is a slave controller, where the virtual and physical twins are both controlled based on one of VW or PW forces.

A system for robot teaching based on RGB-D images and a teach pendant, including an RGB-D camera, a host computer, a posture teach pendant, and an AR teaching system which includes an AR registration card, an AR module, a virtual robot model, a path planning unit and a posture teaching unit. The RGB-D camera collects RGB images and depth images of a physical working environment in real time. In the path planning unit, path points of a robot end effector are selected, and a 3D coordinates of the path points in the basic coordinate system of the virtual robot model are calculated; the posture teaching unit records the received posture data as the postures of a path point where the virtual robot model is located, so that the virtual robot model is driven to move according to the postures and positions of the path points, thereby completing the robot teaching.