A user interface (UI) mapper for robotic process automation (RPA) is disclosed. The UI mapper may initially capture UI elements to fetch UI elements faster for later use and allow an RPA developer to “map” the UI elements for automating an application. This may enable subsequent developers who potentially do not have programming knowledge to build RPA workflows using these predefined “target” UI elements.

A program transfer system includes: a communication unit communicating with a robot and outputting a result of the communication; a determination unit checking a work program provided in the robot, based on the result of the communication, and determining whether to import a different target work program from the work program or not; a program import unit importing the target work program to the robot when the determination unit determines the importing; a collection unit collecting work data of complete work from the robot; and a program update unit performing machine learning using the work data collected by the collection unit, and updating the target work program, based on a result of the machine learning.

A lab system configures robots to performs protocols in labs. The lab automation system receives, via a user interface, an instruction from a user to perform a protocol within a lab. The instruction may comprise text, and the lab may comprise a robot configured to perform the protocol. The lab system converts, using a machine learned model, the text into steps and, for each step, identifies one or more of an operation, lab equipment, and reagent associated with the step. In response to detecting an ambiguity/error associated with the step, the lab system notifies the user via the user interface of the ambiguity/error. The lab system may receive one or more indications from the user that resolve the ambiguity/error and update the associated steps. For each step, the lab system configures the robot to perform an identified operation, interact with identified lab equipment, and/or access/use an identified reagent.

A lab system accesses a first protocol for performance by a first robot in a first lab. The first protocol includes a set of steps, each associated with an operation, reagent, and equipment. For each of one or more steps, the lab system modifies the step by: (1) identifying one or more replacement operations that achieve an equivalent or substantially similar result as a performance of the operation, (2) identifying replacement equipment that operates substantially similarly to the equipment, and/or (3) identifying one or more replacement reagents that, when substituted for the reagent, do not substantially affect the performance of the step. The lab system generates a modified protocol by replacing one or more of the set of steps with the modified steps. The lab system selects a second lab including a second and configures the second robot to perform the modified protocol in the second lab.

A lab system identifies a set of steps associated with a protocol for a lab meant to be performed by a robot within the lab using equipment and reagents. The lab system renders, within a user interface, a virtual representation of the lab, a virtual robot, and virtual equipment and reagents. Responsive to operating in a first mode, the lab system simulates the identified set of steps identify virtual positions of the virtual robot within the lab as the virtual robot performs the steps and modifies the virtual representation of the lab to mirror the identified positions of the virtual robot in real-time. Responsive to operating in a second mode, the lab system identifies positions of the robot within the lab as the robot performs the identified set of steps and modifies the virtual representation of the lab to mirror the identified positions of the robot in real-time.

A lab system calibrates robots and cameras within a lab. The lab system accesses, via a camera within a lab, an image of a robot arm, which comprises a visible tag located on an exterior. The lab system determines a position of the robot arm using position sensors located within the robot arm and determines a location of the camera relative to the robot arm based on the determined position and the location of the tag. The lab system calibrates the camera using the determined location of the camera relative to the robot arm. After calibrating the camera, the lab system accesses, via the camera, a second image of equipment in the lab that comprises a second visible tag on an exterior. The lab system determines, based on a location of the second visible tag within the accessed second image, a location of the equipment relative to the robot arm.

A control system including a control device and a development supporting device for developing a plurality of programming languages executed in the control device, wherein the development supporting device includes an input unit that inputs source codes of the plurality of different programming languages, a mapping information producing unit that performs mapping of shared variables selected in the source codes, respectively, and that produces shared variable mapping information, and a transmit unit that transmits source codes and shared variable mapping information to the control device, wherein the control device includes a program executing unit that executes programs described by source codes, and a shared variable processing unit that processes each of mapped shared variables as common shared variables based on shared variable mapping information.

This disclosure provides systems and methods for robotic task planning when a complex task instruction is provided in natural language. Conventionally robotic task planning relies on a single task or multiple independent or serialized tasks in the task instruction. Alternatively, constraints on space of linguistic variations, ambiguity and complexity of the language may be imposed. In the present disclosure, firstly dependencies between multiple tasks are identified. The tasks are then ordered such that a dependent task is always scheduled for planning after a task it is dependent upon. Moreover, repeated tasks are masked. Thus, resolving task dependencies and ordering dependencies, a complex instruction with multiple interdependent tasks in natural language facilitates generation of a viable task execution plan. Systems and methods of the present disclosure finds application in human-robot interactions.

An object of a robot educational material is to acquire a mechanism of a robot. The robot educational material includes the robot for which middleware for hardware input-output is usable, and a textbook including at least a manual for operating the robot as desired.

A display part of a mobile terminal displays an operation program. The operation program includes an operation icon indicating an operation of a robot or an operation tool, and an auxiliary icon having a shape sandwiching the operation icon. The auxiliary icon indicates control of adding an operation of the robot apparatus. The display part is configured to display a screen configured to set setting information related to the operation of the robot apparatus in such a manner that as operator is enabled to set the setting information. The display part displays the operation icon and the auxiliary icon so as to align the operation icon and the auxiliary icon in order of the operations of the robot apparatus.