A system comprises a first robotic arm adapted to support and move a tool and a second robotic arm adapted to support and move a camera. The system also comprises an input device, a display, and a processor. The processor is configured to, in a first mode, command the first robotic arm to move the camera in response to a first input received from the input device to capture an image of the tool and present the image as a displayed image on the display. The processor is configured to, in a second mode, display a synthetic image of the first robotic arm in a boundary area around the captured image on the display, and in response to a second input, change a size of the boundary area relative a size of the displayed image.

There is provided an apparatus including a storage unit configured to store, in association with each of a plurality of devices existing in a plant, a storage address of measurement data which is related; a detection unit configured to detect, among the plurality of devices, a target device that a user is looking at; an acquisition unit configured to acquire, from the storage address associated with the target device, measurement data related to the target device; and a display unit configured to display a content acquired by the acquisition unit.

An augmented reality (AR) node location and activation technique for use in an AR system in a process plant or other field environment quickly and easily detects an AR node in a real-world environment and is then able to activate an AR scene within the AR system, which improves the usability and user experience of the AR system. The AR node location and activation system generally enables users to connect to and view an AR scene within an AR system or platform even when the user is not directly at an existing AR node, when the user is experiencing poor lighting conditions in the real-world environment and in situations in which the user is unfamiliar with the AR nodes that are in the AR system database. As a result, the user can quickly and easily activate the AR system and connect to an AR scene for an AR node close to the user in the field environment under varying weather and lighting conditions in the field and without requiring a large amount of image processing to locate the correct AR scene based on photographic images provided by the user.

A method includes receiving sensor data for an environment about the robot. The sensor data is captured by one or more sensors of the robot. The method includes detecting one or more objects in the environment using the received sensor data. For each detected object, the method includes authoring an interaction behavior indicating a behavior that the robot is capable of performing with respect to the corresponding detected object. The method also includes augmenting a localization map of the environment to reflect the respective interaction behavior of each detected object.

An augmented reality (AR) node location and activation technique for use in an AR system in a process plant or other field environment quickly and easily detects an AR node in a real-world environment and is then able to activate an AR scene within the AR system, which improves the usability and user experience of the AR system. The AR node location and activation system generally enables users to connect to and view an AR scene within an AR system or platform even when the user is not directly at an existing AR node, when the user is experiencing poor lighting conditions in the real-world environment and in situations in which the user is unfamiliar with the AR nodes that are in the AR system database. As a result, the user can quickly and easily activate the AR system and connect to an AR scene for an AR node close to the user in the field environment under varying weather and lighting conditions in the field and without requiring a large amount of image processing to locate the correct AR scene based on photographic images provided by the user.

This invention is directed to a plant management system that presents a position to be measured by a sensor that performs measurement for inspection of a plant or the like when the measurement is executed, thereby preventing a measurement error. The plant management system includes a measurement position instructor that, when performing the measurement by the sensor on a work site, instructs the position to be measured by the sensor on a measurement target, and a presenter that, when the measurement is executed, performs presentation of an image representing the position to be measured by the sensor on the work site by superimposing the image on an image of the measurement target or projecting the image onto the measurement target. Here, the presenter includes a superimposition display that superimposes and displays the image representing the position to be measured by the sensor on the image of the measurement target using an augmented reality technology or a projection device that projects the image representing the position to be measured by the sensor in association with the measurement target.

A robotic system may comprise a first robotic arm operatively coupleable to a first tool. The first tool has a first working end. The system may also comprise an image capture device, a display, and a processor. The processor may be configured to cause an image of a work site, which was captured by the image capture device from a perspective of an image reference frame, to be displayed on the display. The image of the work site includes an image of the first working end of the first tool. The processor may also determine a position of the first working end of the first tool in the image of the work site and render a tool information overlay at the position of the first working end of the first tool in the image of the work site. The tool information overlay visually indicates state information for the first tool. The processor may also change the tool information overlay while the first tool is in a first operational state by changing a brightness of the tool information overlay.

A method at least including the steps of aligning an image capturing element of a mobile display apparatus on a multi-link actuated mechanism by a user, capturing at least the multi-link actuated mechanism by means of the image capturing element of the mobile display apparatus, identifying the multi-link actuated mechanism in the captured image data of the image capturing element of the mobile display apparatus, indicating in three dimensions the multi-link actuated mechanism on the basis of the captured image data together with the depth information items, and overlaying the virtual representation of the multi-link actuated mechanism on the multi-link actuated mechanism in the display element of the mobile display apparatus, wherein the overlay is implemented taking account of the geometric relationships of the multi-link actuated mechanism.

A sensing system provided with a target detecting unit which applies predetermined processing to data obtained by a detecting device and thereby detects a target object, the sensing system including an augmented-reality-image creating unit which creates an image corresponding to a candidate in a plurality of candidates that appear in the processed data to which the predetermined processing has been applied, a predetermined parameter of the candidate being equal to or greater than a detection threshold, and creates a candidate image corresponding to a candidate in the plurality of candidates, the predetermined parameter of the candidate for the candidate image is less than the detection threshold and equal to or greater than a detection candidate threshold, and a display device that displays the image and the candidate image.

A display device includes an augmented reality display part that displays a virtual robot that operates in accordance with a predetermined program together with objects in real space, a position detection part that detects the position of a work target in real space by measuring a distance to the work target from the augmented reality display part, and a control part that causes the virtual robot displayed on the augmented reality display part to operate based on the position of the work target detected by the position detection part to perform predetermined work on the work target.