A device includes: an image sensor to capture images of an object within an FOV, a guide projector and a processor. The processor is to analyze some of the images to detect entry of an encoded data marking carried by the object into the FOV, and in response to detecting such entry into the FOV: operate the guide projector to project a visual guide onto the object to guide movement of the encoded data marking to a first location indicated by the visual guide within the FOV; analyze more of the images to detect such movement to the first location, and then to a second location within the FOV; and in response to the movement to the second location, interpret the movement to the second location as receipt of manual input; and in response to the manual input, transmit data decoded from the encoded data marking to another device.

Systems and methods for image space control of a medical instrument are provided. In one example, a system is configured to display a two-dimensional medical image including a view of at least a distal end of an instrument. The system can determine, based on one or more fiducials on the instrument, a roll estimate of the instrument. The system further can receive a user input comprising a heading command to change a heading of the instrument within a plane of the medical image, or an incline command to change an incline of the instrument into or out of the plane of the medical image. Based on the roll estimate and the user input, the system can generate one or more motor commands configured to cause a robotic system coupled to the medical instrument to move the robotic medical instrument.

An electronic device is provided. The electronic device includes a camera module, a communication circuit, a memory, and a processor, wherein the processor may execute a first application using the camera module, obtain a first image through the camera module while the first application operates, recognize an object of a specified type in the first image, obtain location information of where the first image is obtained, determine a first virtual point corresponding to the location information on a three-dimensional (3D) virtual map, obtain a second image corresponding to the first image at the first virtual point, and update data of the 3D virtual map on the object based on a comparison between the first image and the second image.

Physical object boundary detection techniques and systems are described. In one example, an augmented reality module generates three dimensional point cloud data. This data describes depths at respective points within a physical environment that includes the physical object. A physical object boundary detection module is then employed to filter the point cloud data by removing points that correspond to a ground plane. The module then performs a nearest neighbor search to locate a subset of the points within the filtered point cloud data that correspond to the physical object. Based on this subset, the module projects the subset of points onto the ground plane to generate a two-dimensional boundary. The two-dimensional boundary is then extruded based on a height determined from a point having a maximum distance from the ground plane from the filtered cloud point data.

A method for generating a multi-modal video dataset with pixel-wise hand segmentation is disclosed. To address the challenges of conventional dataset creation, the method advantageously utilizes multi-modal image data that includes thermal images of the hands, which enables efficient pixel-wise hand segmentation of the image data. By using the thermal images, the method is not affected by fingertip and joint occlusions and does not require hand pose ground truth. Accordingly, the method can produce more accurate pixel-wise hand segmentation in an automated manner, with less human effort. The method can thus be utilized to generate a large multi-modal hand activity video dataset having hand segmentation labels, which is useful for training machine learning models, such as deep neural networks.

A method of identifying a flange specification based on an augmented reality interface includes steps of: providing a first interface, through which an operator picks real-time images of a flange at different viewing angles, and creating a virtual flange surface according to the real-time images; providing a second interface, through which the operator picks three circumferential points corresponding to the flange, and creating a virtual flange model having a virtual outer diameter; providing a third interface, through which the operator adjusts a virtual pitch circle diameter of the virtual flange model; providing a fourth interface, through which the operator adjusts a virtual thickness of the virtual flange model; providing a fifth interface, through which the operator inputs a count of bolts; and searching a database according to the virtual outer diameter, the virtual pitch circle diameter and the virtual thickness of the virtual flange model to obtain searched results.

The disclosure relates to a system for analysis of medical image data, which represents a two-dimensional or three-dimensional medical image. The system is configured to read and/or determine, for the medical image, a plurality of image quality metrics and to determine a combined quality metrics based on the image quality metrics. The system is further configured so that the determination of the combined quality metrics takes into account an interaction between the image quality metrics in their combined effect on the combined quality metrics.

A method for vision-based tool localization (VTL) in a robotic assembly system including one or more calibrated cameras, the method comprising capturing a plurality of images of the tool contact area from a plurality of different vantage points, determining an estimated position of the tool contact area based on an image, and refining the estimated position based on another image from another vantage point. The method further comprises providing the refined position to the robotic assembly system to enable accurate control of the tool by the robotic assembly system.

Techniques for tuning an image editing operator for reducing a distractor in raw image data are presented herein. The image editing operator can access the raw image data and a mask. The mask can indicate a region of interest associated with the raw image data. The image editing operator can process the raw image data and the mask to generate processed image data. Additionally, a trained saliency model can process at least the processed image data within the region of interest to generate a saliency map that provides saliency values. Moreover, a saliency loss function can compare the saliency values provided by the saliency map for the processed image data within the region of interest to one or more target saliency values. Subsequently, the one or more parameter values of the image editing operator can be modified based at least in part on the saliency loss function.

A medication dispensing system includes an automated dispensing device that includes cells with electronically activated locks. The device is configured to detect when medication counts in the cells fall below predetermined thresholds. The system further includes a plurality of first electronic devices that are associated with some of the cells and that have imagers. In response to any of the medication counts in the cells being below the predetermined threshold, the automated dispensing device is configured to automatically send a replenishment needed notification to the first electronic device associated with the cell. That first electronic device is configured to transmit a picture of a medication to a second electronic device. In response to a positive verification by a user of the second electronic device that the medication in the picture is the correct medication, the second electronic device is configured to transmit an unlock signal to the automated dispensing device.