Systems and methods for determining safe zones in a workspace calculate safe actions in real time based on all sensed relevant objects and on the current state of the machinery (e.g., a robot) in the workspace. Various embodiments forecast, in real time, both the motion of the machinery and the possible motion of a human within the space, and continuously update the forecast as the machinery operates and humans move in the workspace.

The subject matter of this specification generally relates to object recognition for robots. In some implementations, a method includes navigating a robot through an area to identify objects located in the area. The robot detects the presence of the objects in the area using one or more sensors. A determination is made that an object detected by the robot is not recognized by the robot. In response to determining that the object is not recognized by the robot, a user interface that includes data describing the object that is not recognized is provided. The user interface can be configured to receive user input that identifies the object. In response to interaction with the user interface, data identifying the object is received. A database for the robot is updated with the data identifying the object.

The present disclosure discloses an autonomous mobile grabbing method for a mechanical arm based on visual-haptic fusion under a complex illumination condition, which mainly includes approaching control over a target position and feedback control over environment information. According to the method, under the complex illumination condition, weighted fusion is conducted on visible light and depth images of a preselected region, identification and positioning of a target object are completed based on a deep neural network, and a mobile mechanical arm is driven to continuously approach the target object; in addition, the pose of the mechanical arm is adjusted according to contact force information of a sensor module, the external environment and the target object; and meanwhile, visual information and haptic information of the target object are fused, and the optimal grabbing pose and the appropriate grabbing force of the target object are selected. By adopting the method, the object positioning precision and the grabbing accuracy are improved, the collision damage and instability of the mechanical arm are effectively prevented, and the harmful deformation of the grabbed object is reduced.

Embodiments disclosed herein relate to methods and apparatus for managing a sensor grid system including configuring and using sensor grids for various tasks. In one embodiment there is provided a method of configuring a sensor grid system having a plurality of sensors arranged to collect data from a working space. The method includes applying an output from the sensors to a task model for performing a task associated with the working space. A task accuracy parameter is corresponding to the accuracy with which the task model performs the task is determine. In response to the task accuracy parameter being below a task accuracy parameter threshold, the resolution of the output from the sensors is increased, and the complexity of the task model is increased.

Methods, systems, and apparatus, including computer programs encoded on computer storage media for evaluating robot learning. In some implementations, a system receives classification examples from a plurality of remote devices over a communication network. The classification examples can include (i) a data representation generated by a remote device based on sensor data captured by the remote device and (ii) a classification corresponding to the data representation. The system assigns quality scores to the classification examples based on a level of similarity of the data representations with other data representations. The system selects a subset of the classification examples based on the quality scores assigned to the classification examples. The system trains a machine learning model using the selected subset of the classification examples.

A robotic manipulation planning system, including at least one processor; and a non-transitory computer-readable storage medium including instructions that, when executed by the at least one processor, cause the at least one processor to: process perception data to detect known, familiar, and unknown objects to generate manipulation candidates; filter manipulation candidates against constraints to reduce the manipulation candidates; and determine quality metrics for the reduced manipulation candidates using a soft-body simulation technique.

A classifying sensing system, a classifying method performed using a sensing system, a tactile sensor, and a method of fabricating a tactile sensor. The classifying sensing system comprises a first spiking neural network, SNN, encoder configured for encoding an event-based output of a vision sensor into individual vision modality spiking representations with a first output size; a second SNN encoder configured for encoding an event-based output of a tactile sensor into individual tactile modality spiking representations with a second output size; a combination layer configured for merging the vision modality spiking representations and the tactile modality spiking representations; and a task SNN configured to receive the merged vision modality spiking representations and tactile modality spiking representations and output vision-tactile modality spiking representations with a third output size for classification.

A machine which repetitively performs an operation, or operations, on mass-produced parts which are subject to part-to-part variations compensates for such variations by self-adjustment of its operation, or operations, at a location, or locations, where an operation, or operations, is, or are, performed.

A machine which repetitively performs an operation, or operations, on mass-produced parts which are subject to part-to-part variations compensates for such variations by self-adjustment of its operation, or operations, at a location, or locations, where an operation, or operations, is, or are, performed.

A method of safely operating machinery in a workspace includes recording images of a portion of a workspace. The method also includes generating a three-dimensional (3D) representation of the portion of the workspace based on the recorded images, where the 3D representation includes one or more volumes that correspond to the portion of the workspace. Additionally, the method includes identifying one or more of the volumes as being either occupied or unoccupied. Further, the method includes mapping one or more safe zones based on the one or more identified volumes, where the safe zones correspond to one or more regions within the portion of the workspace for safe operation of machinery.