Efficient and effective workspace condition analysis systems and methods are presented. In one embodiment, a method comprises: accessing information associated with an activity space, including information on a newly discovered previously unmodeled entity; analyzing the activity information, including activity information associated with the previously unmodeled entity; forwarding feedback on the results of the analysis, including analysis results for the updated modeled information; and utilizing the feedback in a coordinated path plan check process. In one exemplary implementation the coordinated path plan check process comprises: creating a solid/CAD model including updated modeled information; simulating an activity including the updated modeled information; generating a coordinated path plan for entities in the activity space; and testing the coordinated path plan. The coordinated path plan check process can be a success. The analyzing can include automatic identification of potential collision points for a first actor, including potential collision points with the newly discovered object. The newly discovered previously unmodeled entity interferes with an actor from performing an activity. The newly discovered object is a portion of a tool component of a product.

Efficient and effective workspace condition analysis systems and methods are presented. In one embodiment, a method comprises: accessing information associated with an activity space, including information on a newly discovered previously unmodeled entity; analyzing the activity information, including activity information associated with the previously unmodeled entity; forwarding feedback on the results of the analysis, including analysis results for the updated modeled information; and utilizing the feedback in a coordinated path plan check process. In one exemplary implementation the coordinated path plan check process comprises: creating a solid/CAD model including updated modeled information; simulating an activity including the updated modeled information; generating a coordinated path plan for entities in the activity space; and testing the coordinated path plan. The coordinated path plan check process can be a success. The analyzing can include automatic identification of potential collision points for a first actor, including potential collision points with the newly discovered object. The newly discovered previously unmodeled entity interferes with an actor from performing an activity. The newly discovered object is a portion of a tool component of a product.

A touch sensation sensor is mounted to a hand part of a robot and includes: an obtaining means, obtaining at least one of visual sensation information, which is target object information relating to a target object operated by using the hand part, and touch sensation information, which is the target object information at a time when the target object operated by using the hand part is gripped; and a control device, changing a sensitivity mode of the touch sensation sensor in accordance with the target object information that is obtained.

Broadly speaking, the present application generally relates to a method for training a machine learning, ML, model to perform real world cross-modal retrieval problems, and to a computer-implemented method and apparatus for performing real world cross-modal retrieval problems such as including text-based video retrieval, sketch-based image retrieval, and image-text retrieval using a trained machine learning, ML, model.

Systems and methods for training an artificial intelligence unit for an automated vehicle are provided. The artificial intelligence unit includes a knowledge configuration. The artificial intelligence unit determines an evaluation value for at least two motion actions for the automated vehicle that considers an input state and the knowledge configuration. The input state characterizes the automated vehicle and at least one other road user. The system selects one motion action from the at least two motion actions, considers the evaluation value of the respective motion actions, and trains the artificial intelligence unit by adapting the knowledge configuration of the artificial intelligence unit based on the selected motion action. The knowledge configuration characterizes at least the empowerment of the at least one other road user.

Methods, apparatus, and computer-readable media for determining and utilizing corrections to robot actions. Some implementations are directed to updating a local features model of a robot in response to determining a human correction of an action performed by the robot. The local features model is used to determine, based on an embedding generated over a corresponding neural network model, one or more features that are most similar to the generated embedding. Updating the local features model in response to a human correction can include updating a feature embedding, of the local features model, that corresponds to the human correction. Adjustment(s) to the features model can immediately improve robot performance without necessitating retraining of the corresponding neural network model.

Methods, apparatus, and computer-readable media for determining and utilizing corrections to robot actions. Some implementations are directed to updating a local features model of a robot in response to determining a human correction of an action performed by the robot. The local features model is used to determine, based on an embedding generated over a corresponding neural network model, one or more features that are most similar to the generated embedding. Updating the local features model in response to a human correction can include updating a feature embedding, of the local features model, that corresponds to the human correction. Adjustment(s) to the features model can immediately improve robot performance without necessitating retraining of the corresponding neural network model.

A system for behaviour mapping and categorization of objects and users in an 3D environment for creating and learning user behaviour map is provided. The system includes a robot 102, a network 104 and a central AI system 106. The robot 102 is embedded with an array of acoustic sensors 108 and visual sensors 110 for behaviour mapping and categorization the objects and users in the 3D environment and generates an auditory behaviour map and a visual behaviour map based on sensory inputs from the acoustic sensors 108 and visual sensors 110. The robot 102 transmits the acoustic source sensory input and the visual source sensory input to the central AI system 106 over the network 104 for generating a global behaviour map. The central AI system 106 tunes the global behaviour map to a specific user by tuning the detection and classification model to data obtained from a specific 3D environment that corresponds to the specific user.

Methods and systems for using a teleoperation system to train a robot to perform tasks using machine learning are described herein. A teleoperation system may be used to record actions of a robot as used by a human teleoperator. The teleoperation system may provide a teleoperator insight into the state of the robot and may provide feedback to the teleoperator allowing the teleoperator to feel what the robot is feeling. For example, sensor information from the robot may be sent to the teleoperation system and output to the teleoperator in various forms including vibrations, video, visual cues, or sound. The teleoperation system may output visual guides to the teleoperator so that the teleoperator may know how to control the robot to complete a task in a desired manner.

Methods and systems for using a teleoperation system to train a robot to perform tasks using machine learning are described herein. A teleoperation system may be used to record actions of a robot as used by a human teleoperator. The teleoperation system may provide a teleoperator insight into the state of the robot and may provide feedback to the teleoperator allowing the teleoperator to feel what the robot is feeling. For example, sensor information from the robot may be sent to the teleoperation system and output to the teleoperator in various forms including vibrations, video, visual cues, or sound. The teleoperation system may output visual guides to the teleoperator so that the teleoperator may know how to control the robot to complete a task in a desired manner.