The present disclosure discloses an autonomous mobile grabbing method for a mechanical arm based on visual-haptic fusion under a complex illumination condition, which mainly includes approaching control over a target position and feedback control over environment information.

According to the method, under the complex illumination condition, weighted fusion is conducted on visible light and depth images of a preselected region, identification and positioning of a target object are completed based on a deep neural network, and a mobile mechanical arm is driven to continuously approach the target object; in addition, the pose of the mechanical arm is adjusted according to contact force information of a sensor module, the external environment and the target object; and meanwhile, visual information and haptic information of the target object are fused, and the optimal grabbing pose and the appropriate grabbing force of the target object are selected.

By adopting the method, the object positioning precision and the grabbing accuracy are improved, the collision damage and instability of the mechanical arm are effectively prevented, and the harmful deformation of the grabbed object is reduced.

According to some embodiments, a tactile information estimation apparatus may include one or more memories and one or more processors. The one or more processors are configured to input at least first visual information of an object acquired by a visual sensor to a model. The model is generated based on visual information and tactile information linked to the visual information. The one or more processors are configured to extract, based on the model, a feature amount relating to tactile information of the object.

Methods and apparatus related to receiving a request that includes robot instructions and/or environmental parameters, operating each of a plurality of robots based on the robot instructions and/or in an environment configured based on the environmental parameters, and storing data generated by the robots during the operating. In some implementations, at least part of the stored data that is generated by the robots is provided in response to the request and/or additional data that is generated based on the stored data is provided in response to the request.

Techniques are described herein for generating a dynamical systems control policy. A non-parametric family of smooth maps is defined on which vector-field learning problems can be formulated and solved using convex optimization. In some implementations, techniques described herein address the problem of generating contracting vector fields for certifying stability of the dynamical systems arising in robotics applications, e.g., designing stable movement primitives. These learning problems may utilize a set of demonstration trajectories, one or more desired equilibria (e.g., a target point), and once or more statistics including at least an average velocity and average duration of the set of demonstration trajectories. The learned contracting vector fields may induce a contraction tube around a targeted trajectory for an end effector of the robot. In some implementations, the disclosed framework may use curl-free vector-valued Reproducing Kernel Hilbert Spaces.

A robot system includes: a robot including an end effector connected to an arm thereof; a vision sensor mounted to the robot; and a controller configured to output an operation signal that enables the robot to operate when an input is generated through a touch screen. Each of an object and a target to which the object is placed is inputted through the touch screen. The touch screen displays a recommendation region of the target in a distinguished manner.

A machine learning method includes: a first learning step which is performed in a phase before a neural network is installed in a mobile robot and in which a stationary first obstacle is placed in a set space and the first obstacle is placed at different positions using simulation so that the neural network repeatedly learns a path from a starting point to the destination which avoids the first obstacle; and a second learning step which is performed in a phase after the neural network is installed in the mobile robot and in which, when the mobile robot recognizes a second obstacle that operates around the mobile robot in a space where the mobile robot moves, the neural network repeatedly learns a path to the destination which avoids the second obstacle every time the mobile robot recognizes the second obstacle.

A robot control method, and associated robot controllers and robots operating with such methods and controllers, providing real-time vibration suppression. The control method involves learning to support real-time, vibration-suppressing control. The method uses state-of-the-art machine learning techniques in conjunction with a differentiable dynamics simulator to yield fast and accurate vibration suppression. Vibration suppression using offline simulation approaches that can be computationally expensive may be used to create training data for the controller, which may be provide by a variety of neural network configurations. In other cases, sensory feedback from sensors onboard the robot being controlled can be used to provide training data to account for wear of the robot's components.

A robot for making coffee and a method for controlling the same are provided to couple or decouple a portafilter to or from an espresso machine without damage to the espresso machine or the portafilter due to a collision between the espresso machine and the portafilter. The robot includes a robot arm to move with a predetermined degree of freedom, a gripper provided in the robot arm to grip a portafilter, a torque sensor provided in the robot arm to detect repulsive force (Fr) when the portafilter makes contact with a group head of an espresso machine, and a controller configured to set a virtual spring having a predetermined elastic modulus (C) based on the repulsive force (Fr) detected by the torque sensor, and to control driving torque (T) of the robot arm depending on the restoring force (Fe) of the virtual spring.

A software compensated robotic system makes use of recurrent neural networks and image processing to control operation and/or movement of an end effector. Images are used to compensate for variations in the response of the robotic system to command signals. This compensation allows for the use of components having lower reproducibility, precision and/or accuracy that would otherwise be practical.

Computerized engineering tool and methodology to develop neural skills for computerized autonomous systems, such as a robotics system (50), are provided. A disclosed computerized engineering tool (10) may involve an integrated arrangement of respective modular functionalities arranged in a closed loop, such as may include a physics engine (14), a neural data editor (16), an experiment editor (18), a neural skills editor (20), and a machine learning environment (22). Disclosed embodiments are conducive to cost-effectively simplifying development efforts involving neural skills, such as by reducing the time involved to develop the neural skills involved in any given robotics system and by reducing the level of expertise involved to develop neural skills.