System for automatically manipulating primary packaging in secondary packaging, comprising a robot having at least one robot arm with a clamping gripper installed at a tool centre point, wherein each tool centre point has a force-torque sensor, an image recording module for recording images of at least the upper segment of the primary packaging, comprising at least two stereo cameras for recording 3-D images, and one or more processors for providing a three-dimensional point cloud, controlling the image recording module and controlling the robot on the basis of the analysis of the three-dimensional point cloud and the measurements from the force-torque sensors.

The present disclosure discloses an autonomous mobile grabbing method for a mechanical arm based on visual-haptic fusion under a complex illumination condition, which mainly includes approaching control over a target position and feedback control over environment information.

According to the method, under the complex illumination condition, weighted fusion is conducted on visible light and depth images of a preselected region, identification and positioning of a target object are completed based on a deep neural network, and a mobile mechanical arm is driven to continuously approach the target object; in addition, the pose of the mechanical arm is adjusted according to contact force information of a sensor module, the external environment and the target object; and meanwhile, visual information and haptic information of the target object are fused, and the optimal grabbing pose and the appropriate grabbing force of the target object are selected.

By adopting the method, the object positioning precision and the grabbing accuracy are improved, the collision damage and instability of the mechanical arm are effectively prevented, and the harmful deformation of the grabbed object is reduced.

According to some embodiments, a tactile information estimation apparatus may include one or more memories and one or more processors. The one or more processors are configured to input at least first visual information of an object acquired by a visual sensor to a model. The model is generated based on visual information and tactile information linked to the visual information. The one or more processors are configured to extract, based on the model, a feature amount relating to tactile information of the object.

Production system for processing workpieces, having a robot module, a workpiece carrier module and a machining module, all of them being working modules, wherein each of said working modules includes an interface surface, the interface surface having a supply interface and a communication interface, wherein the robot module includes a robot and a robot controller for handling workpieces, wherein the workpiece carrier module includes a plurality of workpiece locations for receiving unmachined and finished workpieces, wherein the machining module includes a processing system for carrying out at least one processing operation on at least one workpiece; wherein a data carrier is assigned to each of the working modules, which a data carrier stores processing data, the processing data including a transfer position for workpieces and being coded for processing in the robot controller of the robot module.

A robotic manipulator includes one or multiple end effectors that can engage with an object, and one or multiple cameras that simultaneously observe each end effector, and the surrounding environment. For example, an end effector can include a contact surface including tactile markers which can deform when the end effector contacts the object.

A tactile sensor including a cap having a top surface and an undersurface. The undersurface includes pins, each pin has a mark. A portion of the undersurface is attachable to a device. A camera positioned in view of the marks, captures images of the marks placed in motion by elastic deformation of the top surface of the cap. A processor receives the captured images and determines a set of relative positions of the marks in the captured images, by identifying measured image coordinates of locations in images of the captured images. Determine a net force tensor acting on the top surface using a stored machine vision algorithm, by matching the set of relative positions of the marks to a stored set of previously learned relative positions of the marks placed in motion. Control the device via a controller in response to the net force tensor determined in the processor.

A system and method for identifying a shape and pose of an object is provided. The system includes a controller and a grasping device. The grasping device includes a plurality of fingers that are moveable relative to a base. The fingers include one or more tactile sensors attached thereto. The tactile sensors are configured to collect data, such as a 2D image or 3D point cloud, based on points of contact with the object when the object is grasped. The grasping device is configured to roll the object within the fingers and collect additional data. The controller may combine the data and determine the shape and pose of the object based on the combined data.

A tactile and/or optical distance sensor includes a housing, which has at least one elongate portion, a measurement arm, which is arranged in the housing, at least partially extends through the elongate portion and has a tactile and/or an optical probe element at one end, a transducer, which is configured to capture a position of the tactile probe element or a signal of the optical probe element and to generate associated probe element measurement signals, and an advance unit, with which the housing is linearly dis-placeable along an advance direction. A strain sensor is located in the region of the measurement arm extending through the elongate portion or at an adjacent region directly adjoining said region. In addition, a system for measuring the roughness of a surface of a workpiece and a method for calibrating a distance sensor or a system are provided.

A position and an orientation of an object are measured with high accuracy. An approximate position-orientation of a target object is obtained, positional information of the target object is obtained by measuring the target object using a noncontact sensor, positional information of contact positions touched by a contact sensor is obtained by bringing the contact sensor into contact with the target object, and a position-orientation of the target object is obtained by associating shape information of the target object with the positional information of the target object and the positional information of the contact positions in accordance with the approximate position-orientation.

An estimation apparatus includes: an acquisition section that acquires a measurement result of a measurement unit that measures an object to be an estimation target of a contact sense in a contactless manner; a determination section that makes a determination as to an aspect of the object or a measurement condition of the object on a basis of the measurement result of the measurement unit; a selection section that selects, on a basis of a result of the determination, an estimation scheme to be used for estimation of the contact sense of the object from among a plurality of estimation schemes; and an estimation section that estimates the contact sense of the object using the selected estimation scheme.