An autonomous control system includes an acquirer configured to acquire state data of a robot, visual data of the robot, and tactile data of the robot and a processor configured to decide on an action of the robot capable of accomplishing a task given to the robot on the basis of the state data, the visual data, and the tactile data. The processor generates first compressed data having a smaller number of dimensions than data obtained by combining the visual data and the tactile data by fusing and dimensionally compressing the visual data and the tactile data. The processor generates second compressed data having a smaller number of dimensions than the tactile data by dimensionally compressing the tactile data. The processor decides on the action on the basis of combined state data obtained by combining the state data, the first compressed data, and the second compressed data into one.

Disclosed are various embodiments of a three-dimensional perception and object manipulation robot gripper configured for connection to and operation in conjunction with a robot arm. In some embodiments, the gripper comprises a palm, a plurality of motors or actuators operably connected to the palm, a mechanical manipulation system operably connected to the palm, a plurality of fingers operably connected to the motors or actuators and configured to manipulate one or more objects located within a workspace or target volume that can be accessed by the fingers. A depth camera system is also operably connected to the palm. One or more computing devices are operably connected to the depth camera and are configured and programmed to process images provided by the depth camera system to determine the location and orientation of the one or more objects within a workspace, and in accordance therewith, provide as outputs therefrom control signals or instructions configured to be employed by the motors or actuators to control movement and operation of the plurality of fingers so as to permit the fingers to manipulate the one or more objects located within the workspace or target volume. The gripper can also be configured to vary controllably at least one of a force, a torque, a stiffness, and a compliance applied by one or more of the plurality of fingers to the one or more objects.

A robot system includes a robot, an image capture apparatus, an image processing portion, and a control portion. The image processing portion is configured to specify in an image of a plurality of objects captured by the image capture apparatus, at least one area in which a predetermined object having a predetermined posture exists, and obtain information on position and/or posture of the predetermined object in the area. The control portion is configured to control the robot, based on the information on position and/or posture of the predetermined object, for the robot to hold the predetermined object.

One variation of a method for monitoring growth of plants within a facility includes: aggregating global ambient data recorded by a suite of fixed sensors, arranged proximal a grow area within the facility, at a first frequency during a grow period; extracting interim outcomes of a set of plants, occupying a module in the grow area, from module-level images recorded by a mover at a second frequency less than the first frequency while interfacing with the module during the period of time; dispatching the mover to autonomously deliver the module to a transfer station; extracting interim outcomes of the set of plants from plant-level images recorded by the transfer station while sequentially transferring plants out of the module at the conclusion of the grow period; and deriving relationships between ambient conditions, interim outcomes, and final outcomes from a corpus of plant records associated with plants grown in the facility.

Mechanisms are provided for classifying an obstacle as an asset type. The mechanisms receive a digital image of an obstacle from an image capture device of an automated robot. The mechanisms perform a classification operation on the digital image of the obstacle to identify a proposed asset type classification for the obstacle. The mechanisms determine a final asset type for the obstacle based on the proposed asset type classification for the obstacle. The mechanisms update a map data structure for a physical premises in which the obstacle is present based on the final asset type.

Disclosed are various embodiments of a three-dimensional perception and object manipulation robot gripper configured for connection to and operation in conjunction with a robot arm. In some embodiments, the gripper comprises a palm, a plurality of motors or actuators operably connected to the palm, a mechanical manipulation system operably connected to the palm, a plurality of fingers operably connected to the motors or actuators and configured to manipulate one or more objects located within a workspace or target volume that can be accessed by the fingers. A depth camera system is also operably connected to the palm. One or more computing devices are operably connected to the depth camera and are configured and programmed to process images provided by the depth camera system to determine the location and orientation of the one or more objects within a workspace, and in accordance therewith, provide as outputs therefrom control signals or instructions configured to be employed by the motors or actuators to control movement and operation of the plurality of fingers so as to permit the fingers to manipulate the one or more objects located within the workspace or target volume. The gripper can also be configured to vary controllably at least one of a force, a torque, a stiffness, and a compliance applied by one or more of the plurality of fingers to the one or more objects.

A control device includes a first statistical processing unit a second statistical processing unit and a movement control unit. The first statistical processing unit acquires relative positions of the object calculated by the visual sensor and performs statistical processing on the acquired relative positions of the object. The second statistical processing unit acquires from the position sensor relative positions of the holding device corresponding to each of the relative positions of the object calculated by the visual sensor, and performs statistical processing on the acquired relative positions of the holding device. The movement control unit performs feedback control of the moving device based on the relative positions of the object and the relative positions of the holding, device and performs alignment of the object with the target position while moving the object closer to the target position.

An application processor processes an application. A sensor processor acquires image data from an image sensor and analyzes the image data. A motion controlling processor controls motion of a movable part of a robot. The motion controlling processor provides posture information for specifying an orientation of the image sensor to the sensor processor, not via the application processor. The posture information includes information for specifying a position of the image sensor.

One variation of a method for monitoring growth of plants within a facility includes: aggregating global ambient data recorded by a suite of fixed sensors, arranged proximal a grow area within the facility, at a first frequency during a grow period; extracting intermediate outcomes of a set of plants, occupying a module in the grow area, from module-level images recorded by a mover at a second frequency less than the first frequency while interfacing with the module during the period of time; dispatching the mover to autonomously deliver the module to a transfer station; extracting intermediate outcomes of the set of plants from plant-level images recorded by the transfer station while sequentially transferring plants out of the module at the conclusion of the grow period; and deriving relationships between ambient conditions, intermediate outcomes, and final outcomes from a corpus of plant records associated with plants grown in the facility.

A control system 4 includes an application processor that executes a first operating system to process an application, a sensor processor that executes a second operating system to process image data acquired by an image sensor, and a motion controlling processor that executes a third operating system to control motion of a movable part of a robot. The first operating system, the second operating system, and the third operating system are different from each other.