An automated drywalling system network that including one or more automated drywalling systems that each has a robotic arm. The automated drywalling system network can also include a computational planner that generates instructions for the one or more automated drywalling systems to perform two or more drywalling tasks associated with a target wall assembly. The two or more drywalling tasks can include a hanging task that includes hanging pieces of drywall on studs of the target wall assembly; a mudding task that includes applying joint compound to pieces of drywall hung on studs of the target wall assembly; a sanding task that includes sanding joint compound applied to the pieces of drywall hung on studs of the target wall assembly; and a painting task that includes painting sanded the joint compound applied to the pieces of drywall hung on studs of the target wall assembly.

An automated drywalling system for cutting and/or hanging drywall that includes one or more vision systems and a computing device executing a computational planner. The computational planner obtains target wall assembly data from the one or more vision systems, the target wall assembly data including information regarding a configuration of a target wall assembly including a plurality of studs that define a portion of the target wall assembly; and generates a plan for a configuration of a plurality of drywall pieces to be disposed on studs of the target wall assembly based on the target wall assembly data.

An automated drywalling system network that include one or more automated drywalling systems that each comprise a robotic arm. The automated drywalling system network can also include one or more drywalling end effectors configured to couple to a distal end of the robotic arms of the one or more automated drywalling systems. The drywalling end effectors can include a drywall hanging end effector; a drywall mudding end effector; a drywalling sanding end effector; and a drywall painting end effector.

An automated drywalling system network that including one or more automated drywalling systems that each has a robotic arm. The automated drywalling system network can also include a computational planner that generates instructions for the one or more automated drywalling systems to perform two or more drywalling tasks associated with a target wall assembly. The two or more drywalling tasks can include a hanging task that includes hanging pieces of drywall on studs of the target wall assembly; a mudding task that includes applying joint compound to pieces of drywall hung on studs of the target wall assembly; a sanding task that includes sanding joint compound applied to the pieces of drywall hung on studs of the target wall assembly; and a painting task that includes painting sanded the joint compound applied to the pieces of drywall hung on studs of the target wall assembly.

An automated drywalling system network that includes one or more automated drywalling systems that each has a robotic arm. The automated drywalling system network can also include a computational planner that generates instructions for the one or more automated drywalling systems to perform two or more drywalling tasks associated with a target wall assembly. The two or more drywalling tasks can include a hanging task that includes hanging pieces of drywall on studs of the target wall assembly; a mudding task that includes applying joint compound to pieces of drywall hung on studs of the target wall assembly; a sanding task that includes sanding joint compound applied to the pieces of drywall hung on studs of the target wall assembly; and a painting task that includes painting sanded the joint compound applied to the pieces of drywall hung on studs of the target wall assembly.

The information processing device 1B mainly includes an abstract model information acquisition unit 34X, a measurement information acquisition unit 34Y, and an abstract model generation unit 34Z. The abstract model information acquisition unit 34X is configured to acquire abstract model information I5 regarding an abstract model in which dynamics in a workspace 6 where a robot 5 performs an objective task is abstracted. The measurement information acquisition unit 34Y is configured to acquire measurement information Im indicating a measurement result in the workspace 6. The abstract model generation unit 34Z is configured to generate an abstract model based on the abstract model information I5 and the measurement information Im.

A method and system for task planning for visual room rearrangement under partial observability is disclosed. The system or the robotic agent utilizes a visual input to efficiently plan a sequence of actions for simultaneous object search and rearrangement in an untidy room, to achieve a desired tidy state. Unlike search networks in the art that follow ad hoc approach, the method discloses a search network utilizing commonsense knowledge from large language models to find unseen objects. A Deep RL network used for task planning is trained with proxy reward, along with unique graph-based state representation to produce a scalable and effective planner that interleaves object search and rearrangement to minimize the number of steps taken and overall traversal of the agent, and to resolve blocked goal and swap cases. Sample efficient cluster-biased sampling is utilized for simultaneous training of the proxy reward network along with the Deep RL network.

An automated painting system that includes a robotic arm and a painting end effector coupled at a distal end of the robotic arm, with the painting end effector configured to apply paint to a target surface. The painting system can also include a computing device executing a computational planner that: generates instructions for driving the painting end effector and robotic arm to perform at least one painting task that includes applying paint, via the painting the end effector, to a plurality of drywall pieces, the generating based at least in part on obtained target surface data; and drives the end effector and robotic arm to perform the at least one painting task.

A computer-implemented method of determining actions for controlling a robot, in particular an assembly robot, includes (i) receiving a first and second input, wherein the first input is a sentence describing an action which should be carried out by the robot, wherein the second input is an image of a current state of an environment of the robot, (ii) feeding the first input into a first machine learning model and feeding the second input into a second machine learning model, wherein the first and second machine learning models are configured to determine tokens for their respective inputs, and (iv) feeding the tokens into a third machine learning model, wherein the third machine learning model outputs two outputs, wherein the first output is a switch for incorporating specialized skill networks and the second output are actions.

An automated painting system that includes a robotic arm and a painting end effector coupled at a distal end of the robotic arm, with the painting end effector configured to apply paint to a target surface. The painting system can also include a computing device executing a computational planner that: generates instructions for driving the painting end effector and robotic arm to perform at least one painting task that includes applying paint, via the painting the end effector, to a plurality of drywall pieces, the generating based at least in part on obtained target surface data; and drives the end effector and robotic arm to perform the at least one painting task.