A robot teaching and testing system and method that performs human-operated robot tasks according to instructions generated from generative AI models. The process starts with a user prompt and combines the user prompt with predefined prompt templates to generate well-formatted text prompts. Generative AI models take the text prompts and convert them into high-level instructions or control codes that can be deployed on a robot. The high-level instructions are then converted into human-operated robot tasks for a human data collector using a mixed reality (MR) device. The human data collector will attempt to follow the instructions to complete the human-operated robot tasks and may overwrite the suggested instructions by performing a different action, demonstrate a task without instructions, or leave feedback or comments regarding the tasks. Feedback data will be captured and saved for improving the robot system.

Aspects of the disclosure are directed towards artificial intelligence-based modeling of target objects, such as aircraft parts. In an example, a system initially trains a machine learning (ML) model based on synthetic images generated based on multi-dimensional representation of target objects. The same system or a different system subsequently further trains the ML model based on actual images generated by cameras positioned by robots relative to target objects. The ML model can be used to process an image generated by a camera positioned by a robot relative to a target object based on a multi-dimensional representation of the target object. The output of the ML model can indicate, for a detected target, position data, a target type, and/or a visual inspection property. This output can then be used to update the multi-dimensional representation, which is then used to perform robotics operations on the target object.

Aspects of the disclosure are directed towards path generation. A method includes a user interface (UI) displaying a first page on a first pane, wherein the first page provides a first control input for registering a working frame of a target object with a reference frame of a robot. The method further includes receiving, via the UI, a first user selection of the first control input for registering the working frame with the reference frame, based on detection of the first user selection. The UI can display a second page on the first pane, wherein the second page provides a second control input for generating a path for the robot to traverse over a surface of the target object. The method further includes receiving, via the UI, a second user selection of the second control input for generating the path, based on detection of the second user selection.

A digital process twin system and method for operating a process plant with at least one automation component to control an industrial process within the process plant with at least one input ingredient and at least one output product, wherein a non-real-time simulation model of the industrial process is used to generated quality attributes as a function of process variables and process parameters, the generated quality attributes and related process variables are used as an input for a machine learning model serving as a soft sensor to estimate quality attributes of the output product as a function of measured or simulated process variables of the industrial process, and the performance of the process plant is optimized based on the estimated quality attributes of the output product, whereby the method and system allow process operations and control that are faster, more efficient, and more reliable.

Implementations described herein relate to training and refining robotic control policies using imitation learning techniques. A robotic control policy can be initially trained based on human demonstrations of various robotic tasks. Further, the robotic control policy can be refined based on human interventions while a robot is performing a robotic task. In some implementations, the robotic control policy may determine whether the robot will fail in performance of the robotic task, and prompt a human to intervene in performance of the robotic task. In additional or alternative implementations, a representation of the sequence of actions can be visually rendered for presentation to the human can proactively intervene in performance of the robotic task.

The present disclosure relates to a multi-tiered computing environment for balancing compute resources in support of robot operations. In an example, a robot is tasked with performing an operation associated with an airplane having an airplane model. To do so, the robot may need another operation that is computationally complex to be performed. An on-premises server can execute a process that corresponds to this computationally-complex operation based on sensor data of the robot and can output the resulting data to the robot. Next, the robot can use the resulting data to execute another process corresponding to its operation and can indicate performance of this operation to the on-premises network. The on-premises network can send the indication about the operation performance to a top-tier server that is also associated with the airplane model.

The present application relates to an autonomous AI quality inspection system that uses robotics, AI and computer vision technology to measure and inspect various parameters of manufactured objects, including but not limited to fasteners such as nuts, bolts, screws, and nails. The autonomous system uses computer vision to capture raw images of the objects, inspect their key parameters, and display the quality assurance (QA) results on an interactive application dashboard and achieve a consistent precision level of up to 0.01 mm. The system comprises of an edge gateway connected to a high-resolution camera that captures images of the objects and a cloud server that detects the images, removes background of the objects, fetches the edges and points, calculates the key characteristics and thereby displays the computed data on a user dashboard for quality control check. The system can be adapted to different types of manufacturing objects by a one-time onboarding process.

A method using AI models to generate patterns or images on a pastry is provided. The method at least includes steps as follows: providing a UI module for user input of keywords and descriptions; generating black and white vector-style images based on this input using an AI-based image module; processing the 2D image into digital instructions via an image-transformer module; sending these instructions to a laser patterning module; and using the laser patterning module to apply the pattern to the cookie's surface.

The disclosed visual RRC-humanoid robot is a computer-based system that has been programmed to reach human-like levels of visualization Artificial Intelligence (AI). Behavioral-programming techniques are used to reach human-like levels of identification AI, recognition AI, visualization AI, and comprehension AI. The system is programmed to identify, recognize, visualize and comprehend the full array of sizes, distances, shapes, and colors of objects recorded in the FOV of the system. The following innovative features have been incorporated into the system: (i) incorporation of the RRC, (ii) incorporation of the Relational Correlation Sequencer (RCS): A proprietary RRC-module, (iii) a paradigm shift in the analytical-programming methodology employed in computer vision systems, (iv) incorporation of a central hub of intelligence, (v) design of a self knowledge capability and Internalization of all data, and (vi) design of an interface circuit compatible with human-like levels of visualization-AI.

The disclosed visual RRC-humanoid robot is a computer-based system that has been programmed to reach human-like levels of visualization Artificial Intelligence (AI). Behavioral-programming techniques are used to reach human-like levels of identification AI, recognition AI, visualization AI, and comprehension AI. The system is programmed to identify, recognize, visualize and comprehend the full array of sizes, distances, shapes, and colors of objects recorded in the FOV of the system. The following innovative features have been incorporated into the system: (i) incorporation of the RRC, (ii) incorporation of the Relational Correlation Sequencer (RCS): A proprietary RRC-module, (iii) a paradigm shift in the analytical-programming methodology employed in computer vision systems, (iv) incorporation of a central hub of intelligence, (v) design of a self knowledge capability and Internalization of all data, and (vi) design of an interface circuit compatible with human-like levels of visualization-AI.