A user interface device includes a frame, rigid body in rotary engagement with the frame, a plurality of force sensors, which in response to force acting on the frame produces information that represents a first force component in a first direction with respect to the frame, and a second force component in a second direction with respect to the frame; and an angle sensor, which in response to torque applied to the rigid body produces information that represents rotary movement of the rigid body with respect to the frame about an axis extending vertically through the rigid body. The rigid body can be sized and dimensioned to accommodate one or more feet of a user. The user interface device advantageously combines or mixes isometric and isotonic control input or sensors.

A robot teaching and testing system and method that performs human-operated robot tasks according to instructions generated from generative AI models. The process starts with a user prompt and combines the user prompt with predefined prompt templates to generate well-formatted text prompts. Generative AI models take the text prompts and convert them into high-level instructions or control codes that can be deployed on a robot. The high-level instructions are then converted into human-operated robot tasks for a human data collector using a mixed reality (MR) device. The human data collector will attempt to follow the instructions to complete the human-operated robot tasks and may overwrite the suggested instructions by performing a different action, demonstrate a task without instructions, or leave feedback or comments regarding the tasks. Feedback data will be captured and saved for improving the robot system.

A user interface device includes a frame, rigid body in rotary engagement with the frame, a plurality of force sensors, which in response to force acting on the frame produces information that represents a first force component in a first direction with respect to the frame, and a second force component in a second direction with respect to the frame; and an angle sensor, which in response to torque applied to the rigid body produces information that represents rotary movement of the rigid body with respect to the frame about an axis extending vertically through the rigid body. The rigid body can be sized and dimensioned to accommodate one or more feet of a user. The user interface device advantageously combines or mixes isometric and isotonic control input or sensors.

A method for training and/or testing a robot control module. The method includes generating an instruction specified by a robot control module configured for robot training and/or testing, the instruction indicating how a human-driven robot task is to be performed when training and/or testing the robot control module; providing the instruction to a mixed reality device worn by a human data collector, the mixed device rendering the instruction in a manner that shows the human data collector how to perform the human-driven robot task; collecting performance data and environmental data in response to the human data collector attempting to perform the human-driven robot task using the data collection device; receiving feedback data in response to the human data collector attempting to perform the human-driven robot task specified by the instruction; and updating the robot control module using the feedback data and the collected performance and environmental data.

A method and apparatus includes a monitor in communication with, e.g., a work glove with built-in sensors to track an operator's physical activities including intermittently using hand tools, hand motion, and position or location in a work area. The sensors provide time-series data representative of the activities, and the monitor classifies and sequences the activities based on the time-series data along with, optionally, machine tool operational data. The classified and sequenced activities may then compared to instructed work procedures, ensuring correct work materials are used in production, and ensuring an operator's safety by detecting unsafe or abnormal work conditions.