A technique for localization of an unmanned aerial vehicle (UAV) includes: acquiring aerial images of a terrain below the UAV with an onboard camera system of the UAV while the UAV is flying a mission along a preplanned route over the terrain; generating a current optical flow map based upon image pixel motion between consecutive images in a sequence of the aerial images; comparing the current optical flow map to reference optical flow maps stored onboard the UAV, wherein the reference optical flow maps are precomputed from a model of the terrain along the preplanned route; and determining a position in at least two lateral dimensions based on the comparing.

A method for unmanned aerial vehicle (UAV) mission planning includes acquiring a target aerial image of a geographic area representative of the geographic area illuminated by one or more artificial light sources, identifying a location of the one or more artificial light sources based on the target aerial image, rendering a simulated aerial image representative of the geographic area illuminated by the one or more artificial light sources at night using a digital surface model of the geographic area, the location of the one or more artificial light sources, and an irradiance parameter for the one or more artificial light sources, identifying one or more regions within the geographic area as having sufficient lighting for UAV operation at night based on the simulated aerial image, and generating a mission plan for the UAV based on the one or more regions within the geographic area.

A waypoint correction device and waypoint correction method are provided. In response to a positioning signal of an unmanned aerial vehicle at a predetermined waypoint, the waypoint correction device obtains a real-time image from the unmanned aerial vehicle. The waypoint correction device calculates a feature point distribution in the real-time image based on the real-time image. The device generates an adjusted viewing angle signal based on the feature point distribution to control the unmanned aerial vehicle to rotate in place based on the adjusted viewing angle signal and capture an adjusted real-time image. The device generates a correction route based on a plurality of three-dimensional feature points, the adjusted real-time image, and a sampling number threshold to control the unmanned aerial vehicle to move from an actual position to a predetermined waypoint based on the correction route.

A mapping system for an autonomous mobile robot includes a 3D convolutional encoder network that generates 3D feature maps from 3D point cloud data. The network sequentially compresses the feature dimension of the 3D input data to reduce the computational complexity and enable feature extraction to be performed in substantially real-time. Skip connections connect the outputs of the encoder layers of the convolutional encoder network to counterpart decoder layers of a 2D convolutional decoder network. An attention-based 3D to 2D projection layer receives the 3D feature maps generated by the encoder layers via the skip connections and projects the 3D feature maps onto 2D BEV feature maps which are provided to the counterpart decoder layers as input. The projection layer automatically estimates ground level of 3D feature maps and filters out overhanging objects that are irrelevant to ground-level navigation.

Disclosed are apparatuses, systems, and techniques that train and use trained language models to assist users with complex systems installation, troubleshooting, and/or maintenance. A method can include determining, responsive to data received from a real robot having one or more real sensors and operating in a real environment, that the real robot needs assistance to navigate from a current state of the real robot within the real environment, causing simulated data to be obtained from one or more simulated sensors within a simulated environment at least partially modeling the real environment, the one or more simulated sensors including at least one simulated sensor different from the one or more real sensors, and using the simulated data to control operation of the real robot within the real environment in order to navigate the real robot from the current state.

A map generation system generates map data for an agricultural machine to automatically travel on a road around a field, and includes a storage to store feature block images associated with different types of road features, and a processor configured or programmed to acquire position distribution data on one or more types of road features, the position distribution data being generated based on at least one of sensor data from a LiDAR sensor and image data from an imager output while a movable body including at least one of the LiDAR sensor and the imager is moving along the road; read from the storage one or more types of feature block images associated with the one or more types of features; and align the feature block images in accordance with the position distribution data to generate map data on a region including the road.

A system and method are provided for mapping obstructions in a work area traversed by a work machine such as a sprayer, combine or other machine having a ground-engaging work implement. While the work machine traverses the work area, and via output signals from sensors associated with the work machine, obstructions are detected at (e.g., using perception sensing on a combine, etc.) and/or below (e.g., using vibration sensing on a planter, dozer, etc.) a ground surface. A mapped data structure associated with the work area is accordingly modified, wherein a sensed location and one or more identified characteristics for each respective one of the detected obstructions are mapped to corresponding locations in the mapped data structure. The sensors may include vibration sensors or implement actuator position sensors to detect an obstruction contacted by the work machine, and/or perception sensors to detect obstructions on the surface/within a field of view.

Systems and methods for performing real-time multi-robot collaboration in dynamic environments are provided. A system may obtain sensor data of an environment of the robot, and generate tokenized sensor data from the sensor data. The system may input the tokenized sensor data into a robotics foundational model (RFM) associated with the robot causing the RFM to generate insight data used for making decisions associated with performing the mission. Generating the insight data includes generating one or more beliefs about the environment and generating one or more risk-reward maps indicating potential risks and/or a. The system implement a token sharing policy causing the robot to generate tokenized insight data and transmit the tokenized insight data to recipient robots of the robot fleet.

A cleaning robot for sensing and cleaning a floor is provided. The cleaning robot includes a floor sensing sensor positioned in front of the cleaning robot in a driving direction of the cleaning robot, and configured to sense whether a surface to be cleaned is a carpet or a hard floor, a wet cleaning module positioned behind the floor sensing sensor in the driving direction, and configured to lower a pad for cleaning the hard floor, and at least one processor configured to control the wet cleaning module to lift the pad while the cleaning robot is driving on the carpet, control the wet cleaning module to lower the pad based on a reference distance moved in the driving direction after sensing the hard floor through the floor sensing sensor when the cleaning robot moves from the carpet to the hard floor along a driving path.

A navigation device and method based on multimodal information are provided. The navigation device includes a storage device configured to store a plurality of waypoint images and a plurality of first movement reference instructions previously generated by using a database, a memory and a processor. The navigation method includes a step of receiving an observed image, a step of extracting a goal image from among the plurality of waypoint images stored in the database and extracting a second movement reference instruction applied to autonomous driving of a robot from among the plurality of first movement reference instructions stored in the database, based on the observed image, and a step of inputting the observed image, the goal image, and the second movement reference instruction to a pre-trained autonomous driving path generation model to generate an autonomous driving path and motion to be applied to the robot.