A smart, human-centered technique that uses artificial intelligence and mixed reality to accelerate essential tasks of the inspectors such as defect measurement, condition assessment and data processing. For example, a bridge inspector can analyze some remote cracks located on a concrete pier, estimate their dimensional properties and perform condition assessment in real-time. The inspector can intervene in any step of the analysis/assessment and correct the operations of the artificial intelligence. Thereby, the inspector and the artificial intelligence will collaborate/communicate for improved visual inspection. This collective intelligence framework can be integrated in a mixed reality supported see-through headset or a hand-held device with the availability of sufficient hardware and sensors. Consequently, the methods reduce the inspection time and associated labor costs while ensuring reliable and objective infrastructure evaluation. Such methods offer contributions to infrastructure inspection, maintenance, management practice, and safety for the inspection personnel.

A method for tracking a lane on a road is presented. The method comprises receiving, by one or more processors from an imaging system, a set of pixels associated with lane markings. The method further includes generating, by the one or more processors, a predicted spline comprising (i) a first spline and (ii) a predicted extension of the first spline in a direction in which the imaging system is moving. The first spline describes a boundary of a lane and is generated based on the set of pixels. The predicted extension of the first spline is generated based at least in part on a curvature of at least a portion of the first spline.

Various implementations disclosed herein include devices, systems, and methods that generate floorplans and measurements using a three-dimensional (3D) representation of a physical environment generated based on sensor data.

An object tracking, in particular adapted for real-time augmented reality applications, involves determining a location of an object (20) in a current frame (10) of a video stream (15), at a point in time following output of a preceding frame (11) of the video stream (15) but preceding output of the current frame (10), by starting from a location of the object (20) determined by an object-detection server (5) for a previous frame (12) of the video stream (15) and recursively track the location of the object (20) in frames (11) of the video stream (15) following the previous frame (12) up to the current frame (10) and recursively update a model of the object (20). Accurate objection detection from an object-detection server (5) can thereby be used even if the object was detected in a past frame (12) of the video stream (15) that has already been visualized.

Systems, methods, and non-transitory computer-readable media can identify a virtual character being presented to a user within a real-time immersive environment. A first animation to be applied to the virtual character is determined. A nonverbal communication animation to be applied to the virtual character simultaneously with the first animation is determined. The virtual character is animated in real-time based on the first animation and the nonverbal communication animation.

Disclosed are systems and methods for improving interactions with and between computers in content searching, hosting and/or providing systems supported by or configured with devices, servers and/or platforms. The disclosed systems and methods provide an image processing framework that sub-divides computer vision techniques into three computationally efficient steps: detection, classification and matching. These steps provide an improved image processing framework that can analyze live stream data of a media file, in real-time, in order to identify and track specific digital objects depicted therein. This enables not only image processing detection results, but also the capabilities of augmenting the video stream with additional data related to the detected object.

Described herein are systems, methods, and non-transitory computer readable media for generating fused sensor data through metadata association. First sensor data captured by a first vehicle sensor and second sensor data captured by a second vehicle sensor are associated with first metadata and second metadata, respectively, to obtain labeled first sensor data and labeled second sensor data. A frame synchronization is performed between the first sensor data and the second sensor data to obtain a set of synchronized frames, where each synchronized frame includes a portion of the first sensor data and a corresponding portion of the second sensor data. For each frame in the set of synchronized frames, a metadata association algorithm is executed on the labeled first sensor data and the labeled second sensor data to generate fused sensor data that identifies associations between the first metadata and the second metadata.

An apparatus for delivering virtual reality data portions to a client device, including a processing unit configured to perform the following in each one of a plurality of iterations: (1) receive from a network a current orientation data indicating a current orientation of a client device, (2) apply a rotation to a segment of a sphere defined in a virtual reality (VR) video file according to the current orientation, (3) crop from the rotated segment of the sphere in an equirectangular projection format an extended field of view (EFOV) frame in the equirectangular projection format according to the current orientation, and (4) instruct the network to transmit the EFOV frame to the client device.

Systems, methods and media are disclosed for identifying the crossing of a virtual barrier. A person in a 3D image of a room may be circumscribed by a bounding box. The position of the bounding box may be monitored over time, relative to the virtual barrier. If the bounding box touches or crosses the virtual barrier, an alert may be sent to the person being monitored, a caregiver or a clinician. Bounding box tracking may be used in addition to or instead of an initial tracking process, such as skeletal tracking.

A method can perform a process with a method including capturing an image, determining an environment that a user is operating a computing device, detecting a hand gesture based on an object in the image, determining, using a machine learned model, an intent of a user based on the hand gesture and the environment, and executing a task based at least on the determined intent.