A method includes receiving an image of a real environment captured using a camera worn by a user, the image comprising a hand of the user and determining a pose of the hand based on the image. Based on a three-dimensional model of the hand having the determined pose, generating a two-dimensional surface representing the hand as viewed from a first viewpoint of the user and positioning the two-dimensional surface representing the hand and one or more virtual-object representations in a three-dimensional space. The method further includes determining that a portion of the two-dimensional surface representing the hand is visible from a second viewpoint in the three-dimensional space, and generating an output image, wherein a set of image pixels of the output image corresponding to the portion of the two-dimensional surface that is visible is configured to cause a display to tur off a set of corresponding display pixels.

A method for low visibility driving includes receiving image data from a visible-light camera. The image data includes an image of an area in front of a vehicle. The method includes receiving sensor data from an object-detecting sensor. The object-detecting sensor is configured to detect an object in front of the vehicle. The sensor data includes information about the object in front of the vehicle. The method further includes detecting the object in front of the vehicle using the sensor data received from the object-detecting sensor and determining whether the visible-light camera is unable to detect the object in front of the vehicle that was detected by the object-detecting sensor. The method further includes commanding a display to generate a virtual image using the sensor data to identify the object in front of the vehicle.

Methods and tiling engines for tiling primitives in a tile based graphics processing system in which a rendering space is divided into a plurality of tiles. The method includes generating a multi-level hierarchy of tile groups, each level of the multi-level hierarchy comprising one or more tile groups comprising one or more of the plurality of tiles; receiving a plurality of primitive blocks, each primitive block comprising geometry data for one or more primitives; associating each of the plurality of primitive blocks with one or more of the tile groups up to a maximum number of tile groups such that if at least one primitive of a primitive block falls, at least partially, within the bounds of a tile, the primitive block is associated with at least one tile group that includes that tile; and generating a control stream for each tile group based on the associations, wherein each control stream comprises a primitive block entry for each primitive block associated with the corresponding tile group.

A method and intersection testing module are provided in a ray tracing system for determining whether a ray intersects a 3D axis-aligned box. The box represents a volume defined by a front-facing plane and a back-facing plane for each of the dimensions of the three-dimensional axis-aligned box. Scaled ray components are determined, wherein a third scaled ray component equals 1. A scaled minimum culling distance and a scaled maximum culling distance are determined. Determined cross-multiplication values are used to identify which of the front-facing planes intersects the ray furthest along the ray and identify which of the back-facing planes intersects the ray least far along the ray. It is determined whether the ray intersects the identified front-facing plane of the box at a position that is no further along the ray than the position at which the ray intersects the identified back-facing plane.

A smart, human-centered technique that uses artificial intelligence and mixed reality to accelerate essential tasks of the inspectors such as defect measurement, condition assessment and data processing. For example, a bridge inspector can analyze some remote cracks located on a concrete pier, estimate their dimensional properties and perform condition assessment in real-time. The inspector can intervene in any step of the analysis/assessment and correct the operations of the artificial intelligence. Thereby, the inspector and the artificial intelligence will collaborate/communicate for improved visual inspection. This collective intelligence framework can be integrated in a mixed reality supported see-through headset or a hand-held device with the availability of sufficient hardware and sensors. Consequently, the methods reduce the inspection time and associated labor costs while ensuring reliable and objective infrastructure evaluation. Such methods offer contributions to infrastructure inspection, maintenance, management practice, and safety for the inspection personnel.

A virtual reality apparatus and method are described for tile-based rendering. For example, one embodiment of an apparatus comprises: a set of on-chip geometry buffers including a first buffer to store geometry data, and a set of pointer buffers to store pointers to the geometry data; a tile-based immediate mode rendering (TBIMR) module to perform tile-based immediate mode rendering using geometry data and pointers stored within the set of on-chip geometry buffers; spill circuitry to determine when the on-chip geometry buffers are over-subscribed and responsively spill additional geometry data and/or pointers to an off-chip memory; and a prefetcher to start prefetching the geometry data from the off-chip memory as space becomes available within the on-chip geometry buffers, the TBIMR module to perform tile-based immediate mode rendering using the geometry data prefetched from the off-chip memory.

A method of tracking a CAD model in real time based on a particle filter according to one embodiment of the present disclosure is a method of detecting and tracking a real object based on target object recognition data for a digital model designed on CAD executed by a CAD object tracking detection program installed in a user computing device. The method includes: acquiring an image captured by photographing a surrounding object; detecting a real object corresponding to a shape of a target object designed in CAD from a first frame image of the captured image; and tracking the detected real object in a second frame image of the captured image, wherein the tracking of the detected real object includes determining a new pose of the real object in the second frame image based on the particle filter with respect to an initial pose of the detected real object.

Three-dimensional object representation and re-rendering systems and methods for producing a 3D representation of an object from 2D images including the object that enables object-centric rendering. A modular approach is used that optimizes a Neural Radiance Field (NeRF) model to estimate object geometry and refine camera parameters and, then, infer surface material properties and per-image lighting conditions that fit the 2D images.

The disclosure is generally directed to systems and methods for trailer hitch ball position location including receiving a plurality of image frames from a camera directed at a front of a trailer coupled to a vehicle at a coupling point, modeling the image frames in a convolutional neural network to form an initial estimate of a pivot point position as a location, optimizing the model using a nonlinear equation to identify the pivot point position, and locating the coupling point as the optimized pivot point position. The convolutional neural network includes a plurality of bounding boxes centered as locations of predetermined markers on the front of the trailer, the modeling via a reverse pinhole projection of a pixel onto a three-dimensional coordinate frame projected on a trailer plane to enable computed marker positions to determine relative dimensions of the trailer and locate the coupling point. The optimizing includes solving a nonlinear least-squares (NLLSQ) optimization formulation for multiple markers on the trailer. The method include the use of geometrical means to determine the angle of articulation between the trailer and tow-vehicle (hitch angle) by tracking the plurality of bounding boxes centered as locations of predetermined markers on the front of the trailer, as the vehicle and trailer move in a circular arc with respect to each other.

Systems and methods are provided to receive, at a processor associated with a vehicle and via one or more image sensors associated with the vehicle, image data associated with an environment surrounding the vehicle and corresponding to a first image captured at a first time, and additional image data associated with an environment surrounding the vehicle and corresponding to a second image captured by at a second time. The provided systems and methods may determine, based on the received additional image data and a machine learning model, that a tracked object identified in the first image is not detected in the second image, and may determine, based on vehicle data and tracking data of the tracked object, that the tracked object should be present in the second image and perform a remedial action on the additional image data to identify the tracked object in the second image.