Display devices include waveguides with metasurfaces as in-coupling and/or out-coupling optical elements. The metasurfaces may be formed on a surface of the waveguide and may include a plurality or an array of sub-wavelength-scale (e.g., nanometer-scale) protrusions. Individual protrusions may include horizontal and/or vertical layers of different materials which may have different refractive indices, allowing for enhanced manipulation of light redirecting properties of the metasurface. Some configurations and combinations of materials may advantageously allow for broadband metasurfaces. Manufacturing methods described herein provide for vertical and/or horizontal layers of different materials in a desired configuration or profile.

Visual-inertial tracking of an eyewear device using a rolling shutter camera(s). The eyewear device includes a position determining system. Visual-inertial tracking is implemented by sensing motion of the eyewear device. An initial pose is obtained for a rolling shutter camera and an image of an environment is captured. The image includes feature points captured at a particular capture time. A number of poses for the rolling shutter camera is computed based on the initial pose and sensed movement of the device. The number of computed poses is responsive to the sensed movement of the mobile device. A computed pose is selected for each feature point in the image by matching the particular capture time for the feature point to the particular computed time for the computed pose. The position of the mobile device is determined within the environment using the feature points and the selected computed poses for the feature points.

Projection systems and components thereof are described that are well suited to miniaturization. These systems and components may use one or more of the following features: a folded optical path, as in a reflective cavity or a beamsplitter; an illumination beam that is converging at the place where it impinges upon the spatial light modulator; a beamsplitter that uses opposed prisms of substantially different sizes; a beamsplitter whose obliquely disposed partial reflector defines a first rectangular reference space, and where at least a portion of the light source or at least a portion of the projector lens is disposed within such first rectangular reference space; a system in which a ratio of areas of the first rectangular reference space and a second rectangular reference space is within a specified range, where the second rectangular reference space is just large enough to encompass the optical components of the projector; a system in which the projector lens is small compared to the spatial light modulator.

A display apparatus includes a first pixel and a second pixel. Each of the first and second pixels includes a first sub-pixel which emits light having a first color, a second sub-pixel which emits light having a second color different from the first color, a third sub-pixel which emits light having a third color different from the first and second colors, and an infrared sub-pixel which emits infrared light. The infrared light emitted from the infrared sub-pixel in the first pixel and the infrared light emitted from the infrared sub-pixel in the second pixel have different intensities from each other.

Four dimensional (4D) energy-field package assembly for projecting energy fields according to a 4D coordinate function. The 4D energy-field package assembly includes an energy-source system having energy sources capable of providing energy to energy locations, and energy waveguides for directing energy from the energy locations from one side of the energy waveguide to another side of the energy waveguide along energy propagation paths.

An optical module in the present disclosure includes a first image display panel including a first display part and a first coupling part, a second image display panel including a second display part and a second coupling part, a third image display panel including a third display part and a third coupling part, and a cross dichroic prism configured to synthesize first, second, and third image light. The cross dichroic prism has a cross axis at which a two of photosynthesis surfaces cross each other. The first, second, and third image display panel are respectively bonded to different surfaces of the cross dichroic prism in a state where a long side of each of the first, second, and third display part extends along the cross axis and where the first, second, and third coupling part are each positioned on an outer side of the cross dichroic prism.

There is provided an optical device, including an input aperture, an output aperture, at least first and second light-transmitting substrates each having two major surfaces and edges, an input surface for coupling light waves into the substrate for effecting total internal reflection inside the substrate, and an output surface for coupling light waves out of the substrate, a major surface of the first substrate is attached to a major surface of the second substrate and the input surface of the first substrate is a partially reflecting surface, such that part of the light waves passing through the input aperture is partially reflected by the partially reflecting input surface and coupled into the first substrate and another part passes through the partially reflecting input surface and is coupled by the input surface of the second substrate into the second substrate.

Various embodiments are provided herein for tracking a user's physical environment, to facilitate on-the-fly blending of a virtual environment with detected aspects of the physical environment. Embodiments can be employed to facilitate virtual roaming by compositing virtual representations of detected physical objects into virtual environments. A computing device coupled to a HMD can select portions of a depth map generated based on the user's physical environment, to generate virtual objects that correspond to the selected portions. The computing device can composite the generated virtual objects into an existing virtual environment, such that the user can traverse the virtual environment while remaining aware of their physical environment. Among other things, the computing device can employ various blending techniques for compositing, and further provide image pass-through techniques for selective viewing of the physical environment while remaining fully-immersed in virtual reality.

A method, includes saving in-flight data from an aircraft during a simulated training exercise, wherein the in-flight data includes geospatial locations of the aircraft, positional attitudes of the aircraft, and head positions of a pilot operating the aircraft, saving simulation data relating to a simulated virtual object presented to the pilot as augmented reality content in-flight, wherein the virtual object was programmed to interact with the aircraft during the simulated training exercise and representing the in-flight data from the aircraft and the simulation data relating to the simulated virtual object as a replay of the simulated training exercise.

A robot system includes a mobile robot configured to move, a portable teaching device including a display section configured to display information, the portable teaching device teaching the mobile robot, a first detecting section configured to detect a present position of the portable teaching device, a second detecting section configured to detect a present position of the mobile robot, and a display control section configured to cause, based on a detection result of the first detecting section and a detection result of the second detecting section, the display section to display the present position of the portable teaching device and the present position of the mobile robot.