An external scene image captured by an external scene imaging electronic camera attached to a head mounted display (HMD) is projected and displayed onto an image display screen arranged in front of the eyes of the user as a virtual image with a suitable viewing distance corresponding to the visual acuity of the user. At this time, for each object image presented in the virtual image of the external scene image, the virtual image is processed and formatted to add a predetermined degree of binocular disparity and image blur to the virtual image projected and displayed on the right and the left image display screen on the basis of a predetermined converted distance calculated from the real distance of each object. Thus, the user is given a sense of a realistic perspective for the virtual image of the external scene, free of the discomfort or unease.

An external scene image captured by an external scene imaging electronic camera attached to a head mounted display (HMD) is projected and displayed onto an image display screen arranged in front of the eyes of the user as a virtual image with a suitable viewing distance corresponding to the visual acuity of the user. At this time, for each object image presented in the virtual image of the external scene image, the virtual image is processed and formatted to add a predetermined degree of binocular disparity and image blur to the virtual image projected and displayed on the right and the left image display screen on the basis of a predetermined converted distance calculated from the real distance of each object. Thus, the user is given a sense of a realistic perspective for the virtual image of the external scene, free of the discomfort or unease.

An optical link system includes a field programmable gate array (FPGA), and the FPGA includes an inter-pupillary distance (IPD) application unit and a barrel distortion execution unit.

An optical link system includes a field programmable gate array (FPGA), and the FPGA includes an inter-pupillary distance (IPD) application unit and a barrel distortion execution unit.

The disclosure provides an eye tracking method and an eye tracking device. The method includes obtaining a reference interpupillary distance value; taking images of a user of a 3D display, and finding a first eye pixel coordinate corresponding to a first eye of the user and a second eye pixel coordinate corresponding to a second eye of the user in each image; detecting a first and a second eye spatial coordinates of the first and the second eyes, and determining projection coordinates based on the first eye spatial coordinate, the second eye spatial coordinate, and optical parameters of image capturing elements; determining an optimization condition related to the first and second eye spatial coordinates based on the first and second eye pixel coordinates, the projection coordinates, and the reference interpupillary distance value of each image; and optimizing the first and second eye spatial coordinates based on the optimization condition.

Systems and methods for determining disparity between two images are disclosed. Such systems and methods include obtaining a first raw pixel image of a scene from a first viewpoint, obtaining a second raw pixel image of the scene from a second viewpoint (e.g., separate from the first viewpoint in a camera baseline direction such as horizontal or vertical), modifying the first and second raw pixel images using component-separated correction to create respective first and second corrected pixel images maintaining pixel scene correspondence in the camera baseline direction from between the first and second raw pixel images to between the first and second corrected pixel images, determining pixel pairs from corresponding pixels between the first and second corrected pixel images in the camera baseline direction, and determining disparity correspondence for each of the determined pixel pairs from pixel locations in the first and second raw pixel images corresponding to respective pixel locations of the pixel pairs in the first and second corrected pixel images.

Systems and methods for determining disparity between two images are disclosed. Such systems and methods include obtaining a first raw pixel image of a scene from a first viewpoint, obtaining a second raw pixel image of the scene from a second viewpoint (e.g., separate from the first viewpoint in a camera baseline direction such as horizontal or vertical), modifying the first and second raw pixel images using component-separated correction to create respective first and second corrected pixel images maintaining pixel scene correspondence in the camera baseline direction from between the first and second raw pixel images to between the first and second corrected pixel images, determining pixel pairs from corresponding pixels between the first and second corrected pixel images in the camera baseline direction, and determining disparity correspondence for each of the determined pixel pairs from pixel locations in the first and second raw pixel images corresponding to respective pixel locations of the pixel pairs in the first and second corrected pixel images.

An information processing apparatus having: a display control unit configured to display a plurality of virtual viewpoint images respectively corresponding to a plurality of virtual viewpoints on a display unit, wherein each of the plurality of virtual viewpoint images is generated based on image data obtained by a plurality of image capturing apparatuses performing image capturing from different directions; a reception unit configured to receive an input in accordance with a user operation for changing a virtual viewpoint corresponding to one virtual viewpoint image of the plurality of virtual viewpoint images displayed on the display unit; and a viewpoint control unit configured to perform control to change a plurality of virtual viewpoints respectively corresponding to the plurality of virtual viewpoint images based on the received input.

An information processing apparatus having: a display control unit configured to display a plurality of virtual viewpoint images respectively corresponding to a plurality of virtual viewpoints on a display unit, wherein each of the plurality of virtual viewpoint images is generated based on image data obtained by a plurality of image capturing apparatuses performing image capturing from different directions; a reception unit configured to receive an input in accordance with a user operation for changing a virtual viewpoint corresponding to one virtual viewpoint image of the plurality of virtual viewpoint images displayed on the display unit; and a viewpoint control unit configured to perform control to change a plurality of virtual viewpoints respectively corresponding to the plurality of virtual viewpoint images based on the received input.

Aspects of the present disclosure are directed to setting a display mode for a virtual object based on a display mode timer that is controlled by context factors. An artificial reality system can associate one or more virtual objects with a corresponding display mode timer. Various ranges on the display mode timer can be mapped to different display modes that the virtual object can assume. The display mode timer can be adjusted to add time based on a determination of a user focusing on the virtual object or other context factors. Display mode timers can also have rules for setting other display mode timer properties, such as how quickly the display mode timer runs down, that are evaluated based on context factors.