Disclosed is a virtual content experience system. In the virtual content experience system, a central server for driving the system contains: a content conversion unit which converts two-dimensional image content, received by means of a data transmission and reception unit or input by a user, into a stereoscopic image; a motion information generation unit which recognizes text information extracted from the two-dimensional image content and converts the text information into motion information; a content playback control unit which is provided to transmit the motion information to a motion information management unit provided in a virtual reality experience chair, or receive start information and end information about the motion information from the motion information management unit to generate and change control information for controlling whether to provide new two-dimensional image content; and a display unit for displaying the content conversion unit, and the motion information or control information.

Methods and apparatus for capturing, communicating and using image data to support virtual reality experiences are described. Images, e.g., frames, are captured at a high resolution but lower frame rate than is used for playback. Interpolation is applied to captured frames to generate interpolated frames. Captured frames, along with interpolated frame information, are communicated to the playback device. The combination of captured and interpolated frames correspond to a second frame playback rate which is higher than the image capture rate. Cameras operate at a high image resolution but slower frame rate than images could be captured with the same cameras at a lower resolution. Interpolation is performed prior to delivery to the user device with segments to be interpolated being selected based on motion and/or lens FOV information. A relatively small amount of interpolated frame data is communicated compared to captured frame data for efficient bandwidth use.

Methods and apparatus relating to encoding and decoding stereoscopic (3D) image data, e.g., left and right eye images, are described. Various pre-encoding and post-decoding operations are described in conjunction with difference based encoding and decoding techniques. In some embodiments left and right eye image data is subject to scaling, transform operation(s) and cropping prior to encoding. In addition, in some embodiments decoded left and right eye image data is subject to scaling, transform operations(s) and filling operations prior to being output to a display device. Transform information and/or scaling information may be included in a bitstream communicating encoded left and right eye images. The amount of scaling can be the same for an entire scene and/or program.

Methods and apparatus relating to encoding and decoding stereoscopic (3D) image data, e.g., left and right eye images, are described. Various pre-encoding and post-decoding operations are described in conjunction with difference based encoding and decoding techniques. In some embodiments left and right eye image data is subject to scaling, transform operation(s) and cropping prior to encoding. In addition, in some embodiments decoded left and right eye image data is subject to scaling, transform operations(s) and filling operations prior to being output to a display device. Transform information and/or scaling information may be included in a bitstream communicating encoded left and right eye images. The amount of scaling can be the same for an entire scene and/or program.

In some aspects, the techniques described herein relate to systems, methods, and computer readable media for data pre-processing for stereo-temporal image sequences to improve three-dimensional data reconstruction. In some aspects, the techniques described herein relate to systems, methods, and computer readable media for improved correspondence refinement for image areas affected by oversaturation. In some aspects, the techniques described herein relate to systems, methods, and computer readable media configured to fill missing correspondences to improve three-dimensional (3-D) reconstruction. The techniques include identifying image points without correspondences, using existing correspondences and/or other information to generate approximated correspondences, and cross-checking the approximated correspondences to determine whether the approximated correspondences should be used for the image processing.

In some aspects, the techniques described herein relate to systems, methods, and computer readable media for data pre-processing for stereo-temporal image sequences to improve three-dimensional data reconstruction. In some aspects, the techniques described herein relate to systems, methods, and computer readable media for improved correspondence refinement for image areas affected by oversaturation. In some aspects, the techniques described herein relate to systems, methods, and computer readable media configured to fill missing correspondences to improve three-dimensional (3-D) reconstruction. The techniques include identifying image points without correspondences, using existing correspondences and/or other information to generate approximated correspondences, and cross-checking the approximated correspondences to determine whether the approximated correspondences should be used for the image processing.

In various implementations, a method of presenting video streams at a head-mountable device (HMD) includes generating a first video stream at a first frame rate for a first display portion. In some implementations, the first frame rate indicates a rate at which frames are presented by the first display portion. In various implementations, the method includes generating a second video stream at a second frame rate for a second display portion. In some implementations, the second frame rate indicates a rate at which frames are presented by the second display portion. In some implementations, the second frame rate is within a threshold relative to the first frame rate. In various implementations, the method includes temporally shifting the second video stream relative to the first video stream so that a majority of refresh times of the first display portion are different from refresh times of the second display portion.

In various implementations, a method of presenting video streams at a head-mountable device (HMD) includes generating a first video stream at a first frame rate for a first display portion. In some implementations, the first frame rate indicates a rate at which frames are presented by the first display portion. In various implementations, the method includes generating a second video stream at a second frame rate for a second display portion. In some implementations, the second frame rate indicates a rate at which frames are presented by the second display portion. In some implementations, the second frame rate is within a threshold relative to the first frame rate. In various implementations, the method includes temporally shifting the second video stream relative to the first video stream so that a majority of refresh times of the first display portion are different from refresh times of the second display portion.

A method for generating stereoscopic images is provided. The method includes: creating a three-dimensional mesh to obtain a stereoscopic scene and capturing a two-dimensional image of the stereoscopic scene; performing image preprocessing to obtain a first image in response to the two-dimensional image not being a side-by-side image; utilizing a graphics processing pipeline to perform depth estimation on the first image to obtain a depth image, to update the three-dimensional mesh according to a depth setting of the depth image, and to map the three-dimensional mesh to a corresponding coordinate system; utilizing the graphics processing pipeline to project the first image onto the mapped three-dimensional mesh to obtain an output three-dimensional mesh, and to capture an output side-by-side image from the output three-dimensional mesh; and utilizing the graphics processing pipeline to weave a left-eye and right-eye image into an output image, and to display the output image.

A stereoscopic image generation box, a stereoscopic image display method and a stereoscopic image display system are provided. The stereoscopic image generation box includes an image receiving and detecting unit, a depth information analysis unit, an image processing unit, a synthesis unit and a data transmission unit. The image receiving and detecting unit is used for receiving a two-dimensional image from an image source. The depth information analysis unit is used for obtaining a depth information according to the two-dimensional image. The image processing unit is used for converting the two-dimensional image into a left-eye image and a right-eye image according to the depth information. The synthesizing unit is used for synthesizing the left-eye image and the right-eye image to generate a stereoscopic image. The data transmission unit is used for outputting the stereoscopic image to a display, so that the display can directly display the stereoscopic image.