Provided is an image processing apparatus including: a virtual viewpoint image generating unit that generates a virtual viewpoint image which is a video, based on captured images captured by image capturing apparatuses from different directions; an electronic sign information obtaining unit that obtains information indicating a timing at which a content displayed on an electronic sign changes, the electronic sign being contained in the virtual viewpoint image and configured to change the content to be displayed on a time basis; and a control unit that performs control to cause a display unit to display the virtual viewpoint image having a virtual content inserted, the virtual content being a content that is virtual and not contained in the captured images. Based on the information, the control unit controls how the virtual content is displayed on the virtual viewpoint image.

A dynamic event capturing and rendering system collects and aggregates video, audio, positional, and motion data to create a comprehensive user perspective 360-degree rendering of a field of play. An object associated with a user collects data that is stitched together and synchronized to provide post event analysis and training. Through an interface actions that occurred during an event can be recreated providing the viewer with information on what the user associated with the object was experiencing, where the user was looking, and how certain actions may have changed the outcome. Using the collected data, a virtual realty environment is created that can be manipulated to present alternative courses of action and outcomes.

A user monitoring system receives a first data stream from a first recording device and a second data stream from a second recording device. Each of the first data stream and the second data stream include data relating to an eye of the user. The first data stream and the second data stream overlap temporally. The system processes the first data stream to determine a first blink sequence of the user, processes the second data stream to determine a second blink sequence of the user, and compares the first blink sequence and the second blink sequence to detect a blink pattern present in both the first blink sequence and the second blink sequence. The system determines a temporal offset of the first data stream and the second data stream by comparing respective positions of the blink pattern in the first data stream and the second data stream.

The present technology relates to an information processing device, an information processing method, a program, and an information processing system capable of providing a user who views a free viewpoint moving image with better user experience.

An information processing device of the present technology includes: a transmission unit that transmits a moving image; and a control unit that controls the transmission unit to transmit a free viewpoint moving image or a real camera viewpoint moving image on the basis of a result of determination on whether or not the free viewpoint moving image has been successfully generated, the free viewpoint moving image being a moving image generated by using a plurality of camera moving images generated by imaging a subject by using a plurality of cameras and viewed from an arbitrary position and direction, the real camera viewpoint moving image being a moving image generated from a camera moving image generated by imaging the subject by using a camera and viewed from a position and direction of the camera. The present technology is applicable to a real-time volumetric system that transmits a free viewpoint moving image in real time.

A wearable electronic device includes a left-eye display configured to output light of a first color corresponding to a 3D left-eye image, a right-eye display configured to output light of a second color corresponding to a 3D right-eye image, a left-eye optical waveguide configured to adjust a path of the light of the first color and output the light of the first color, a right-eye optical waveguide configured to adjust a path of the light of the second color and output the light of the second color, a left-eye display control circuit configured to supply a driving power and a control signal to the left-eye display, a right-eye display control circuit configured to supply a driving power and a control signal to the right-eye display, a communication module configured to communicate with a mobile electronic device, and a second control circuit configured to supply a driving power and a control signal to the communication module.

A method for determining one or more groups of exposure settings to use in a 3D image acquisition process carried out with an imaging system, the 3D image acquisition process comprising capturing one or more sets of image data on the image sensor using the respective groups of exposure settings, wherein the one or more sets of image data are such as to allow the generation of one or more 3D point clouds defining the three-dimensional coordinates of points on the surface(s) of one or more objects being imaged, each group of exposure settings specifying a value for one or more parameters of the imaging system, wherein the method comprises identifying one or more candidate groups of exposure settings and selecting from the candidate groups of exposure settings, one or more groups of exposure settings that satisfy one or more optimization criteria.

A method for producing a magnified three dimensional (3D) image of an object having a parallax modified according to the magnification including the procedures of acquiring a first image of the object by a first camera and a second image of the object by a second camera, determining a magnification parameter, generating a first magnified image and a second magnified image by respectively magnifying a portion of the first image containing the object and a portion of the second image containing the object according to the magnification parameter, modifying a geometry of the first magnified image and of the second magnified image as a function of the magnification parameter, thereby producing a first modified image and a second modified image, and displaying the magnified 3D image to a user by displaying the first modified image and the second modified image.

A system for generating high-resolution de-warped omni-directional stereo image from captured omni-directional stereo image by correcting optical distortions using projection patterns is provided. The system includes a projection pattern capturing arrangement, a projector or a display, and a de-warping server. The projection pattern capturing arrangement includes one or more omnidirectional cameras to capture projection patterns from the captured omni-directional stereo image from each omni-directional stereo camera. The projector or the display displays the projection patterns. The de-warping server obtain the projection patterns and processes the projection patterns to generate high resolution de-warped omni-directional stereo image by correcting optical distortions in the captured omni-directional stereo image and mapping the captured omni-directional stereo image and the high resolution de-warped omni-directional stereo image.

A method for compressing geometric data and video is disclosed. The method includes receiving video and associated geometric data for a physical location, generating a background video from the video, and generating background geometric data for the geometric data outside of a predetermined distance from a capture point for the video as a skybox sphere at a non-parallax distance. The method further includes generating a geometric shape for a first detected object within the predetermined distance from the capture point from the geometric data, generating shape textures for the geometric shape from the video, and encoding the background video and shape textures as compressed video along with the geometric shape and the background geometric data as encoded volumetric video.

There is provided an imaging element that photographs multiple viewing point images corresponding to images observed from different viewing points and an image processing unit separates an output signal of the imaging element, acquires the plurality of viewing point images corresponding to the images observed from the different viewing points, and generates a left eye image and a right eye image for three-dimensional image display, on the basis of the plurality of acquired viewing point images. The image processing unit generates parallax information on the basis of the plurality of viewing point images obtained from the imaging element and generates a left eye image and a right eye image for three-dimensional image display by 2D3D conversion processing using the generated parallax information. By this configuration, a plurality of viewing point images are acquired on the basis of one photographed image and images for three-dimensional image display are generated.