A system and method of controlling a virtual camera. The method comprises receiving a camera path of the virtual camera, the camera path defining movement of the virtual camera over a period of time to capture video data of a scene; and determining a plurality of control points on the camera path using a plurality of time markers within the period of time, each of the time markers corresponding to at least one event of a predetermined type being identified in the scene, wherein the control points at least partially define the camera path. The method further comprises controlling the virtual camera using the camera path modified based on the plurality of control points in response to a user input in relation to at least one of the control points.

A method of retargeting camera motion is disclosed. Data is received that describes a position of a target along a target path. Data is received that describes a position and an orientation of a first camera along a first camera path wherein the first camera is filming the target. A second camera path is computed, the second path corresponding to a second camera. The second camera path corresponds to the first camera path with one or more exceptions. The one or more exceptions include one or more of modification of orientation data corresponding to the second camera path to place the target at the center of a camera screen associated with the second camera and removing of a camera rotation roll for the second camera. A difference is computed between the first camera path and the second camera path. The computed difference is applied to a third camera path.

A system configured to be coupled with a participant of an activity. The system comprises: a participant activity monitoring unit configured for monitoring a performance of the activity by the participant; an activity information module configured for storing performance information corresponding to the activity; and a participant performance correlator configured for delivering comparative performance data based on the monitored performance of the activity by the participant and the stored performance information.

An example game system includes a controller device, and a game process section for performing a game process based on an operation on the controller device. The controller device includes a plurality of direction input sections, a sensor section for obtaining a physical quantity used for calculating an attitude of the controller device, and a display section for displaying a game image. The game process section first calculates the attitude of the controller device based on the physical quantity obtained by the sensor section. Then, the game process section controls an attitude of a virtual camera in a virtual space based on the attitude of the controller device, and controls a position of the virtual camera based on an input on the direction input section. A game image to be displayed on the display section is generated based on the position and the attitude of the virtual camera.

The techniques disclosed herein provide a high fidelity, rich, and engaging experience for spectators of streaming video services. The techniques disclosed herein enable a system to receive, process and, store session data defining activity of a virtual reality environment. The system can generate recorded video data of the session activity along with rendered spatial audio data, e.g., render the spatial audio in the cloud, for streaming of the video data and rendered spatial audio data to one or more computers. The video data and rendered spatial audio data can provide high fidelity video clips of salient activity of a virtual reality environment. In one illustrative example, the system can automatically create a video from one or more camera positions and audio data that corresponds to the camera positions.

A design for a cooperative live action virtual reality experience is described. A client makes use of a 360-degree display unit to play a 360-degree video file, a corresponding audio file, and an annotation requesting the user to make a selection that will dictate the progression of the virtual reality experience. The user input is processed in view of a script tree to determine the subsequent 360-degree video, audio, and annotation to send to the client. In some embodiments, progression of the virtual reality experience is dictated by annotation selections made by a second user or by both users who are cooperatively participating in the virtual reality experience.

Handheld location based games are provided in which a user's physical location correlates to the virtual location of a virtual character on a virtual playfield.

Augmented Reality (AR) systems are provided in which video game indicia are overlaid onto .a user's physical environment. A landscape detector is provided that may obtain information about the user's landscape, in addition to the user's location, in order to provide overlaying information to an AR head-mounted display and control information to non-user controlled video game characters.

A system configured to be coupled with a participant of an activity. The system comprises: a participant activity monitoring unit configured for monitoring a performance of the activity by the participant; an activity information module configured for storing performance information corresponding to the activity; and a participant performance correlator configured for delivering comparative performance data based on the monitored performance of the activity by the participant and the stored performance information.

An example game system includes a controller device, and a game process section for performing a game process based on an operation on the controller device. The controller device includes a plurality of direction input sections, a sensor section for obtaining a physical quantity used for calculating an attitude of the controller device, and a display section for displaying a game image. The game process section first calculates the attitude of the controller device based on the physical quantity obtained by the sensor section. Then, the game process section controls an attitude of a virtual camera in a virtual space based on the attitude of the controller device, and controls a position of the virtual camera based on an input on the direction input section. A game image to be displayed on the display section is generated based on the position and the attitude of the virtual camera.

Handheld location based games are provided in which a user's physical location correlates to the virtual location of a virtual character on a virtual playfield. Augmented Reality (AR) systems are provided in which video game indicia are overlaid onto a user's physical environment. A landscape detector is provided that may obtain information about the user's landscape, in addition to the user's location, in order to provide overlaying information to an AR head-mounted display and control information to non-user controlled video game characters.