Embodiments relate to storing hierarchically structured sub-items of scene entities in a flattened list of sub-items and performing time-constrained tasks on the sub-items in the flattened list. By storing the sub-items in the flattened list, an approximate time for processing the sub-items can be estimated more accurately, and therefore, reduces the likelihood of making overly conservative estimate of time for processing the sub-items. One or more sub-items of updated scene entities are extracted by a plurality of collectors that are executed in parallel to store the one or more sub-items in the flattened list. The sub-items are then accessed by multiple tasks executed in parallel to determine priority information associated with inclusion and rendering in subsequent frames. Sub-items with higher priority according to the priority information is given higher priority for retrieving from secondary memory and saving in primary memory.

Embodiments relate to storing hierarchically structured sub-items of scene entities in a flattened list of sub-items and performing time-constrained tasks on the sub-items in the flattened list. By storing the sub-items in the flattened list, an approximate time for processing the sub-items can be estimated more accurately, and therefore, reduces the likelihood of making overly conservative estimate of time for processing the sub-items. One or more sub-items of updated scene entities are extracted by a plurality of collectors that are executed in parallel to store the one or more sub-items in the flattened list. The sub-items are then accessed by multiple tasks executed in parallel to determine priority information associated with inclusion and rendering in subsequent frames. Sub-items with higher priority according to the priority information is given higher priority for retrieving from secondary memory and saving in primary memory.

Multiple snapshots of a scene are captured within an executing application (e.g., a video game). When each snapshot is captured, associated color values per pixel and a distance or depth value z per pixel are stored. The depth information from the snapshots is accessed, and a point cloud representing the depth information is constructed. A mesh structure is constructed from the point cloud. The light field(s) on the surface(s) of the mesh structure are calculated. A surface light field is represented as a texture. A renderer uses the surface light field with geometry information to reproduce the scene captured in the snapshots. The reproduced scene can be manipulated and viewed from different perspectives.

Multiple snapshots of a scene are captured within an executing application (e.g., a video game). When each snapshot is captured, associated color values per pixel and a distance or depth value z per pixel are stored. The depth information from the snapshots is accessed, and a point cloud representing the depth information is constructed. A mesh structure is constructed from the point cloud. The light field(s) on the surface(s) of the mesh structure are calculated. A surface light field is represented as a texture. A renderer uses the surface light field with geometry information to reproduce the scene captured in the snapshots. The reproduced scene can be manipulated and viewed from different perspectives.

A system in which players participate as characters in a virtual game world via game clients executing on client devices. An observer mode is provided that enables users to navigate in the game world without requiring a game client so that a user can navigate the game world while not actively playing the game. When a user accesses the game in observer mode, a view of the game world is rendered from the viewpoint of an observer's character; the observer's character may not correspond to a character in the game and the observer's character may not be visible to the players in the game. The rendered view of the game world is streamed to a viewing application on a personal device. Navigation inputs allow the observer's character to move freely within the game world.

A method is provided for rendering a mixed reality video. The method includes operations for capturing a head mounted display (HMD) game play by a user of a video game that is being executed on a computing system where the HMD game play is being captured from game play point of view (POV). The method further includes operations for identifying, by the computing system, a coordinate location of a camera that has a camera POV used to view the user during the HMD game play. In addition, the method further includes replaying the HMD game play to adjust the game play POV so that it substantially aligns with the camera POV. Moreover, the method includes rendering the mixed reality video by compositing video from the HMD game play after adjusting the game play POV and video from the camera POV. Rendering the mixed reality video also includes removing the background captured in the video from the camera POV so that the user appears partially within a scene of the video game when rendered in the mixed reality video.

Embodiments of this application provide a virtual scene display method performed at a computing device. The method includes: obtaining animation data and video data; displaying an animation interface corresponding to the animation data in an animation display area of a display interface, the animation interface including a first virtual scene including a plurality of animation elements; and playing a video clip in the video data in response to an operation on the animation elements in the animation interface, the video clip presenting a picture of transformation from the first virtual scene to a second virtual scene. A virtual scene display solution provided in the embodiments of this application can reduce resource consumption of an intelligent device and reduce picture stutter.

Embodiments of this application provide a virtual scene display method performed at a computing device. The method includes: obtaining animation data and video data; displaying an animation interface corresponding to the animation data in an animation display area of a display interface, the animation interface including a first virtual scene including a plurality of animation elements; and playing a video clip in the video data in response to an operation on the animation elements in the animation interface, the video clip presenting a picture of transformation from the first virtual scene to a second virtual scene. A virtual scene display solution provided in the embodiments of this application can reduce resource consumption of an intelligent device and reduce picture stutter.

An application processing system provides an interaction with a virtual character to a user. The system outputs an image of a virtual space including the virtual character to a head mounted display. The system determines that a menu icon has been selected based on detection information obtained by the head mounted display. The system sets, based on the menu icon being selected, a first selection point and a second selection point on the virtual character, and determines, based on one or more items of detection information obtained by the head mounted display, that the first selection point or the second selection point has been selected. The system specifies, based on the first selection point being selected, a first user action as an action of the user, and specifies, based on the second selection point being selected, a second user action as an action of the user.

An application processing system provides an interaction with a virtual character to a user. The system outputs an image of a virtual space including the virtual character to a head mounted display. The system determines that a menu icon has been selected based on detection information obtained by the head mounted display. The system sets, based on the menu icon being selected, a first selection point and a second selection point on the virtual character, and determines, based on one or more items of detection information obtained by the head mounted display, that the first selection point or the second selection point has been selected. The system specifies, based on the first selection point being selected, a first user action as an action of the user, and specifies, based on the second selection point being selected, a second user action as an action of the user.