A method is provided, including the following method operations: receiving captured images of an interactive environment in which a head-mounted display (HMD) is disposed; receiving inertial data processed from at least one inertial sensor of the HMD; analyzing the captured images and the inertial data to determine a current and predicted future location of the HMD; using the predicted future location of the HMD to adjust a beamforming direction of an RF transceiver towards the predicted future location of the HMD; tracking a gaze of a user of the HMD; generating image data depicting a view of a virtual environment for the HMD, wherein regions of the view are differentially rendered; generating audio data depicting sounds from the virtual environment, the audio data being configured to enable localization of the sounds by the user; transmitting the image data and the audio data via the RF transceiver to the HMD.

An information processing apparatus configured to communicate with an image generating apparatus that generates a virtual viewpoint image that is based on a plurality of images acquired by image capturing with a plurality of cameras from a plurality of directions, comprises a transmission unit which transmits an inquiry to the image generating apparatus that is obtaining a data set for generating a virtual viewpoint image, an obtaining unit which obtains time information regarding a time corresponding to a virtual viewpoint image that can be generated by the image generating apparatus, based on a response to the inquiry transmitted by the transmission unit, and an output unit which outputs, to the image generating apparatus, a generation instruction to generate a virtual viewpoint image corresponding to a time determined based on the time information obtained by the obtaining unit.

Systems and methods of generating, selecting and rendering level of detail (LOD) visual assets are described. In an offline process for auto-generating the LOD visual assets, a LOD management module receives data representative of a LOD visual asset of an object model and its associated switch distance and iteratively reduces a polygon mesh complexity of the LOD visual asset to generate a lower complexity LOD visual asset along with its associated offline authored switch distance. Data representative of the LOD visual assets is presented in one or more GUIs to enable a user to optimize the data. Subsequently, a rendering module selects a LOD visual asset based on its associated switch distance or a modulated switch distance. The modulated switch distance is determined by applying one or more corrective factors to the associated switch distance.

A system and method for building and rendering in-game objects. In some embodiments, the system includes a plurality of metadata records describing available assets, wherein each metadata record comprises an asset identifier, which identifies an asset, and a property tag. In some embodiments, the system receives the metadata asset record or the asset identified by the metadata asset record from an asset interface to instruct a rendering engine to load the asset identified by the metadata asset record into a 3D environment.

A display control unit executes control to display images requested to be rendered, including effects in a plurality of formats and of a plurality of kinds according to rendering requests, on a display medium via a GPU by using effect data in which the amounts of data have been reduced in accordance with the rendering frequencies of the effects. Specifically, the display control unit determines the amount of data to be reduced for an effect in a format and of a kind according to a rendering request in accordance with the rendering frequency of the effect, extracts the effect data in which data has been reduced by the amount of data from effect-data storage DBs, and provides the effect data to the GPU.

A providing apparatus configured to provide three-dimensional geometric data to be used to generate a virtual viewpoint image receives a data request from a communication apparatus, decides which of a plurality of pieces of three-dimensional geometric data including first three-dimensional geometric data and second three-dimensional geometric data with a different quality than the first three-dimensional geometric data is to be provided to the communication apparatus from which the received data request was transmitted, and provides the three-dimensional geometric data decided on from among the plurality of pieces of three-dimensional geometric data, to the communication apparatus as a response to the received data request.

A display control unit executes control to display images requested to be rendered, including effects in a plurality of formats and of a plurality of kinds according to rendering requests, on a display medium via a GPU by using effect data in which the amounts of data have been reduced in accordance with the rendering frequencies of the effects. Specifically, the display control unit determines the amount of data to be reduced for an effect in a format and of a kind according to a rendering request in accordance with the rendering frequency of the effect, extracts the effect data in which data has been reduced by the amount of data from effect-data storage DBs, and provides the effect data to the GPU.

By using the user head and/or eye tracking capabilities of an extended reality (XR) system, a region of gaze (ROG) of the user towards the virtual screen can be determined and signaled to non-XR games and applications running on the host system connected to the XR system. The existing non-XR games/applications can be patched, upgraded, or remastered to support the application of the ROG and ROG based enhancements of the visuals and user interfaces, using the provided by the system software on the host system that is rendering the game for the XR system.

The systems and methods described herein provide techniques for constructing computer-generated three-dimensional objects to be rendered within a virtual scene. In various implementations, base meshes, base textures, and texture overlays may be stored on a client computer system and used to render a three-dimensional object based on instructions received from a game server. Each base mesh may correspond to a different shape for a given type of three-dimensional object. For example, the base mesh may be for a body type of a character, the base mesh may represent a body shape for the character, and the one or more textures may represent skin tone and/or one or more types of clothing to be rendered on the character. In various implementations, the client computer system may receive instructions from the game server in which the data needed to identify the three-dimensional object may be encoded using a single-digit number of bytes.

The systems and methods described herein provide techniques for performing fast, incremental, dynamic loading of game assets. A client computer system may be configured to receive updated game state data from a game server and request game assets as needed based on the game state data. The game server may make game assets available for download by the client computing system in the form of downloadable resource comprising, for example, mesh data and texture data. The mesh data and the texture data may be organized within the downloadable resource based on levels of detail (LODs) and provided as a progressive stream. As data related to a game asset is downloaded from the game server, the client computer system may be configured to update the version of the game asset rendered within the virtual scene to incrementally improve the quality of the game asset until it reaches a target LOD.