In one implementation, a non-transitory computer-readable storage medium stores program instructions computer-executable on a computer to perform operations. The operations include obtaining first content representing a physical environment in which an electronic device is located using an image sensor of the electronic device. A physical feature corresponding to a physical object in the physical environment is detected using the first content. A feature descriptor corresponding to a physical parameter of the physical feature is determined using the first content. Second content representing a computer generated reality (CGR) environment is generated based on the feature descriptor and presented on a display of the electronic device.

An image source (407) provides an image divided into segments of different sizes with only a subset of these comprising image data. A metadata generator (409) generates metadata structured in accordance with a tree data structure where each node is linked to a segment of the image. Each node is a branch node linking the parent node to child nodes linked to segments that are subdivisions of the parent node, or a leaf node which has no children. A leaf node is either an unused leaf node linked to a segment for which the first image comprises no image data or a used leaf node linked to a segment for which the first image comprises image data. The metadata indicates whether each node is a branch node, a used leaf node, or an unused leaf node. An image signal generator (405) generates an image signal comprising the image data of the first image and the metadata.

Systems and methods for image processing are described. Embodiments of the present disclosure receive an image depicting an object; generate a sequence of tokens including a set of tokens corresponding to the object and a set of mask tokens corresponding to an additional object to be inserted into the image; generate a placement token value for the set of mask tokens based on the sequence of tokens using a sequence encoder, wherein the placement token value represents position information of the additional object; and insert the additional object into the image based on the position information to obtain a composite image.

In some implementations, a method includes obtaining an end state of a first content item spanning a first time duration. In some implementations, the end state of the first content item indicates a first state of a synthesized reality (SR) agent at the end of the first time duration. In some implementations, the method includes obtaining an initial state of a second content item spanning a second time duration subsequent the first time duration. In some implementations, the initial state of the second content item indicates a second state of the SR agent at the beginning of the second time duration. In some implementations, the method includes synthesizing an intermediary emergent content item spanning over an intermediary time duration that is between the end of the first time duration and the beginning of the second time duration.

A method for scene interaction includes identifying a first real scene interacting with a virtual scene and obtaining media information of the first real scene. The method also includes determining a scene feature associated with the first real scene based on a feature extraction on the media information, and mapping the scene feature associated with the first real scene to the virtual scene according to a correspondence between the virtual scene and the first real scene. Apparatus and non-transitory computer-readable storage medium counterpart embodiments are also contemplated.

To provide an information processing apparatus, an information processing method, a reproduction processing apparatus, and a reproduction processing method that provide a user with a viewing experience with a high degree of freedom. A data input unit acquires data of a three-dimensional space configured as a single spatial three-dimensional object including a plurality of three-dimensional objects. A preprocessing unit generates three-dimensional object-specifying information for identifying a specific three-dimensional object included in a three-dimensional space. A file generation unit generates a file including data of a three-dimensional space and three-dimensional object-specifying information generated by the preprocessing unit.

A method and system for providing access to and control of parameters within a scenegraph includes redefining components or nodes' semantic within a scenegraph. The set of components or nodes (depending on the scenegraph structure) are required to enable access from the Application User Interface to selected scenegraph information. In one embodiment, a user interface is generated for controlling the scenegraph parameters. In addition, constraints can be implemented that allow or disallow access to certain scenegraph parameters and restrict their range of values.

Methods and apparatus provide for generating content for display including one or more objects to be rendered, including: defining a plurality of virtual elements; setting one or more parameters associated with each of the plurality of virtual elements; and generating content for display using the plurality of virtual elements and corresponding parameter values, where setting the one or more parameters includes setting one or more parameters in accordance with the output of a machine learning algorithm adapted to determine appropriate parameters for the plurality of virtual elements so as to simulate an object to be rendered.

Systems and methods of the present disclosure relate to fine grained interleaved rendering applications in path tracing for cloud computing environments. For example, a renderer and a rendering process may be employed for ray or path tracing and image-space filtering that interleaves the pixels of a frame into partial image fields and corresponding reduced-resolution images that are individually processed in parallel. Parallelization techniques described herein may allow for high quality rendered frames in less time, thereby reducing latency (or lag, in gaming applications) in high performance applications.

This technology relates to rendering content from discrete applications. In this regard, one or more computing devices may receive a global scene graph containing resources provided by two or more discrete processes, wherein the global scene graph is instantiated by a first process of the two or more discrete processes. The one or more computing devices may render and output for display, the global scene graph in accordance with the resources contained there.