Method for Adapting the Rendering of a Scene

Abstract

A method for adapting the rendering of a virtual environment based on the performance of a computer system in which the method is run modifies graphic parameters of generated images. The method enables the highest possible graphic quality to be dynamically maintained in a virtual reality or augmented reality simulation, depending on availability of the computer system's hardware resources.

Claims

1. A method implemented by a computer system to adapt a rendering of a scene belonging to a virtual environment, wherein the scene comprises objects and textures, the scene being rendered in a plurality of images with a level of detail, and wherein for each image of the plurality of images the method comprises the steps of: obtaining available processing capacity of the computer system based on at least a central processing unit (CPU) processing load of the computer system; obtaining available memory of the computer system based on at least available RAM of the computer system; calculating processing cost of rendering the scene if the processing cost of rendering the scene is greater than the available processing capacity of the computer system, then: reducing the level of detail of the image; calculating required to render the scene, if the memory required to render the scene is greater than the available memory of the computer system, then: removing non-visible objects from the image.

2. The method according to claim 1, wherein the computer system comprises motion sensors, the method further comprising steps of: obtaining a motion vector based on signals from the motion sensors of the computer system; calculating a velocity vector based on at least the motion vector.

3. The method according to claim 2, wherein the level of detail of the image is reduced based on the velocity vector.

4. The method according to claim 1, wherein each image is rendered by making at least one draw call for each object and texture in the scene, and wherein if the processing cost of rendering the scene is greater than the available processing capacity of the computer system, the method further comprises the step of: reducing the number of draw calls.

5. The method according to claim 4, wherein the number of draw calls is reduced based on the velocity vector.

6. The method according to claim 2, wherein each image is rendered according to a screen resolution of a display screen of the computer system, and wherein if processing cost of rendering the scene is greater than the available processing capacity of the computer system, then the method further comprises the step of: reducing a percentage of the screen resolution at which the image is rendered.

7. The method according to claim 6, wherein the percentage of screen resolution is reduced based on the velocity vector.

8. The method according to claim 1, wherein if the memory required to render the scene is greater than the available memory of the computer system, then the method further comprises the step of: removing non-visible textures from the image.

9. The method according to claim 1, wherein the computer system comprises a graphics processing unit (GPU), wherein, in the step of obtaining the available processing capacity of the computer system, the available processing capacity of the computer system is also obtained based on the GPU processing load.

10. The method according to claim 1, wherein the computer system comprises a graphics processing unit (GPU), wherein, in the step of obtaining the available memory of the computer system, the available memory of the computer system is also obtained based on the available GPU memory.

11. The method according to claim 1, wherein the computer system comprises a graphics processing unit (GPU), wherein the method further comprises the step of: allocating memory of the GPU of the computer system for textures.

12. The method according to claim 11, wherein each texture in the scene comprises a plurality of MIP maps, and wherein the method further comprises the step of: if the GPU memory is insufficient, then replacing or more textures with one of the corresponding MIP maps thereof.

13. A computer program with instructions that, when run by a computer system, causes the computer system to carry out the steps of the method according to claim 1.

14. A medium readable by a computer system with instructions that, when run by a computer system, cause the computer system to carry out the steps of the method according to claim 1.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0057] FIG. 1 shows a flow chart of a first embodiment of the method.

[0058] FIG. 2 shows a flow chart of a second embodiment of the method.

DETAILED DESCRIPTION OF AN EXEMPLARY EMBODIMENT

[0059] The following detailed description provides numerous specific details as examples to provide a thorough understanding of the relevant teachings. However, it will be apparent to those skilled in the art that the present teachings can be implemented without such details.

[0060] In a first exemplary embodiment, the method, illustrated by the chart in FIG. 1, is run on a moderate-performance home computer with at least one CPU, one RAM unit, one peripheral device with sensors to detect user interactions during simulation, and display media. The computer has a virtual reality program installed that enables a user to explore a virtual environment generated from an architectural design. In order to not exceed the capacity of the computer and keep the graphic quality of the representation as high as possible, the program code includes an algorithm that enables the method for adapting the rendering according to the invention to be run, and it is run iteratively during the rendering process of each image. A three-dimensional geometric model stored as a scene file has been generated from the architectural design; this scene file comprises the objects of the architectural design and a set of textures stored in bitmap format to cover said objects.

[0061] The virtual reality program includes a graphics computing application that enables the scene to be rendered dynamically based on the user's movement. To represent the virtual environment, the program generates a plurality of images according to the geometric model of the scene and the user's virtual position and movement. The images are successively displayed through the display media at a frequency high enough to achieve a continuous sense of the image; for a conventional computer monitor, 30 FPS (frames per second) or even 20 FPS is sufficient; in the case of an HMD device, a higher frequency of approximately 90 FPS is recommended. [0062] For each one of the images that are to be rendered, the available processing capacity of the computer system (SW) at a given moment is first obtained (10); in a simple example, it is obtained (10) by subtracting the CPU processing load (CW) from the CPU total processing capacity. [0063] The available memory of the computer system (SR) is obtained (11) from the available RAM (CA) at a given moment. [0064] In many computers, both the available processing capacity of the computer system (SW) and the available memory of the computer system (SR) can be obtained for example from the operating system or an API (application programming interface) for graphics computing; however, if not possible, any other known method can be used. [0065] Next, the processing cost of rendering the scene (SCR) is calculated (12) based on the level of detail and the number of objects and textures in the scene, and the memory cost of rendering the scene (SMC) is also calculated (14). [0066] If the processing cost of rendering the scene (SCR) is greater than the available processing capacity of the computer system (SW), the overall level of detail (LOD) is reduced (13). [0067] If the memory required to render the scene (SMC) is greater than the available memory of the computer system (SR), the memory allocation for the objects that are not visible from the user's point of view is removed (15).

[0068] Finally, rendering (16) of the image is performed, image which is subsequently shown to the user through the display media, and the program repeats the method again for the next image.

[0069] The program also has two possible modes of operation: a simulation mode and an edit mode for the user to modify the geometric model. In addition to use on a computer, the method can be used on a smartphone, via a mobile application (app) installed on the same, or through the Internet, via a web browser.

[0070] In a second exemplary embodiment, the computer system comprises a user terminal, an HMD (head mounted display) device and a server wherein the program is installed, and which is accessed through the Internet. The user terminal again comprises a moderate-performance computer with a graphics card with a GPU. The HMD (head mounted display) device integrates the display media and sensors that capture the user's movement. In other examples, the user accesses the server through their terminal, from which they download the program, to then install it in their own terminal.

[0071] In this example of the method, illustrated in the diagram of FIG. 2, the program obtains (20), from the sensors of the HMD device that the user carries on his head, a set of signals grouped into a motion vector (VM) with which the program calculates (21) a velocity vector (VS) that will be used to determine from the user's point of view which objects or textures in the scene are visible and which are not. The velocity vector (VS) is also used as a reduction factor for the parameters through which the graphic quality of the image is adapted, so that at a greater motion speed the reduction in the quality of the rendered images is greater in order to offset the higher processing cost and memory.

[0072] The available processing capacity of the computer system (SW) is obtained (10) in this example, the method considering both the CPU processing load (CW) and the GPU processing load (GW). The available memory of the computer system (SR) is obtained (11) from the available RAM (CA) and the available GPU memory (GA).

[0073] As an alternative or in addition to reducing (13) the overall level of detail (LOD) and removing (15) the memory allocation for non-visible objects, if the available processing capacity of the computer system (SW) is less than the processing cost of rendering the scene (SCR), then: [0074] the number of draw calls (DC) is reduced (17) and/or [0075] the screen resolution (SP) is reduced (18).

[0076] The reduction of any of these parameters (LOD, DC, SP) can be a fixed percentage, for example, between 10% and 50%, or it can be a variable value obtained from the product of a factor by the modulus of the velocity vector (VS), based on the performance of the equipment.

[0077] In addition, if the memory required to render the scene (SMC) is greater than the available memory of the computer system (SR), as an alternative or in addition to the previous actions, the memory allocation for the non-visible textures is removed (19).

[0078] In this example of the method, the step of allocating (22) GPU memory for the textures is included. If the GPU memory cache is not sufficient, the program replaces (23) the full-size textures with the corresponding larger MIP map, and optionally in the corresponding objects, it also replaces the MIP maps with a smaller MIP map.

Clauses of the Invention

[0079] 1. A method implemented by a computer system to adapt the rendering of a scene belonging to a virtual environment, wherein the scene comprises objects and textures, said scene being rendered in a plurality of images with a level of detail (LOD), and wherein for each image of the plurality of images the method comprises the steps of: [0080] obtaining (10) the available processing capacity of the computer system (SW) based on at least the CPU processing load (CW) of the computer system; [0081] obtaining (11) the available memory of the computer system (SR) based on at least the available RAM (CA) of the computer system; [0082] calculating (12) the processing cost of rendering the scene (SCR) [0083] if the processing cost of rendering the scene (SCR) is greater than the available processing capacity of the computer system (SW), then: [0084] reducing (13) the level of detail (LOD) of the image; [0085] calculating (14) the memory required to render the scene (SMC), [0086] if the memory required to render the scene (SMC) is greater than the available memory of the computer system (SR), then: [0087] removing (15) non-visible objects from the image.

[0088] 2. The method according to the preceding clause, wherein the computer system comprises motion sensors, and wherein the method further comprises the steps of: [0089] obtaining (20) a motion vector (VM) based on signals from the motion sensors of the computer system; [0090] calculating (21) a velocity vector (VS) based on at least the motion vector (VM).

[0091] 3. The method according to the preceding cl, wherein the level of detail (LOD) of the image is reduced based on the velocity vector (VS).

[0092] 4. The method according to any of the preceding clauses, wherein each image is rendered by making at least one draw call (DC) for each object and texture in the scene, and wherein if the processing cost of rendering the scene (SCR) is greater than the available processing capacity of the computer system (SW), then the method further comprises the step of: [0093] reducing (17) the number of draw calls (DC). [0094] 5. The method according to the preceding clause when it depends on clause 2, wherein the number of draw calls (DC) is reduced based on the velocity vector (VS). [0095] 6. The method according to any of the preceding clauses, wherein each image is rendered according to a screen resolution (SP) of the computer system display screen, and wherein if the processing cost of rendering the scene (SCR) is greater than the available processing capacity of the computer system (SW), then the method further comprises the step of: [0096] reducing (18) the percentage of screen resolution (SP) at which the image is rendered.

[0097] 7. The method according to the preceding clause when it depends on clause 2, wherein the percentage of screen resolution (SP) is reduced based on the velocity vector (VS).

[0098] 8. The method according to any of the preceding clauses, wherein if the memory required to render the scene (SMC) is greater than the available memory of the computer system (SR), then the method further comprises the step of: [0099] removing (19) non-visible textures from the image.

[0100] 9. The method according to any of the preceding clauses, wherein the computer system comprises a GPU, wherein, in the step of obtaining the available processing capacity of the computer system (SW), said available processing capacity of the computer system (SW) is also obtained based on the GPU processing load (GW).

[0101] 10. The method according to any of the preceding clauses, wherein the computer system comprises a GPU, wherein, in the step of obtaining the available memory of the computer system (SR), said available memory of the computer system (SR) is also obtained based on the available GPU memory (GA).

[0102] 11. The method according to any of the preceding clauses, wherein the computer system comprises a GPU, wherein the method further comprises the step of: [0103] allocating (22) memory of a GPU of the computer system for textures.

[0104] 12. The method according to the preceding clause, wherein each texture in the scene comprises a plurality of MIP maps, and wherein the method further comprises the step of: [0105] if the GPU memory is insufficient, then replacing (23) one or more textures with one of the corresponding MIP maps thereof.

[0106] 13. A computer program with instructions that, when run by a computer system, causes the computer system to carry out the steps of the method according to any of clauses 1-12.

[0107] 14. A medium readable by a computer system with instructions that, when run by a computer system, cause the computer system to carry out the steps of the method according to any of clauses 1-12.