Methods and Systems for Viewing and Editing AR/VR Computer-based Designs Allowing Impaired Vision Simulation

Abstract

A method of providing a user with a filtered image of a computer-based design, the method includes the steps of: storing, in a location accessible by a central host, master data representing the design; identifying view data corresponding to a portion of the master data representing a portion of the design to be viewed by a user; applying a filter to the view data to generate filtered view data; receiving the filtered view data at a display component of a viewer; displaying an image of the design corresponding to the filtered view data using the display component of the viewer.

Claims

1. A method of providing a user with a filtered image of a computer-based design, the method including the steps of: (a) storing, in a location accessible by a central host, master data representing the design; (b) identifying view data corresponding to a portion of the master data representing a portion of the design to be viewed by a user; (c) applying a filter to the view data to generate filtered view data; (d) receiving the filtered view data at a display component of a viewer; (e) displaying an image of the design corresponding to the filtered view data using the display component of the viewer.

2. A method according to claim 1, wherein the central host includes a processor having a filtering module, and wherein step (c) includes the steps of: (i) receiving the view data at the filtering module; (ii) applying the filter to the view data using the filtering module, to generate filtered view data, and (iii) outputting the filtered view data from the filtering module, towards the viewer, or the display component thereof.

3. (canceled)

4. A method according to claim 1, wherein the filter is configured to simulate conditions affecting eyesight, or varying levels of visual acuity.

5. (canceled)

6. A method according to claim 1, wherein the step of applying a filter takes place before the identification of view data, and wherein the filter is independent of the view data.

7. A method according to claim 1, wherein step of applying a filter to the view data includes: applying a left-eye filter to a portion of the view data corresponding to a viewpoint of the user's left eye; and applying a right-eye filter to a portion of the view data corresponding to a viewpoint of the user's right eye, wherein the left-eye filter is different from the right-eye filter.

8. (canceled)

9. A method according to claim 1, further including the step of eye-tracking using an eye-tracking device, wherein the view data is based, at least in part, on eye-tracking information from the eye-tracker.

10. A method according to claim 1, wherein the design is a 2D design, and the viewer is in the form of a PC, a tablet, a smartphone or a laptop.

11. A method according to claim 1, wherein the design is a 3D design.

12. A method according to claim 1, wherein: the user is able to move around within the 3D design, and the view data depends on either: the location and the orientation of the user within the 3D design, or the location and the orientation of the viewer within the 3D design.

13-14. (canceled)

15. A method according to claim 11, wherein: the viewer is in the form of one of: a PC, a tablet, a smartphone, or a laptop; and the user is able to navigate the 3D design using controls on the viewer, thus changing the view data.

16. A method according to claim 11, wherein the viewer is in the form of a virtual reality viewer, wherein the display component is in the form of a screen within the virtual reality viewer.

17-18. (canceled)

19. A method according to claim 11, wherein the view data is in the form of real-time data, and wherein the view data is in the form of a series of frames of data, each frame corresponding to a view of a region of the 3D design at a given instant.

20. A method according to claim 19, wherein the view data is in the form of a series of frames of data, each frame corresponding to a view of a region of the 3D design at a given instant.

21. A method according to claim 20, wherein steps (b) to (e) are performed for each frame of data, to create a series of filtered images representing successive frames of the view data.

22. A method according to claim 11, wherein: the step of applying a filter includes applying a filter to a portion of the view data representing an object within the 3D design, an effect of the filter is dependent on the distance, within the 3D design, between the location of the user within the 3D design and the object within the 3D design.

23.-24. (canceled)

25. A method of editing a computer-based design while reviewing it, including the steps of: (i) providing a user with an image of a computer-based design by performing the steps of: storing, in a location accessible by a central host, master data representing the design; identifying view data corresponding to a portion of the master data representing a portion of the design to be viewed by a user; applying a filter to the view data to generate filtered view data; receiving the filtered view data at a display component of a viewer; displaying an image of the design corresponding to the filtered view data using the display component of the viewer; (ii) receiving an editing input from a user, to generate data representing an edited portion of the design; (iii) updating the master data corresponding to the edited portion of the design.

26. A method according to claim 25, wherein editing input is received at a graphical user interface.

27. A method according to claim 25, wherein editing input is in the form of a gesture.

28. A method according to claim 25, wherein the data representing the edited portion of the design is cached before the step of updating the master data.

29. A method of providing a plurality of users with a filtered image of a computer-based design, the method including the steps of: storing, in a location accessible by a central host, master data representing a design; for a first viewer, performing the steps of: identifying view data corresponding to a portion of the master data representing a portion of the design to be viewed by a user; applying a filter to the view data to generate filtered view data; receiving the filtered view data at a display component of a viewer; displaying an image of the design corresponding to the filtered view data using the display component of the viewer; and for a second viewer, performing the steps of: identifying view data corresponding to a portion of the master data representing a portion of the design to be viewed by a user; applying a filter to the view data to generate filtered view data; receiving the filtered view data at a display component of a viewer; displaying an image of the design corresponding to the filtered view data using the display component of the viewer.

30-34. (canceled)

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0100] Embodiments of the invention will now be described by way of example with reference to the accompanying drawings in which:

[0101] FIG. 1 shows a system architecture diagram highlighting three ways in which a VR environment containing the 3D design may be accessed by users.

[0102] FIG. 2 shows a flowchart of methods employed in embodiments of the first, second, third and fourth aspects of the invention.

[0103] FIG. 3 shows a flowchart of a method to be used in some embodiments of the first aspect of the invention.

[0104] FIG. 4 shows a flowchart of an alternative method to be used in some embodiments of the first aspect of the invention.

[0105] FIGS. 5 and 6 show an example of a user's view through the display component, highlighting the interface which may be used to edit the design.

[0106] FIGS. 7 and 8 show an example of a user's view through the display component, when a filter is applied to simulate age-related macular degeneration.

[0107] FIG. 9 shows an example of a user's view through the display component, when a filter is applied to simulate glaucoma.

[0108] FIGS. 10 and 11 show examples of what may be presented on a computer monitor, when a filter is used to simulate glaucoma.

DETAILED DESCRIPTION OF THE DRAWINGS

[0109] FIG. 1 shows a typical system architecture which might be employed in embodiments of the present invention. FIG. 1 illustrates three different ways by which the 3D design may be accessed by users.

[0110] Central host 100 is at the centre of the architecture, functionally speaking. Central host 100 may be in the form of a computer, server, or the like. In some embodiments, central host 100 may include a storage area (not shown) upon which the master data representing the 3D design is stored. In other embodiments, there may be a direct connection between the central host 100 and a storage area 102, on which the master data is stored. In other embodiments, the central host 100 may be connected to a storage area 104 via a network 106. This may be the case if the master data is stored on a cloud storage area 104, and the network via which the cloud storage 104 is accessed may be the internet. In other embodiments, network 106 may be a local area network, or the like.

[0111] FIG. 1 illustrates three ways in which the master data stored on one of the storage areas 102, 104 may be accessed: [0112] In the first example, the master data may be accessed using PC 108 having loaded thereon a VR design program 110. In this case, the PC 108 represents the virtual reality display device, and the user is able to navigate the 3D design on their computer screen, and to edit it as described previously, using the VR design program 110. [0113] Secondly, and similarly, the master data may be accessed using mobile device 112, which represents the virtual reality display device. The mobile device 112 has an application 114 loaded thereon, which allows the user to navigate and edit the 3D design using the mobile device 112. The mobile device 112 may be in the form of a smartphone or a tablet. In these embodiments, the mobile device 112 may be provided with a frame into which the mobile device 112 may be inserted. The frame is preferably configured so that it converts the mobile device 112 into a wearable headset, so that a user can place it on his or her head, with the mobile device 112 screen in front of their eyes. This may correspond to the case set out above in which the virtual reality display device is in the form of a virtual reality headset. [0114] Finally, in the final example shown in FIG. 1, a PC 116 is connected to the central host 100 via the internet, as in the first example. The PC also includes the VR design program 120. However, in this case, two virtual reality headsets 118 are connected to the PC 116. Then, in use, a user wears one of the headsets 118 to explore and edit the 3D design. This example is effectively a mixture of the first and second examples.

[0115] The skilled person understands that the invention is in no way confined to the system architecture shown in FIG. 1, and that other layouts are possible.

[0116] FIGS. 2A to 2D show a series of steps which may be employed when using the present invention. These are described in depth below.

[0117] In step S1, a user locates a computer-aided design (CAD) file on e.g. their PC, smartphone or tablet. The CAD file may contain a 3D design of any kind, but in the present embodiment, the CAD file is a 3D architectural design of a building. Then, in step S2 the user uploads this file to the central host 100 as shown in FIG. 1. The upload may be via the internet, or may be via a local network. Steps S3 to S7 relate to the creation of the 3D design in the virtual reality environment from the initial CAD model.

[0118] In step S8, having generated the 3D design in a virtual environment based on the CAD file uploaded in step S1, the camera start position is defined by a user. In other words, the user determines whereabouts in the design that they would like to begin their exploration. Once this is done, in step S9, the scene is built. This means that the virtual environment, from the viewpoint of the selected camera start position, is constructed and displayed to the user, e.g. via a virtual reality headset. Then, at step S10, the user begins to explore the environment. Given that this embodiment relates to a building design, the user may want to see how the design appears from the perspective of someone with a visual impairment. To this end, in step S11, the user may select a visual impairment using e.g. a graphical user interface displayed to him or her on the virtual reality headset. Then, in step S12 the simulation is applied. Specifically, this is done by filtering (e.g. at the host 100) the image data which is to be received by the virtual reality headset before it is displayed to the user. Then in step S13, the user continues to explore the virtual environment this time with a filtered view. Based on the simulated impairment, the user may decide that the 3D design needs editing, which is performed in step S14, e.g. by providing gestural inputs to the graphical user interface, as described in the summary section of this application. The changes may be saved (i.e. the master data updated) in step S15. Then, in step S16, the user may share the updated 3D design with others, who may be exploring the virtual environment at the same time.

[0119] FIG. 3 shows a method which may be used in embodiments of the first aspect of the invention. The method comprises the following steps: calling a user interface within a rendered interface displayed on a computer monitor, calling a corresponding visual impairment simulation image using the user interface, through a HLSL Shader controlled by a raster image library based upon the visual impairment type selected by a user, and adjusting raster image parameters corresponding to a user's impairment severity selection; acquiring a live feed of the Computer Monitor rendered interface and displaying the processed image combined with both passive and reactive filters to visualise visual impairment simulation overlaid on the Computer Monitor rendered interface; capturing images and information from the rendered interface with the inclusion of both passive and active augmented visual impairment filters.

[0120] FIG. 4 shows another method which may be used in embodiments of the first aspect of the invention. The method comprises the following steps: calling a user interface within a multi-user real-time virtual environment; calling a corresponding visual impairment simulation image using the user interface, through a HLSL Shader controlled by a raster image library based upon the visual impairment type selected by a user, and adjusting raster image parameters corresponding to a user's impairment severity selection; acquiring an image feed of the multi-user real-time virtual environment and displaying the processed image of the virtual scene to visualise visual impairment simulation within the head mounted virtual reality device; making design changes to the multi-user real-time virtual environment using the acquired real-time image feed inclusive of visual impairment.

[0121] FIGS. 5 and 6 show examples of a user's view, through the display component of the present invention. Specifically, the drawings show an example of the user interface which may be displayed to the user in embodiments of the second aspect of the invention, so that the user is able to edit the environment. The user interface includes a wheel provided the user with a number of options which may be used for editing the design.

[0122] FIG. 7 shows an example of a user's view when the view data is filtered to simulate dry age-related macular degeneration within a CAD-based virtual environment. Using the wheel of options shown at the right-hand side of the image, it is possible to adjust the filter, e.g. to change the type of impairment simulated, or the degree of impairment. FIG. 8 shows something similar, except the filter is for the simulation of wet age-related macular degeneration. FIG. 9 again shows something similar, where a glaucoma filter is used.

[0123] FIG. 10 shows an example in which methods of the present invention are used to view web pages displayed on a computer monitor through the eyes of a user with a visual impairment. Here, an intermediate stage of glaucoma is simulated. In FIG. 11, the degree of glaucoma is more severe. In preferred embodiments of the present invention, an eye tracking unit would be used in conjunction with the monitor. This allows the filter to move with respect to the image displayed on the computer monitor, effectively uncovering different parts of the webpage as the user moves their eyes.

[0124] While the invention has been described in conjunction with the exemplary embodiments described above, many equivalent modifications and variations will be apparent to those skilled in the art when given this disclosure. Accordingly, the exemplary embodiments of the invention set forth above are considered to be illustrative and not limiting. Various changes to the described embodiments may be made without departing from the spirit and scope of the invention.

[0125] All references referred to above are hereby incorporated by reference.