SPORT AND GAME SIMULATION SYSTEMS AND METHODS

Abstract

Disclosed herein are sport and game simulation systems, and methods of use thereof. One example method is performed at an electronic device and includes: determining that a first ball of a plurality of balls enters an inner zone of a target and in response to determining that the first ball enters the inner zone, assigning a first point value to a game participant associated the first ball. The method also includes determining that a second ball, different from the first ball, of the plurality of balls enters at least one outer zone of the target, and in response to determining that the second ball enters the at least one outer zone, assigning a second point value to a game participant, the second point value being less than the first point value. The method also includes providing instructions to display the first point value and second point value.

Claims

1. A method of managing a ball-based game comprising (i) a plurality of balls each having an associated identifier, and (ii) a remote target with an inner zone and at least one outer zone, whereby the inner zone is closer to a center of the remote target relative to the at least one outer zone, the method comprising: at an electronic device with one or more processors: determining that a first ball of the plurality of balls enters the inner zone of the remote target; in response to determining that the first ball enters the inner zone, assigning a first point value to a game participant associated the first ball; determining that a second ball, different from the first ball, of the plurality of balls enters the at least one outer zone of the remote target; in response to determining that the second ball enters the at least one outer zone, assigning a second point value to a game participant, the second point value being less than the first point value; and providing first instructions to display the first point value and second point value.

2. The method of claim 1, further comprising, at the electronic device: providing second instructions to deliver the first ball to a game surface, wherein the first ball, once delivered, is to be hit by a game participant towards the remote target.

3. The method of claim 2, further comprising: before determining that the first ball of the plurality of balls enters the inner zone of the remote target: receiving data indicating that the game participant hit the first ball; and determining, based on the data, that the game participant's hit caused the first ball to enter the inner zone.

4. The method of claim 2, further comprising: after the first ball is hit by the game participant: providing third instructions to deliver the second ball to the game surface, wherein the second ball, once delivered, is to be hit by a game participant towards the remote target.

5. The method of claim 4, further comprising: after the second ball is hit by the game participant: receiving data indicating that the game participant hit the second ball; and determining, based on the data, that the game participant's hit caused the second ball to enter the at least one outer zone.

6. The method of claim 2, further comprising: receiving a request to play the ball-based game, wherein providing the second instructions to deliver the first ball is performed in response to receiving the request.

7. The method of claim 2, wherein: the plurality of balls is a plurality of golf balls; and the game surface is a golfing surface.

8. The method of claim 7, wherein the golfing surface is an artificial golfing surface.

9. The method of claim 8, wherein providing the second instructions to deliver the first ball causes the first ball to be placed at a predetermined position on the artificial golfing surface.

10. The method of claim 2, further comprising: before providing the second instructions to deliver the first ball, determining whether a different game participant has finished hitting balls from the game surface, wherein providing the second instructions to deliver the first ball is performed in accordance with a determination that the different game participant has finished hitting balls from the game surface.

11. The method of claim 1, further comprising: determining that a third ball, different from the first ball and the second ball, of the plurality of balls fails to enter the remote target; in response to determining that the third ball fails to enter the remote target, assigning a third point value to a game participant, the third point value being less than the first point value and the second point value; and providing additional instructions to display the third point value.

12. The method of claim 11, wherein the third point value is zero.

13. The method of claim 1, wherein the identifier associated with each of the plurality of balls is a radio-frequency identification (RFID) identifier, a near-field communication (NFC) identifier, or a distinct color.

14. The method of claim 1, wherein the inner zone and the at least one outer zone are geometric shapes.

15. The method of claim 14, wherein the geometric shapes are circular shapes.

16. The method of claim 1, wherein the remote target it a circular target.

17. The method of claim 1, wherein providing the first instructions to display the first point value and second point value comprises communicating the first and second point values to a display device for displaying the first and second point values.

18. A non-transitory computer-readable storage medium storing one or more programs for programmatically managing a ball-based game, comprising (i) a plurality of balls each having an associated identifier, and (ii) a remote target with an inner zone and at least one outer zone, whereby the inner zone is closer to a center of the remote target relative to the at least one outer zone, that, when executed by an electronic device including one or more processors and memory, cause the electronic device to: determining that a first ball of the plurality of balls enters the inner zone of the remote target; in response to determining that the first ball enters the inner zone, assigning a first point value to a game participant associated the first ball; determining that a second ball, different from the first ball, of the plurality of balls enters the at least one outer zone of the remote target; in response to determining that the second ball enters the at least one outer zone, assigning a second point value to a game participant, the second point value being less than the first point value; and providing instructions to display the first point value and second point value.

19. An electronic device for managing a ball-based game, comprising (i) a plurality of balls each having an associated identifier, and (ii) a remote target with an inner zone and at least one outer zone, whereby the inner zone is closer to a center of the remote target relative to the at least one outer zone, the electronic device comprising: one or more processors; and memory storing one or more programs for execution by the one or more processors, the one or more programs including instructions for: determining that a first ball of the plurality of balls enters the inner zone of the remote target; in response to determining that the first ball enters the inner zone, assigning a first point value to a game participant associated the first ball; determining that a second ball, different from the first ball, of the plurality of balls enters the at least one outer zone of the remote target; in response to determining that the second ball enters the at least one outer zone, assigning a second point value to a game participant, the second point value being less than the first point value; and providing instructions to display the first point value and second point value.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0126] For a better understanding of the various described embodiments, reference should be made to the Description of Embodiments below, in conjunction with the following drawings in which like reference numerals refer to corresponding parts throughout the figures.

[0127] FIG. 1A is a block diagram of a sport simulation system, in accordance with some embodiments.

[0128] FIG. 1B is a schematic diagram of a sport simulation system, in accordance with some embodiments.

[0129] FIG. 1C is a block diagram of a first embodiment of a system controller for a sport simulation system, in accordance with some embodiments.

[0130] FIG. 1D is a block diagram of a second embodiment (an alternative to the first embodiment) system controller for a sport simulation system, in accordance with some embodiments.

[0131] FIG. 1E is a schematic diagram showing environmental simulation modules that are part of a sport simulation system in accordance with some embodiments.

[0132] FIGS. 2A-2C are schematic diagrams of hitting maps used in a sport simulation system, in accordance with some embodiments.

[0133] FIGS. 3A-3B are schematic diagrams of a first embodiment of a dynamic playing surface used in a sport simulation system, in accordance with some embodiments.

[0134] FIG. 3C is a schematic diagram showing a second embodiment (an alternative to the first embodiment) of a dynamic playing surface used in a sport simulation system, in accordance with some embodiments.

[0135] FIG. 4 is a side cross-sectional view of a dynamic target used in a sport simulation system, in accordance with some embodiments.

[0136] FIG. 5 is a functional block diagram of a ball delivery system, in accordance with some embodiments.

[0137] FIGS. 6A and 6B are flowcharts showing assembly, testing, and calibration of a sport simulation system, in accordance with some embodiments.

[0138] FIG. 7 is a flowchart of a method of managing a practice makes perfect game at a sport simulation system, in accordance with some embodiments.

[0139] FIG. 8 is a flowchart of a method of managing a round of golf at a sport simulation system, in accordance with some embodiments.

[0140] FIGS. 9A-9B are flowcharts of methods of playing a 6-Pack Challenge game at a sport simulation system, in accordance with some embodiments.

[0141] FIG. 10 is a flowchart of a method of managing a Laser Crunch game at a sport simulation system, in accordance with some embodiments.

[0142] FIG. 11 is a flowchart that identifies the hardware and software process required for the tracking function during a typical game played at a sport simulation system, in accordance with some embodiments.

[0143] FIG. 12 is a flowchart of the hardware and software process required for the guidance function during the 18 with a bullet game mode selected, in accordance with some embodiments.

[0144] FIGS. 13A-13C are schematics of a mobile device used to illustrate exemplary user interfaces for selecting various games or modes, in accordance with some embodiments.

[0145] FIG. 14 is a flowchart of a method of projecting guidance lines onto a playing surface, in accordance with some embodiments.

[0146] FIGS. 15A-15B are schematics of a best fit line projected onto a playing surface of a sport simulation system, in accordance with some embodiments.

[0147] FIGS. 15C-15K are coordinate frame diagrams of a playing surface used to illustrate proper determination of a best fit line, in accordance with some embodiments.

[0148] FIG. 16 is a flowchart of a method of managing a game of hand golf at a sport simulation system, in accordance with some embodiments.

[0149] FIG. 17 is a schematic of a putting green surface used to illustrate a robotic assistant projected on the putting green surface, in accordance with some embodiments.

[0150] FIGS. 18A-18B are schematics of a putting green surface used to illustrate a graphic projected on the surface of the putting green surface that illustrates changing height positions for linear actuators, in accordance with some embodiments.

[0151] FIG. 19A is a schematic diagram showing a physical gaming suite with anamorphic images projected therein, in accordance with some embodiments.

[0152] FIGS. 19B-19D are schematic diagrams showing how the anamorphic mages of FIG. 19A appear from different viewpoints within the physical gaming suite, in accordance with some embodiments.

[0153] FIGS. 20A-20B are schematic diagrams showing an example anamorphic image that is projected on a bottom surface of a physical gaming suite, in accordance with some embodiments.

[0154] FIGS. 20C-20E are schematic diagrams showing an example anamorphic image that has a first component anamorphic image projected on a bottom surface of a physical gaming suite and a second component anamorphic image projected on back surface of a physical gaming suite, in accordance with some embodiments.

[0155] FIGS. 20F-20G are schematic diagrams showing an example anamorphic image that is projected on a back surface of a physical gaming suite, in accordance with some embodiments.

[0156] FIGS. 21A-21C are flowcharts of a method of programmatically generating anamorphic images for presentation (and 3D viewing) in a physical gaming suite, in accordance with some embodiments.

[0157] FIG. 22 is a flowchart of a method of programmatically generating anamorphic images for presentation (and 3D viewing) in a physical gaming suite, in accordance with some embodiments.

DESCRIPTION OF EMBODIMENTS

[0158] As discussed above and in more detail below, there is a need for sport, gaming, and entertainment simulation systems that provide user-specific guidance, provide interactive 3D effects and features (without causing discomfort for users), and training using a dynamic playing surface. Disclosed herein are novel systems, games played using the systems, methods, and interfaces to address these needs.

[0159] Reference will now be made in detail to embodiments, examples of which are illustrated in the accompanying drawings. In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of the various described embodiments. However, it will be apparent to one of ordinary skill in the art that the various described embodiments may be practiced without these specific details. In other instances, well-known methods, procedures, components, circuits, and networks have not been described in detail so as not to unnecessarily obscure aspects of the embodiments.

[0160] It will also be understood that, although the terms first, second, etc. are, in some instances, used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. For example, a first contact could be termed a second contact, and, similarly, a second contact could be termed a first contact, without departing from the scope of the various described embodiments. The first contact and the second contact are both contacts, but they are not the same contact.

[0161] The terminology used in the description of the various described embodiments herein is for the purpose of describing particular embodiments only and is not intended to be limiting. As used in the description of the various described embodiments and the appended claims, the singular forms a, an, and the are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that the term and/or as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items. It will be further understood that the terms includes, including, comprises, and/or comprising, when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.

[0162] As used herein, the term if is, optionally, construed to mean when or upon or in response to determining or in response to detecting, depending on the context. Similarly, the phrase if it is determined or if [a stated condition or event] is detected is, optionally, construed to mean upon determining or in response to determining or upon detecting [the stated condition or event] or in response to detecting [the stated condition or event], depending on the context.

[0163] Attention is now directed to FIG. 1A, showing a block diagram of a sport simulation system, in accordance with some embodiments. As shown in FIG. 1A, a sport simulation system 100 includes a tracking system 102, audio system 112, gaming interfaces 110, a playing surface 104, a ball delivery system 108, and a system controller 114 that is coupled with each of the aforementioned components. In some embodiments, the sport simulation system 100 further includes environmental simulation devices 192 for simulating various virtual reality conditions.

[0164] In some embodiments, the tracking/guidance system 102 includes visual sensors 103 (e.g., visual sensors 103-1, 103-2, 103-3 . . . 103-n) and projectors 105 (e.g., projectors 105-1, 105-2, 105-3 . . . 105-n, also referred to herein as display devices or display screens 105-1 through 105-n). In some embodiments, the projectors 105 are display screens that are positioned within a physical gaming suite (e.g., physical gaming suite 1900, FIG. 19A). In some embodiments, the tracking/guidance system 102 is integrated, while in other embodiments, a separate tracking system and a separate guidance system is provided. For ease of explanation, references herein to tracking/guidance system 102 encompass embodiments in which the tracking and the guidance system are separate.

[0165] In some embodiments the visual sensors 103 and the projectors 105 are connected at various locations within the sports simulation system 100 (e.g., to the ceiling, to walls within the system, etc.) in order to provide enough angles to view all activities within the sports simulation system 100 and to project images at all angles within the sports simulation system 100 (e.g., including anamorphic images that are projected in 2D, such that one or more game participants are able to see the images in 3D).

[0166] In some embodiments, the tracking system 102 includes one or more high resolution digital cameras connected to a network of computers that perform complex computer-vision tracking of ball location, velocity, and associates each ball with a unique player. In some embodiments, the tracking system maintains a model of the playing field and state of each ball (e.g., locations of each ball on the playing surface 104) during game play. This allows for virtual or augmented reality features (see Laser Crunch game description below). In some embodiments, multiple cameras are used to attempt to keep the ball in a field of view at all times. Many players being on the green at the same time may occlude the view of one camera. In some embodiments, the tracking data from vision system is sent to a master system that will coordinate the data. In some embodiments, this master system will also have information about the topography of the green, location of the holes and the locations of virtual targets that are drawn by the master system.

[0167] In some embodiments, guidance system software is provided as a part of the tracking/guidance system 102, and this guidance system software uses classical mechanics (i.e., physics) to provide models for gravity, ballistics, and friction. As mentioned above, in some embodiments, the guidance system 102 has a model of the green topography. In some embodiments, using an optimizer/solver system allows the guidance system 102 to determine an optimal path, strike vector, and pendulum swing to help sink a putt on the playing surface 104. In some embodiments, guidance system software to determine these quantities solves numerous differential equations in near real-time to compute proper forces and trajectory for each putt. In some embodiments, a participant's progress (e.g., their skill level and improvement as they use the sport simulation system 100) is scored using the ball's trajectory and resting place relative to the hole and virtual targets (e.g., a more difficult made putt can be assigned more points than a simple or close-range made putt).

[0168] In some embodiments, the guidance system 102 also provides audio and visual aids that help guide and teach each participant how to putt. In some embodiments, the visual aids are provided using both digital projectors and 7-color (RGB) digital laser projectors that are programmed dynamically by the system controller 114 using a standard laser projector interface. In some embodiments, the audio component is supplied using a high fidelity public address system using digital audio samples that are stored and sequenced on a local server relative to the system controller 114. In some embodiments, the purpose of the visual aids is to provide a high contrast, easily visible, precise best fit line, ball target point (point to aim for on the playing surface 104, given a current topography of the playing surface 104), a putter back and forth motion required (pendulum), and a grid that identifies the contour of the green, which each depends on position of the ball with respect to the hole and the topography in between. In some embodiments, the visual component is established per putt/per player based upon image from camera imaging devices (e.g., cameras 103) that scan the entire green surface and those images are then used by the system controller 114 to determine how to render each of the aforementioned visual aids. In some embodiments, the visual system will also place concentric circles (e.g., concentric targets 310, FIG. 3C) that will have different points scale as you get closer to the hole and point values and locations/sizes of the concentric circles are determined based upon data from the cameras 103.

[0169] In some embodiments, the visual sensors 103 are configured to track and monitor participants within the sports simulation system 100, to associate participants with sports balls (e.g., golf balls), track paths followed by the sports balls, and to send information regarding the aforementioned to the system controller 114 (or one of its components, such as ball path determining module 226, ball color detection module 224, ball location determining module 236, FIG. 1C).

[0170] In some embodiments, the one or more projectors 105 are configured to render images onto a playing surface (e.g., playing surface 104) of the sport simulation system 100. Exemplary images that the projectors 105 are capable of projecting onto the playing surface 104 are described below in reference to FIGS. 15A-15B (an ideal ball path for a golf ball travelling towards a hole on a putting green surface), 17 (a robotic assistant that is used to follow a ball path travelled after a user putted a ball), and 18A-18B (a dynamic graphic that presents a visualization of actuators moving beneath the playing surface 104 while the playing surface 104 is changing its shape (e.g., either by tilting or by contouring the surface)). In some embodiments, the one or more projectors 105 are configured to adaptively present graphics so that the graphics render properly even while the playing surface 104 is changing shape.

[0171] In some embodiments, the visual sensors 103 are also configured to monitor visual characteristics associated with one or more game participants within the sports simulation system 100 (e.g., these monitored visual characteristics may include head position, physical position, eye game, and the like, which may be used to generate and then provide data to the projectors 105 that is used to display anamorphic images within the sports simulation system 100).

[0172] In some embodiments, the one or more projectors 105 are configured to render images onto a playing surface (e.g., playing surface 104) of the sport simulation system 100. Exemplary images that the projectors 105 are capable of projecting within the sports simulation system include anamorphic images that are projected in 2D for 3D viewing by one or more game participants within a physical gaming suite of the sports simulation system. Some example anamorphic images are described below in reference to FIGS. 19A-19D and FIGS. 20A-20G. In some embodiments, the one or more projectors 105 are configured to adaptively present anamorphic images so that the imagers are viewable in 3D even after participants have moved within the gaming suite, gaming events have been recognized (e.g., a user strikes a golf ball and sees 3D flames following the golf ball), and/or while the playing surface 104 is changing shape.

[0173] In some embodiments, the audio system 112 receives instructions from the system controller 114 to provide audio feedback to participants in games conducted at the sport simulation system 100. For example, in response to a respective participant sinking a challenging putt, the system controller 114 sends instructions to the audio system 112 to provide encouraging feedback to the participant (such as cheering noise). As another example, in response to a respective participant missing a putt, the system controller 114 sends instructions to the audio system 112 to provide instructional feedback to the participant (e.g., instructions on how to improve their putting stroke, align their feet properly, or other ways to improve their putting skills). In some embodiments and also in response to a respective participant missing a putt, the system controller 114 sends instructions to the projectors 105 to render video feedback (in conjunction with the instructional feedback) that supplements the auditory feedback (e.g., a video showing the participant's missed putting stroke and information about what aspects of the missed putting stroke caused the participant to miss the putt (such as their feet were improperly aligned or their struck the ball with too much force)). In some embodiments, guidance information (such as a best fit line or best path for a golf ball) is projected onto the surface by the projectors 105 and this guidance information also helps to improve participant's chances of making a shot (additional information regarding projecting best path graphics is presented below in references to FIGS. 15A-15K).

[0174] In some embodiments, the gaming interfaces 110 include leaderboards 110-1, simulators 110-2, and mobile devices 110-3. In some embodiments, the leaderboards 110-1 present scoring information for each player of a respective game being played at the sports simulation system 100 (e.g., points allocated to each player of a game of hand golf, as discussed below in reference to FIG. 16. In some embodiments, the leaderboards 110-1 are displayed on large display in front of the participants and may also be accessible through mobile devices 110-3 (e.g., by the mobile phones held by the individual participants). In some embodiments, the simulations 110-2 include golf simulators (or other ball tracking technology) that allows participants to simulate a round of 18-hole golf. In some embodiments, the simulators 110-2 provide, to the system controller 114, information about a location of a respective participant's last shot (e.g., an approach shot to a virtual putting green of a simulated golf hole). In response to receiving the provided information, the system controller 114 determines an appropriate topography for the playing surface 104 (in order to simulate a topography of the virtual putting green based on the respective participant's approach shot and its landing location on the virtual green).

[0175] In some embodiments, the mobile devices 110-3 include touch-sensitive displays that allow users to interact with and control the sports simulation system 100 (e.g., to select new games or view other data available through the system 100, as discussed below for example in reference to FIGS. 13A-13C).

[0176] In some embodiments, the playing surface 104 is a dynamic playing surface 104 that is capable of simulating a variety of various putting shots (as discussed below). In some embodiments, the dynamic playing surface 104 is capable of contouring to match topographies of real-life greens (e.g., by configuring one or more surface modification elements 106 to produce a desired topography at the playing surface 104, discussed below in reference to FIG. 3C). In other embodiments, the dynamic playing surface 104 is rigid and incapable of contouring, but is capable of tilting to simulate a variety of putting shots (e.g., as discussed below in reference to FIGS. 3A-3B). In some embodiments, the playing surface 104 is a fully actuated assembly controlled by the system controller 114 to be shaped into many different topographies. In some embodiments, the playing surface 104 is configured to be tipped, tilted, and/or contoured to replicate any putting green. In some embodiments, the playing surface 104 is rigid and is only capable of being tilted (as explained below in reference to FIGS. 3A-3C). In some embodiments, the playing surface 104 is controlled by many surface modification elements 106 (e.g., linear actuators) that push and pull on a sub-floor to make peaks and valleys on the playing surface 104. In some embodiments, the playing surface 104 includes a compliance layer (e.g., compliance layer 304, FIG. 3C) that moves with the contouring and provides compliance and density that a real green would have. In some embodiments, above the compliance layer there, a green surface (e.g., top surface 302, FIG. 3C) is provided that has surface roughness and properties of a real green to establish proper response of a golf ball rolling over it. In some embodiments, there are multiple holes that can be used independently depending upon the putting green configuration requested. In some embodiments, there can be an oversized hole to make it fun and a little easier for beginner golfers or children.

[0177] In some embodiments, the playing surface 104 is coupled with a hitting mat 116 that deploys over the playing surface 104 at an appropriate time (e.g., when the user is hitting an iron shot, a chip shot, or a drive) and goes to a storage position when the user is putting. Hitting mat 116 is discussed in more detail below in reference to FIGS. 2A-2C. In some embodiments, one or more of hitting mats 116 include a portion (e.g., a sub-component) of a ball delivery system 108, so that balls are quickly and easily delivered to the one or more hitting mats 116.

[0178] In some embodiments, the ball delivery system 108 is configured to send balls onto the playing surface 104. In some embodiments, the ball delivery system 108 sends the balls to predetermined locations on the playing surface 104. The predetermined locations are based on practice spot locations, game-specific putting locations, and the like. In some embodiments, the predetermined locations are based on where each respective participant hit their golf ball while using the simulators 110-2 and the ball delivery system 108 sends balls to predetermined locations corresponding to where each participant hit their golf ball in the simulator. In some embodiments, the ball delivery system 108 is configured to change ball delivery settings in order to achieve controlled ball delivery (such as speed, spin, launch angle) and, in some embodiments, the system controller 114 communicates with the visual sensors 103 in order to verify that balls have reached the predetermined locations. In accordance with a determination that a ball has not reached its corresponding predetermined location, the controller 114 instructs the surface modification elements 106 so that tilting or contouring the playing surface 104 will move the ball to the corresponding predetermined location.

[0179] In some embodiments, the ball delivery system 108 ensures that participants are not running all over to fetch golf balls on the playing surface 104. In some embodiments, the ball delivery system 108 is configured to place balls on the playing surface 104 based upon final shots on a VR simulator. In some embodiments, the ball delivery system 108 accounts for interaction of balls at edges of the playing surface 104.

[0180] In some embodiments, the ball delivery system 108 also delivers balls to a predetermined location for each participant to make multiple shots from the predetermined location (e.g., a difficult putt location from which a respective participant has missed many previous putts) while in a practice mode for the sport simulation system 100. In some embodiments, the ball management system that will funnel a ball to the hole for beginner players or kids to make it a positive experience. In some embodiments, bumpers are provided on the playing surface 104 in a children's practice mode to add an extra challenge. In these embodiments, cameras 103 and the guidance system 102 as a whole accommodates for the presence of the bumpers, while determining how to render visual aids on the playing surface 104.

[0181] In some embodiments, the system controller 114 includes a management module 114-1, 114-2 that includes a number of modules that are responsible for exchanging information with each of the components of the sports simulation system (additional details are provided below in reference to FIGS. 1C and 1D).

[0182] In some embodiments, the system controller 114 interfaces (e.g., using gaming components interface 214, FIGS. 1C and 1D) with a mobile device that is provided to a participant and includes user interfaces that allow participants to get established, system mode to be selected, and selection of a type of game to be played at the sport simulation system 100 (e.g., using one or more of the gaming user interfaces shown in FIGS. 13A-13C). In some embodiments, there is provided a touch display that the game and guidance systems can use to display video capture of putts from each participant, video display of a proper putting motion, scores from either games or putting evaluations after a putt from a respective participant. In some embodiments, a tablet system is provided that any participant can use to command different features of each mode at the sport simulation system 100.

[0183] In some embodiments, the system controller 114 interfaces with a Virtual Reality Golf Simulator (e.g., one of the simulators 110-2, FIG. 1A) that identifies a topography of specific putting green the players are on for an 18-hole course that is being played. In some embodiments, the system controller 114 also interfaces (e.g., using one or more playing surface interfaces 212, FIGS. 1C and 1D) with one or more surface modification elements 106 (e.g., one or more linear actuators) below the playing surface 104 to push and pull to create the identified topography for the specific putting green.

[0184] In some embodiments, the system controller 114 additionally interfaces with hole actuators (e.g., one or the surface modification elements 106 that is coupled with a removable, substantially circular portion of the playing surface 104) to enable correct hole position for the specific putting green being played. Any putting green can be simulated with the surface modification elements 106 (e.g., surface and hole control actuators). Once a respective simulator 110-2 indicates that that a shot made it to a putting green of a simulated golf course hole, the system controller 114 instructs a ball delivery system 108 to delivers a golf ball to a position on the playing surface 104 that matches the location reached by the shot. In some embodiments, the ball delivery system 108 is instructed to deliver balls to the playing surface 104 for all participants in a current 18-hole simulated golf round. In some embodiments, system controller 114, takes images using the visual sensors 103 (e.g., one or more cameras) to determine positions for each player's putt.

[0185] In some embodiments, the sport simulation system is configured to operate in multiple modes based on experience levels for the game participants. In accordance with a determination that a current mode of operation for the sport simulation system 100 is a training mode, the projectors 105 (also used to control a guidance system) is controlled (by the system controller 114) to show a best fit line (also referred to interchangeably herein as a best fit curve, best path, best putting path, and ideal putting line) from the ball to a hole on the playing surface 104. While in training mode, the projectors 105 are also instructed, in some embodiments, to render a ball target on the playing surface 104 (e.g., target alignment graphic 1502, FIG. 15B), a putter back-and-forth distance and speed required (e.g., backswing graphic 1508 and follow-through graphic 1506, FIG. 15B), and a grid showing the lay of the green or sloping gradient of the green. In some embodiments, colors or water movement graphics are projected onto the playing surface 104 in order to show topography of the playing surface 104. In some embodiments, the grid is utilized in combination with colors or water movement while, in other embodiments, only one graphic is utilized to show the topography of the playing surface 104. In some embodiments, the aforementioned graphics are provided for each participant in the game. In this way, a guidance system is provided that teaches or trains a non-golfer or even an experienced golfer how to play or better their short games and read greens.

[0186] In some embodiments, the guidance system further includes micro-chips located in each golf ball utilized with sport simulation system 100 to interface with the system controller 114 and inform each player what went wrong during a putt and to advise on possible corrective actions. In some embodiments, the golf balls do not include micro-chips.

[0187] In some embodiments, system controller 114 also interfaces with an audio system (e.g., using audio system interface 216, FIGS. 1C and 1D) to give feedback to each participant on their training progress and tips for subsequent shots, if required. In some embodiments, the audio system 112 is controlled by the system controller 114 and is instructed to output various phrases as the ball travels to the hole. For example, there can be audio for looks like a good putt at the start, to cheers as the ball gets close, to Aws if the putt is missed, and the like.

[0188] Attention is now directed to FIG. 1B, presenting a schematic diagram of the sport simulation system 100 described above in reference to FIG. 1A. In some embodiments, a base and frame is provided that houses numerous surface modification elements 106 (e.g., numerous linear actuators) that change the topography of a playing surface 104 (also referred to as a dynamic surface 104) that is moved by the surface modification elements 106. In some embodiments, the playing surface 104 includes a putting green surface 104-1, a compliance layer 104-2, and a sub-floor structure 104-3. The components and make-up of the playing surface 104 is described in more detail below in reference to FIGS. 3A-3B (embodiment in which the playing surface 104 is rigid, not contour-able, but still tilt-able) and FIG. 3C (embodiment in which the playing surface 104 is contour-able).

[0189] In some embodiments, the base system of the playing surface 104 is constructed from steel or aluminum and welded or otherwise fastened together. In some embodiments, the surface modification elements (e.g., deck linear actuators 305 shown in FIGS. 1B and 3B, such as hydraulic or pneumatic linear actuators). In these embodiments, the actuators interface to the framing system and are bolted in place. In some embodiments, the surface modification elements are below three layers of the playing surface 104 (e.g., a first layer such as a putting green, a second layer such as a compliance layer, a third layer such as sub-floor structure). Additional information regarding the three layers is provided in reference to FIG. 3C.

[0190] In some embodiments, the number of actuators required below the three layers is dependent upon the structural support required and the contour needed to generate different green shapes and multiple configurations. In some embodiments, only three actuators are utilized to tilt a rigid playing surface 104 (as explained below in reference to FIGS. 3A-3B) while, in other embodiments, numerous actuators are utilized to contour a flexible playing surface 104 (as explained below in reference to FIG. 3C).

[0191] In some embodiments, the playing surface 104 is positioned over a movable underlying layer (including but not limited to the actuators, bearings, compliance layer, sub floor structure, and/or any other materials and mechanisms which allow the overlying surface to change topography while bearing weight of up to 12 players). Additional information regarding layers of the playing surface 104 is provided in reference to FIG. 3C.

[0192] In some embodiments, the movable underlying layer is coupled to each actuator to allow the actuators to push and pull the floor to distort the surface topography. In some embodiments, a compliance layer (e.g., compliance layer 304, FIG. 3C) made of foam that is of an appropriate density and is laminated to the subfloor (e.g., sub-floor structure 306, FIG. 3C). In some embodiments, the density of the compliance layer will replicate the density of an actual putting green. In some embodiments, the final layer is the green layer or the top surface (e.g., top surface 302, FIG. 3C) and that is laminated to the compliance layer. In some embodiments, the top green layer is selected to meet the surface properties of an actual green so that the golf ball motion is very similar to a real putting green.

[0193] In some embodiments, the controller or computer system (e.g., system controller 114, FIGS. 1A and 1B) is a commercial-off-the-shelf high speed rack mounted processing system. In some embodiments, the controller 114 has interfaces to all the peripherals (e.g., tracking system interface 204, audio system interface 216, ball delivery interface 210, playing surface interface 212, and gaming components interface 214, FIG. 1C), memory for image processing and control algorithms (e.g., modules included in memory 206, FIG. 1C), and houses the control software in non-volatile memory (e.g., memory 206, FIG. 1C) that can be reprogrammed during testing and integration.

[0194] In some embodiments, an audio and visual guidance system (e.g., tracking system 102, FIG. 1A, including cameras 103 and projectors 105 shown in both FIGS. 1A and 1B) is located above the playing surface 104. In some embodiments, a camera system is coupled to the ceiling (e.g., cameras 103). In some embodiments, a lighting system (e.g., as a component or additional piece of projectors 105) is coupled to the ceiling and the audio system. In some embodiments, the interfaces are RF or wireless to reduce cabling between the guidance system and the control computer.

[0195] Some embodiments include multiple hole locations under the different green configurations. For example, there may be a current hole being used and other holes not currently being used (as shown in FIG. 1B). These embodiments may include a ball management system; ball delivery system (e.g., ball delivery system 108); a ball funneling system (described in more detail in reference to FIG. 4); a ball retrieval system (described in more detail in reference to FIG. 5); ball bumpers; etc.

[0196] Also in some embodiments, the system controller 114 is configured to move the movable underlying layer (e.g., the surface modifying layer 308, FIG. 3C). This controller includes a processor and memory for executing instructions for controlling surface topography (shown in FIG. 1C). An interface, such as a touch screen interface, is provided for player input. The controller (or control computer) and its components are shown in FIGS. 1C and 1D.

[0197] In some embodiments, a guidance system may also be provided (e.g., using projectors 105 of the tracking system 102, FIG. 1A). The guidance system may include a doppler system; a camera; one or more displays; and a visual guidance system for overlaying lines, circles, symbols, numbers, and annotations over the topography. In some embodiments, the lines include a grid, gradient/contour lines, a best fit line, a stroke length line-back; a stroke length line-front; an ideal ball path; a ball target point; a variable speed ball path; etc. all overlaid over the topography. The guidance system may also calculate and provide a distance-to-hole measurement; an elevation measurement; computer generated imagery for ball and surface technology for player feedback; etc. The guidance system may also include guidance system-audio; a controller; ball microchip technology for ball motion and for training; instructions for visual guidance based upon ball location from hole and surface properties; instructions for generating audio feedback based upon closed loop camera interface; instructions for mode control; a virtual reality simulator interface.

[0198] FIG. 1C is a block diagram illustrating a management module 114-1, in accordance with some embodiments of the sport and game simulation system shown in FIGS. 1A and 1B. Management module 114-1 typically includes one or more processing units (sometimes called CPUs or processors) 202-1 for executing modules, programs, and/or instructions stored in memory 206 (and thereby performing processing operations), memory 206 (sometimes called controller memory), and one or more communication buses 208 for interconnecting these components. The one or more communication buses 208 optionally include circuitry (sometimes called a chipset) that interconnects and controls communications between system components. Management module 114-1 is coupled to tracking system interface 204 (to allow the system controller 114 to exchange information and provide instructions to the tracking/guidance system 102), audio system interface 216 (to allow the system controller 114 to exchange information and provide instructions to the audio system 112), ball delivery interface 210 (to allow the system controller 114 to exchange information and provide instructions to the ball delivery system 108), playing surface interface 212 (to allow the system controller 114 to provide instructions to the playing surface 104 and the surface modification elements 106), and gaming components interface 214 (to allow the system controller 114 to exchange information and provide instructions to the leaderboards 110-1, simulators 110-2, and the mobile devices 110-3).

[0199] Memory 206 includes high-speed random access memory, such as DRAM, SRAM, DDR RAM or other random access solid state memory devices, and may include non-volatile memory, such as one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, or other non-volatile solid state storage devices. Memory 206 optionally includes one or more storage devices remotely located from the CPU(s) 202-1. Memory 206, or alternatively the non-volatile memory device(s) within memory 206, comprises a non-transitory computer readable storage medium.

[0200] In some embodiments, memory 206, or the non-transitory computer-readable storage medium of memory 206 stores the following programs, modules, and data structures, or a subset or superset thereof: [0201] surface modification module 222 for providing instructions to one or more surface modification elements (e.g., surface modification elements 106, FIG. 1A) in order to contour or tilt a playing surface (e.g., playing surface 104); [0202] ball color detection module 224 for detecting colors of balls on a playing surface (e.g., playing surface 104, FIG. 1A) and for detecting colors of balls as they pass through a hole on the playing surface; [0203] ball path determining module 226 for determining an ideal path from a position on a playing surface (e.g., playing surface 104, FIG. 1A) at which a ball is currently located and to a hole on the playing surface; [0204] graphics rendering module 228 for providing instructions to one or more projecting devices (e.g., one or more projectors 105, FIG. 1A) in order to render graphics on a playing surface (e.g., playing surface 104, FIG. 1A); [0205] game selection module 230 for providing user interfaces (e.g., gaming user interfaces shown in FIGS. 13A-13C) that allow participants to control operations and gaming modes at a sport simulation system (e.g., sport simulation system 100, FIG. 1A); [0206] ball delivery module 232 for determining an appropriate location on a playing surface at which to deliver a ball and sending instructions to a ball delivery system (e.g., ball delivery system 108) so that the ball delivery system is able to deliver the ball to the appropriate location (e.g., sending information about speed, launch angle, spin, etc. to send the ball to the appropriate location); [0207] ball retrieval module 234 for retrieving balls from a playing surface (e.g., playing surface 104, FIG. 1A); [0208] ball location determining module 236 for identifying locations of balls on a playing surface (e.g., playing surface 104, FIG. 1A); [0209] user-specific putting data 218 for storing information about putting characteristics and historical putts for each respective participant at a sport simulation system (e.g., sport simulation system 100, FIG. 1A) so that the stored information is available for future processing and use by the sport simulation system (e.g., to provide more accurate and user-tailored best fit lines); and [0210] leaderboard updating module 220 for updating a leaderboard (e.g., one of the leaderboards 110-1) to include point data for each participant in a particular game at a sport simulation system (e.g., sport simulation system 100, FIG. 1A).

[0211] Each of the above identified elements may be stored in one or more of the previously mentioned memory devices, and corresponds to a set of instructions for performing a function described above. The above identified modules or programs (i.e., sets of instructions) need not be implemented as separate software programs, procedures or modules, and thus various subsets of these modules may be combined or otherwise re-arranged in various embodiments. In some embodiments, memory 206 may store a subset of the modules and data structures identified above. Furthermore, memory 206 may store additional modules and data structures not described above. In some embodiments, the programs, modules, and data structures stored in memory 206, or the non-transitory computer readable storage medium of memory 206, provide instructions for implementing some of the methods described below. In some embodiments, some or all of these modules may be implemented with specialized hardware circuits that subsume part or all of the module functionality.

[0212] Although FIG. 1C shows management module 114-1 in accordance with some embodiments, FIG. 1C is intended more as a functional description of the various features which may be present in management module 114-1 than as a structural schematic of the embodiments described herein. In practice, and as recognized by those of ordinary skill in the art, the programs, modules, and data structures shown separately could be combined and some programs, modules, and data structures could be separated.

[0213] FIG. 1D is a block diagram illustrating a management module 114-2, in accordance with some embodiments of the sport and game simulation system shown in FIGS. 1A and 1B. Management module 114-2 typically includes one or more processing units (sometimes called CPUs or processors) 202-2 for executing modules, programs, and/or instructions stored in memory 207 (and thereby performing processing operations), memory 207 (sometimes called controller memory), and one or more communication buses 208 for interconnecting these components. The one or more communication buses 208 optionally include circuitry (sometimes called a chipset) that interconnects and controls communications between system components. Management module 114-2 is coupled to tracking system interface 204 (to allow the system controller 114 to exchange information and provide instructions to the tracking/guidance system 102), audio system interface 216 (to allow the system controller 114 to exchange information and provide instructions to the audio system 112), ball delivery interface 210 (to allow the system controller 114 to exchange information and provide instructions to the ball delivery system 108), playing surface interface 212 (to allow the system controller 114 to provide instructions to the playing surface 104 and the surface modification elements 106), gaming components interface 214 (to allow the system controller 114 to exchange information and provide instructions to the leaderboards 110-1, simulators 110-2, and the mobile devices 110-3), and environmental simulation devices interface(s) 270 (to allow the system controller 114 to exchange information and provide instructions to the environmental simulation devices 192, FIG. 1E).

[0214] Memory 207 includes high-speed random access memory, such as DRAM, SRAM, DDR RAM or other random access solid state memory devices, and may include non-volatile memory, such as one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, or other non-volatile solid state storage devices. Memory 207 optionally includes one or more storage devices remotely located from the CPU(s) 202-2. Memory 207, or alternatively the non-volatile memory device(s) within memory 206, comprises a non-transitory computer readable storage medium.

[0215] In some embodiments, memory 207, or the non-transitory computer-readable storage medium of memory 207 stores the following programs, modules, and data structures, or a subset or superset thereof: [0216] surface modification module 222 for providing instructions to one or more surface modification elements (e.g., surface modification elements 106, FIG. 1A) in order to contour or tilt a playing surface (e.g., playing surface 104); [0217] ball path determining module 226 for determining an ideal path from a position on a playing surface (e.g., playing surface 104, FIG. 1A) at which a ball is currently located and to a hole on the playing surface; [0218] game selection module 230 for providing user interfaces that allow participants to control operations and gaming modes at a sport simulation system (e.g., sport simulation system 100, FIG. 1A); [0219] ball location determining module 236 for identifying locations of balls on a playing surface (e.g., playing surface 104, FIG. 1A); [0220] user-specific putting data 218 for storing information about putting characteristics and historical putts for each respective participant at a sport simulation system (e.g., sport simulation system 100, FIG. 1A) so that the stored information is available for future processing and use by the sport simulation system (e.g., to provide more accurate and user-tailored best fit lines); [0221] leaderboard updating module 220 for updating a leaderboard (e.g., one of the leaderboards 110-1) to include point data for each participant in a particular game at a sport simulation system (e.g., sport simulation system 100, FIG. 1A); [0222] graphics rendering module 228 for providing instructions to one or more projecting devices (e.g., one or more projectors 105, FIG. 1A) in order to render graphics (e.g., generated anamorphic images that are based on image data 228-1) on a playing surface (e.g., playing surface 104, FIG. 1A); [0223] visual characteristics monitoring module 250 for monitoring visual characteristics associated with one or more game participants within a physical gaming suite; and [0224] viewpoint determining module 252 for determining respective viewpoints for each game participant and for determining an average/common viewpoint for two or more game participants (also referred to herein as an optimal viewpoint).

[0225] In some embodiments, memory 207 of management module 114-2 also includes one or more of: an operating system that includes procedures for handling various basic system services and for performing hardware dependent tasks; a network communication module that is used for connecting the controller 114 to other subsystems of the system 100 via the one or more network interfaces (wired or wireless) to one or more networks; a VR module that communicates with the subsystems to receive information tracked by the subsystems (e.g., eye gaze and movement of each user and motion of golf balls tracked by the high speed cameras, voice commands received by the microphones, etc.), and to process and provide data related to the VR experience to the users (e.g., 3D images to be projected to each user, 3D sound effects to be played to the user, etc.); and a VR database 242 that stores data related to simulated scenes for different games and user profiles. In some embodiments, memory 206 also includes a user profile, which stores user records, each including, for example, a user profile (e.g., a user ID, an account name, login credentials, and/or custom parameters such as a user's age, a user's home location, and/or one or more parameters indicating interests of the user), custom parameters for the user (e.g., age, location, hobbies, etc.), social network contacts, groups of contacts to which the user belongs, and identified trends and/or likes/dislikes of the user.

[0226] In some embodiments, memory 206 also includes Golf Course Library 244, which stores preset data that is used to simulate different golf courses, including, but is not limited to: Golf Scenery Environment Data that is related to various golf course views, environment sounds (e.g., bird chirping or ocean breeze), and scents (e.g., fresh cut grass scent); Putting Surface Data that is related to topography, textures, roughness, and hole layouts of various putting surfaces; and Tournament Sound Effect Data that is related to sound effect (e.g., crowd cheering) that provides a simulated tournament sound experience to the user. Memory 206 may also include data for games other that golf, such as Surfing Library which stores preset data that is used to simulate various surfing environments at different beaches; Snowboarding Library which stores preset data that is used to simulate various snowboarding environments at different resorts or mountains; and Touring Library 256 which stores preset data that is used to simulate tourism spots all around the world at present, a historical time, or a future time.

[0227] Examples of one or more networks (e.g., connecting controller 114 to the various devices that are positioned within the system 100) include local area networks (LAN) and wide area networks (WAN) such as the Internet. One or more networks are, optionally, implemented using any known network protocol, including various wired or wireless protocols, such as Ethernet, Universal Serial Bus (USB), FIREWIRE, Global System for Mobile Communications (GSM), Enhanced Data GSM Environment (EDGE), code division multiple access (CDMA), time division multiple access (TDMA), Bluetooth, Wi-Fi, voice over Internet Protocol (VoIP), Wi-MAX, or any other suitable communication protocol.

[0228] Each of the above-identified elements may be stored in one or more of the previously mentioned memory devices, and corresponds to a set of instructions for performing a function described above. The above identified modules or programs (i.e., sets of instructions) need not be implemented as separate software programs, procedures or modules, and thus various subsets of these modules may be combined or otherwise re-arranged in various embodiments. In some embodiments, memory 206 may store a subset of the modules and data structures identified above. Furthermore, memory 206 may store additional modules and data structures not described above. In some embodiments, the programs, modules, and data structures stored in memory 206, or the non-transitory computer readable storage medium of memory 206, provide instructions for implementing some of the methods described below. In some embodiments, some or all of these modules may be implemented with specialized hardware circuits that subsume part or all of the module functionality.

[0229] Although FIG. 1D shows management module 114-2 in accordance with some embodiments, FIG. 1D is intended more as a functional description of the various features which may be present in management module 114-2 than as a structural schematic of the embodiments described herein. In practice, and as recognized by those of ordinary skill in the art, the programs, modules, and data structures shown separately could be combined and some programs, modules, and data structures could be separated.

[0230] As shown in FIGS. 1A-1B, in some embodiments, the sport simulation system 100 includes a number of environmental simulation devices 192 that are positioned within the sport simulation system 100. FIG. 1E is a schematic diagram showing environmental simulation devices 192 that are part of the sport simulation system 100 in accordance with some embodiments.

[0231] As shown in FIG. 1E, the environmental simulation devices 192 include lighting input/output modules 180 (e.g., for controlling visual ambiance within a physical gaming suite), audio input/output modules 181 (e.g., including audio system 112 and one or more microphones, speakers, and the like), video input/output modules 182 (e.g., including projectors 105 and cameras 103), olfactory input/output modules 183 (e.g., including one or more odor emitters and particle counters), humidity input/output modules 184 (e.g., including one or more humidifiers and misters), temperature input/output modules 185 (e.g., including one or more a/c and heating units), metabolic input/output modules 186 (e.g., including one or more human body sensors), heat mapping movement modules 187, and kinetic tracking modules 188 for floor kinetic tracking.

[0232] In some embodiments, the above environmental simulation modules are positioned within a physical gaming suite that includes many (or all) of the components described above with respect to FIGS. 1A-1B (e.g., physically coupled with a portion of a framing system for the physical gaming suite). In some embodiments, the floors and walls of the physical gaming suite (e.g., such as physical gaming suite 300, FIG. 3A) are made of many different types of materials, by a supply system for materials with quantity and quality controlled in order to insure that surface shape is configurable (e.g., tilt-able and/or contour-able as explained below in reference to FIGS. 2A-2C), resulting in physical surfaces that are both aesthetically-pleasing and highly functional for producing an immersive VR experience.

[0233] In some embodiments, the kinetic tracking modules 188 and the heat mapping movement modules 187 comprise a plurality of floor sensors, visible light cameras, and infrared cameras. In some embodiments, the cameras and floor sensors track and monitor motions of the players during the game play to generate data. In some embodiments, the floor sensors monitor the motion of the player to analyze driving and putting motions and to monitor the game play. In some embodiments, the kinetic tracking system collects data during putting and driving, and the collected data is processed and used to give the player tips on their swing and putting to improve the player's skills and form good habits.

[0234] In some embodiments, the images taken by both visible light cameras and infrared cameras are combined and stitched to provide information in both visible physical features and human body thermal mapping. In some embodiments, the thermal images present intensities of human body thermal signature characteristics corresponding to intensities of thermal response of various body parts. In some embodiments, facial feature thermal mapping is used to determine head direction and position. In some embodiments, high rate visible images are the primary source for eye tracking.

[0235] In the system 100, both the visible and infrared images are used for eye and head tracking. In some embodiments, the floor sensors track the player's motion and position information during a game play via the player's feet position during putting and swinging a club and weight transfer during club motion. In some embodiments, the system uses the images taken by the cameras and the information tracked by the sensors to provide feedback to the player to improve their swing and putting to advance their game. In some embodiments, the system performs complex image processing and swing analytics for club and putting, to provide tips to improve the player's form and to enhance their game.

[0236] The system 100 also uses the environmental simulation devices 192 to provide environmental effects that enhance a game participant's experience and interaction with the system. In some embodiments, the temperature input/output modules 185 are used to adjust temperatures according to a simulated location for a game participant during a current game, for example, ranging from conditions at Pebble Beach, Calif. in the summer to a golf course in Ireland in the winter. In some embodiments, the lighting input/output modules 180 also adjust lighting for golf play in the morning, afternoon, or evening, while taking into account random and changing variables such as clouds. In some embodiments, the humidity input/output modules 184 adjust humidity depending upon the simulated game played in different locations, for example, in Las Vegas or Florida. In some embodiments, the environmental simulation devices 192 are also utilized to generate gentle wind to simulate the general environment at a physical golf course.

[0237] In some embodiments, the user predefines a set of environmental variables in which to practice, in order to improve their golf skills. In some embodiments, the environmental effects are monitored via sensors. In some embodiments, the computer system controls sensors to monitor humidity, temperature, wind speed, and/or lighting of the game space. In some embodiments, the computer system controls the environmental conditions by interfacing with the HVAC system (e.g., via temperature input/output modules 185), wind generation system, humidity generation system (e.g., via humidity input/output modules 184), and lighting system (e.g., via lighting input/output modules 180). In some embodiments, as the clouds go by in the simulated environment, the system adjusts the lighting, temperature, and projection accordingly to simulate environmental conditions during an outdoor game of golf.

[0238] In some embodiments, sounds within the sports simulation system 100 are also controlled in order to provide and improve the immersive virtual reality experience during game play. In particular, a sound transforms a game participant's feeling from happiness, to nervousness, to excitement. Along with tactile interactions within the sport simulation system, sound allows game participants to immerse themselves into the reality of game play. In some embodiments, based upon where the golf course is and what hole the players are on, the audio input/output modules 181 (e.g., the audio system 112, FIG. 1A) adjusts sound coming from within the virtual environment accordingly. In an example, when a putt is made, the sound of cheering from a crowd is generated and provided. In another example, the sound of the ocean when the user is playing at a golf course next to a sea cliff is generated and provided. In another example, the modules 181 generate a sound of birds chirping on the course. In yet another example, the modules 181 generate a sound of a fountain next to the clubhouse at the last hole (e.g., and provide those sounds via audio system 112). In some embodiments, the modules 181 generate the sound of the wind through the trees.

[0239] Smell is another sense that evokes emotion. For example, when a person passes a French bakery and smells that fresh baked croissant, this person may feel happy and at peace in response. In some embodiments, the olfactory input/output modules 183 replicate smells based upon gaming data for a particular game participant. For example, the modules 183, when implemented in the system 100, generate the smell of fresh cut grass, the smell of a flower garden, the smell of the green next to the ocean, the smell of the pine trees next to the fairway, and/or the smell of cooked hotdogs and burgers at the end of the 18th hole located at the club house. The olfactory input/output modules 183 invoke the human sense of smell and thus further enhance and improve user experiences in the system 100.

[0240] In some embodiments, audio input/output modules 181 also include one or more microphones. In some embodiments, the one or more microphones are integrated within the system 100 for voice recognition as well as for system commanding and control. In some embodiments, voice recognition is used for the system 100 to identify user identity, the user profile, the user database, the user golf driving and putting habits, and/or the user long-term positive and negative trends associate with golf play, such that the system 100 customizes tips associated with game play as the user continues to use the system. In some embodiments, the modules 181 are configured to personally addresses users and act as a caddie with tips to help with game play, and the assistance provided changes over time as the player advances his or her game play. In some embodiments, the game participants use voice commands to control game play within the system 100. In some embodiments, the system 100 provides further practice, repeats putts or approach shots, and shows replays to the user using user's voice commands to enhance user experience.

[0241] In some embodiments, the one or more microphones are also utilized to collect feedback about presentation of images within the system 100. For example, after a 2D image is presented for 3D viewing within the physical gaming suite, the one or more microphones detect and collect feedback from game participants related to the presentation of the images (e.g., whether the 3D effect is perceived, whether the image is presented too brightly or takes up too much space, and the like). After collecting the auditory feedback, the system 100 analyzes the feedback and uses it to improve future presentation of 2D images within the physical gaming suite (e.g., in order to improve appreciation of a 3D effect by the game participants).

[0242] In some embodiments, the environmental simulation devices 192 are also used to project 2D images (e.g., anamorphic images) for 3D viewing by game participants within a physical gaming suite. For example, some embodiments provide a real-time visual subsystem (e.g., visual input/output modules 182) within the sports simulation system 100 described above and the real-time visual subsystem comprises a plurality of digital projectors (e.g., multiple projectors 105 of FIG. 1A) for projecting 3D images onto a 2-dimensional (2D) space (e.g., a wall or the ceiling of a physical gaming suite). In some embodiments, the projectors project images on one, two, three, or four walls (and/or a ceiling) to present 2D images for 3D viewing by one or more game participants within the physical gaming suite (as described in detail below in reference to FIGS. 19A-19D, 20A-4G, 21A-21C, and 22).

[0243] In some embodiments, the images projected by the plurality of projectors are constantly updated based upon the data tracked by one or more high speed digital cameras (e.g., one or more of visual sensors 103, FIG. 1A) using a tracking algorithm. For example, while playing a golf game in a physical gaming suite, the plurality of projectors present to a player dynamic views that are related to Pebble Beach hole one from the tee location, on the fairway for drives and approach shots, and all the way up to the putting green, so that the player has a real and immersive 3D experience as if the player is playing at the Pebble Beach.

[0244] In some embodiments, the sports simulation system 100 accommodates one or more players, e.g., from 1 to 4 players. In some embodiments, while the players are playing a game within the physical gaming suite, the system uses an optimization algorithm to generate 2D images (e.g., anamorphic images) that are concurrently presented to each player and projects the generated 2D images on one or more surfaces of the physical gaming suite. In some embodiments, the optimization algorithm is used to optimize projected views based upon the images collected from the cameras 103. For example, the optimization is performed based upon where each player is standing in the game space, and where the eyes of each player are looking (or an average estimate of an optimal viewpoint based on visual characteristics associated with multiple game participants, as described below in reference to FIGS. 19A-19D, 20A-4G, 21A-21C). In some embodiments, the optimization algorithm uses images tracked by the high rate digital cameras and data received from the cameras and motion sensors in the system.

[0245] In some embodiments, synchronized high rate (e.g., >30 Hz) cameras are placed overhead and at discrete locations around the game space. In some embodiments, the cameras capture the players' interactions with the environment. In some embodiments, the number and positions of the cameras are designed to allow a plurality of cameras to view any object in the game space simultaneously. In some embodiments, the number and positions of cameras are optimized, depending on the gaming environment, to enhance the ability to view any object on the deformable surface for a minimum of 3 views to improve triangulation accuracy.

[0246] In some embodiments, the real-time visual system comprising the projectors is designed to monitor the environment, compute the correct view geometry for each player, and project the relevant images corrected based on viewpoint into the immersive environment at a framerate of each camera with a latency of less than 0.05 second. In some embodiments, the real-time visual system displays images at about 120 Hz to eliminate flicker and motion artifacts from the players. In some embodiments, these high computational rates are achieved by combining efficient image processing algorithm and predictive motion modeling of the players and objects in the environment.

[0247] In some embodiments, the view presented to one or more game participants depends primarily on a viewpoint that is based on visual characteristics associated with one or more game participants. In some embodiments, the viewpoint is determined based at least in part on visual characteristics for multiple game participants (e.g., standing/sitting position, head position, eye game, and the like). In some embodiments, the overhead cameras are used to compute and maintain a continuous 3D reconstruction of the environment of the game space, which includes the game participants and their head positions. In some embodiments, the game participant's gaze direction is determined by face tracking and geometric inference from facial features, such as the location of the eyes in the camera imagery. In some embodiments, eye tracking technology is used to further refine the computation of gaze direction to determine the focus of each player.

[0248] In some embodiments, the tracking and computation of each game participant's viewpoint is continuous, determining the imagery to project into the environment is continuous, and adjusting the environment to the focus of each individual player is also continuous. In some embodiments, the combination of face tracking and geometric tracking allows the sports simulation system 100 to maintain the identity of each game participant at all times. In some embodiments, as objects leave a physical environment (e.g., driving a golf ball), the system 100 seamlessly hands-off from the tracking of the object in the physical environment to displaying a dynamic copy of the object in the projected virtual space.

[0249] In some embodiments, the VR sports system 100 provides each of the one or more users in the game space with 360-degree views. In some embodiments, the eye-tracking technology as disclosed above tracks each user's eye gaze and eye movements, and the 3D view projected to the place each user is looking at is customized to display a portion of the content in 3D to the particular user. In some embodiments, the projected 360-degree views are not 3D to all users at the same time, but the system projects a portion of the 360-degree views to a particular user such that the user perceives a 3D view of this portion. In some embodiments, the system uses visual cues to attract/direct different user's eye gaze to different parts of the scene concurrently to avoid overlap and non-3D views. In some embodiments, even if multiple users are looking at the same spot, they are immersed in their individual 3D experience and focus on their own 3D views, such that they are not distracted by the non-3D content.

[0250] Additional details regarding presentation of 2D graphics for 3D viewing are provided below in reference to FIGS. 19A-19D, 20A-4G, 21A-21C.

[0251] Attention is now directed to FIGS. 2A-2C, showing schematic diagrams of hitting maps used in a sport simulation system, in accordance with some embodiments. As shown in FIG. 2A, a hitting mat 116 is coupled with a playing surface 104. In some embodiments, the hitting mat 116 includes two tilting platforms (e.g., including tilting platforms 254 and 256) that have fairway-like surfaces from which a golf ball can be driven off of. In some embodiments, the fairway-like surface can be tailored to have thick grass, thin grass, sand, marsh, rock, shallow water, and/or wood chips in order to simulate various conditions found on real-life golf courses. In some embodiments, the hitting mat 116 can support a tee for driving, or a ball can be hit directly off a surface of the hitting mat 116. In some embodiments, each of the two tilting platforms of hitting mat 116 tilt against the playing surface 104 from either side to allow left-hand and right-hand golfers. In some embodiments, the tilting action can be initiated with a manual push-button switch, or automatically via software that is streamlined with the operations of a sport simulation system 100 (FIGS. 1A and 1B).

[0252] In some embodiments, the platforms also have provisions for surface-mounted or top-mounted commercial virtual reality sensor heads such that it can be used with a variety of existing virtual reality systems. When in use, the platforms are horizontal, resting on the green, and the golfer stands on them to strike the ball against a VR Screen (e.g., VR screen 250 FIG. 2A) in a sport simulation system (as shown in FIG. 2A). When not in use, the platforms are stowed vertically along the sides of the green (as shown in FIG. 2B the hitting mats 116-1 and 116-2 and stowed in a vertical position).

[0253] In some embodiments, systems including the hitting mats 106 have built-in safety features such as ultrasonic distance measuring sensors, proximity sensors, and limit switches that prevent the tilting platforms 254 and 256 (FIG. 2A) from tilting onto foreign objects or bystanders.

[0254] As shown in FIG. 2C, in some embodiments, the mechanisms used to deploy the hitting mats 116 include a base structure (e.g., base structure 270), a platform, a pivot shaft, and an actuator (e.g., actuator 260). In some embodiments, the base structure 270 is a solid rigid platform that attaches to the main structure of the playing surface or the surrounding concrete floor. Atop the base structure 270 is a pair of bearing pillow-blocks (e.g., bearing pillow blocks 262) through which the platform attaches via a precision shaft. The platform is allowed to pivot through the bearing pillow-blocks. The platform overhangs the bearing pillow blocks at the rear to allow for attachment to the actuator, to form a Class-1 lever. In some embodiments, the actuator 260 is a linear actuator 260, which extends and retracts to pivot the platform through a range of motion of approximately 90 degrees.

[0255] Attention is now directed to FIGS. 3A-3B and 3C, showing alternative embodiments, respectively, of a playing surface used in a sport simulation system, in accordance with some embodiments.

[0256] As shown in FIGS. 3A-3B, a playing surface 104 is a rigid playing surface 104 (i.e., a surface that is incapable of contouring) that is tilt-able using one or more actuators 305. In some embodiments rigid playing surface 104 includes a top surface (e.g., including a surface texture or material that simulates a putting green), floor panels (e.g., inch aluminum floor panels that are beneath the top surface, one or more module deck segments (e.g., modular deck segments 301, FIG. 3B, that may be welded aluminum C-channel with cross-bracing), an i-beam platform (e.g., i-beam platform 303 that is bolted together to provide floor strength and a stiff/rigid support unit). In some embodiments, each i-beam is bolted together to form the i-beam platform.

[0257] In some embodiments, rigid playing surface 104 further includes one or more actuators (e.g., actuators 305) that are located at specific positions relative to the i-beam platform 303 in order to tilt the playing surface 104. In embodiments in which the rigid playing surface 104 is included in a golf simulation system, by tilting the playing surface 104, many various breaking putts are possible. Moreover, by requiring only a limited number of actuators 305 to produce sufficient tilt, cost-savings is also achieved over embodiments in which numerous actuators are utilized (e.g., as discussed below in reference to FIG. 3C).

[0258] FIG. 3C illustrates an alternative embodiment to the playing surface 104 discussed above with respect to FIGS. 3A-3B, in which playing surface 104 is contour-able. In these alternative embodiments, the playing surface 104 includes four distinct layers. A first layer is a surface modifying layer 308 that includes a plurality of surface modification elements 106. A second layer is a sub-floor structure 306. A third layer is a compliance layer 304 that is used to ensure that the playing surface 104 feels like a real-life green while a participant is walking on the playing surface 104. A fourth layer is a top surface 302. In some embodiments, by modifying positions (e.g., a first lowered position and a second raised position for each of the elements 106) of the surface modification elements 106, the top surface 302 is contoured to match topographies of real-life greens (as discussed throughout). In some embodiments, the elements 106 are actuators that are configured to raise or lower to one or more predefined positions. In some embodiments, each actuator 106 is configured to position at a lowered position and a raised position. In other embodiments, each actuator 106 is configured to position at a lowered position and at a number of raised positions (e.g., 10 inches, 14 inches, 16 inches, etc.).

[0259] In some embodiments, a tracking/guidance system (e.g., tracking/guidance system 102) projects images/graphics onto the top surface 302. For example, the system 102 receives instructions from a system controller (e.g., system controller 114) to render representations of concentric targets 310 (could be of any predetermined shape and these are used to provide scores to participants during a game), best path putting guidance 312 (discussed in more detail below in reference to FIGS. 15A-15K), and pendulum putting guidance 313 (discussed in more detail below in reference to FIGS. 15A-15K).

[0260] In some embodiments, 2D images are generated for viewing in 3D and are presented on the top surface 302 (and/or over one or more other surfaces or objects within a physical gaming suite. In some embodiments, the 2D images are modified based on a current tilt and/or contour of the playing surface 104. Additional details regarding presentation of 2D graphics for 3D viewing are provided below in reference to FIGS. 19A-19D, 20A-20G, 21A-21C, and 22.

[0261] FIG. 4 is a functional block diagram of a ball delivery system (e.g., ball delivery system 108, FIG. 1A), in accordance with some embodiments. In some embodiments, the ball management system is important so players are not continuously fetching golf balls on a playing surface (e.g., playing surface 104). In some embodiments, the ball delivery system is integral to (i.e., coupled with) the playing surface 104 (or one or more components thereof, such as a deck assembly supporting the playing surface 104) and interfaces with the playing surface 104 at many locations (e.g., ball delivery interface 402-1, ball delivery interface 402-2, ball delivery interface 402-3, ball delivery interface 402-4, ball delivery interface 402-5, ball delivery interface 402-6, ball delivery interface 402-7, ball delivery interface 402-8, and ball delivery interface 402-9). In some embodiments, the camera system supports the ball management system by providing images during play to the control system. In some embodiments, the control system interfaces with the ball management system over the various modes of the game. In some embodiments, ball delivery mechanisms are integrated with the deck assembly and some embodiments have them located at 9 locations around the deck (e.g., ball delivery interfaces 402-1 to 402-9). In some embodiments, the ball delivery mechanisms deliver golf balls to the deck surface at predefined locations location. For example, the ball delivery system has the capability to place the balls on the green surface based upon a final shot (e.g., an approach shot) on a VR simulator. In some embodiments, the ball delivery mechanisms are configured to deliver multiple balls for multiple putts during a practice putting mode at a sport simulation system. During the practice putting mode, system controller 114 (FIG. 1A) collects camera images (e.g., as sent to the system controller 114 by the cameras 103, FIG. 1A) during practice putts and determines when to deliver a next ball to the player on the playing surface 104 at the same location as a previous practice putt.

[0262] In some embodiments, rotating wheels are configured to operate at various calibrated speeds and will deliver balls to the playing surface 104 over varying distances required for the balls to reach their intended locations. In some embodiments, the ball delivery mechanism is mounted to a rotating stage to enable various angles of trajectory. In some embodiments, the system controller 114 determines what mechanism is used based upon the gaming mode, and a required ball location on the green.

[0263] In some embodiments, a calibrated spring delivery device is utilized in which spring compression is adjusted based upon distance of delivery required for a respective ball. This type of mechanism also interfaces with a rotation stage to cover various angles required over the playing surface 104. In some embodiments, balls are delivered to the mechanism via a vacuum tubing system (e.g., ball delivery tubes 404). For example, during a 6-pack challenge game, the delivery system delivers different color balls to the playing surface 104 (e.g., the different color balls are held in a golf ball holding assembly 406).

[0264] In some embodiments, the holding assembly 406 is divided into three different holding areas for each ball color. In some embodiments, the system controller 114 releases a ball of a desired color (e.g., based on a particular round of a 6-pack challenge game) via a simple mechanism so that the vacuum tubing system can deliver it to the appropriate delivery mechanism. In some embodiments, the simple mechanism for releasing balls from holding assembly 406 and for selecting the proper vacuum tubing for the correct delivery mechanism channel is all done via shutter actuators at specific locations. In some embodiments, the actuations are controlled by system controller 114. In some embodiments, the system controller 114 controls ball management and delivery processes by using cameras to track current ball locations (e.g., cameras 103, FIG. 1A) and identifying a mode of a game being played.

[0265] In some embodiments, the playing surface 104 includes multiple hole locations (or targets) depending upon green configuration or mode of a game being played (e.g., target 415). In some embodiments, there are two aspects of ball retrieval from the playing surface 104. The first includes using a hole (e.g., target 415) to deliver balls back to the golf ball holding assembly 406. The second is using actuators (e.g., surface modification elements (FIG. 1A) to change the playing surface 104 so that balls are funneled into the hole or to a back edge of the playing surface 104 to channel golf balls through vacuum tubing to the holding assembly 406 (e.g., using one or more of the ball delivery tubes 404).

[0266] FIG. 5 is a side cross-sectional view of a dynamic target used in a sport simulation system, in accordance with some embodiments. As shown in FIG. 5, an exemplary hole assembly 502 is integral to a playing surface 104 (or a constituent component thereof, such as deck assembly supporting the playing surface 104). In some embodiments, 5 holes are provided on the deck assembly. In some embodiments, one of the holes is larger for use during some of the kid's games. In some embodiments, the hole assembly 502 is first made of a hole can assembly that provides structure and interfaces with other components of the hole assembly 502. In some embodiments, the hole can interfaces structurally to a subfloor structure (FIG. 3C) and holes are allowed to move up and down with a deck assembly that supports the playing surface 104. Depending upon the gaming mode of the sport simulation system, only have one hole will be used at a time and a system controller (e.g., system controller 114, FIG. 1C) identifies a proper hole location and controls system components for retrieving balls from the playing surface 104. In some embodiments, a linear actuator (e.g., actuator 504) is mounted in the hole can. The linear actuator controls the height of the hole.

[0267] In some embodiments, each hole is configured to have two or more height positions. A first position is higher and is used for 18 hole traditional golf game, e.g., with 18 with a Bullet Mode (this height position is shown with dotted lines in FIG. 5). This allows the players the ability to take their ball out of the hole after making their putt prior to moving off the green and onto the next fairway on a VR simulator. A second position is a lower linear actuator position. This position is to allow the ball retrieval aspect to be used via vacuum tubing to a golf ball holding assembly (discussed above in reference to FIG. 4). The second position is also used for Practice Makes Perfect mode, where multiple putts are made from the same spot. Based upon the mode of the system, the system controller 114 controls the actuator 504 and sets height of the hole. In some embodiments, modules of system controller 114 drive actuator 504 to a desired height based upon feedback from a position sensor that is integral to an assembly for the actuator 504. In some embodiments, a tilt stop (e.g., tilt-stop 506, FIG. 5) is provided that allows the hole to tip balls towards ball retrieval tubing access points (discussed above in reference to FIG. 4). This tilt is required to use gravity to help keep multiple balls heading towards the direction of ball retrieval tubing (e.g., ball retrieval tube 508).

[0268] In some embodiments, a bottom of the hole is constructed from the same material as the deck assembly. In some embodiments, the system has a subfloor, compliance layer, and green surface (discussed above in reference to FIG. 3C). Some embodiments require the material that comprises the hole to be the same as the playing surface 104, so that it is uniform on the playing surface 104 when that specific hole is not being used for a particular game. In some embodiments, a ball joint is required at the top of the actuator 504 that interfaces with the subfloor to enable the tipping action required to deliver balls. In some embodiments, a ball trigger sensor that is mounted to the hole assembly 502. In some embodiments, the ball trigger sensor is read by the system controller 114 and it sends a signal every time a ball passes by, a player makes a putt, or the playing surface 104 is funneling balls into the hole. The sensor enables the system controller 114 to turn on the vacuum capability, so that balls reliably get returned to a holding assembly (e.g., holding assembly 406, FIG. 4) and do not get trapped in the tubing. In an 18-hole traditional golf mode, the sensor is used to confirm a made putt and to update the scoreboard and trigger audio feedback. In some embodiments, the ball retrieval tube 508 is used to deliver balls from the holes to the golf ball holding assembly (5). The vacuum system ensures that a golf ball does not get stuck in the system so a game does not get halted or delayed.

[0269] In some embodiments, one way to retrieve golf balls off of playing surface 104 is to use surface modification elements (e.g., actuators 106, FIG. 1A) to funnel them into the hole being used. In some embodiments, the system is designed to use the hole to retrieve the balls and to have a channel at one end of the deck assembly at the edge to be used to deliver the balls back to the holding assembly. In some embodiments, the end channel is at the end of the deck that will typically have the holes being used for the various modes the most often. The following example is in the practice makes perfect mode where a player will have 5, 10, or 15 putts from the same location on the green. With the use of the cameras 103 (FIG. 1A), the system controller 114 determines when a player is done putting the allocated number of balls. The cameras 103 also update scores as putting occurs and will confirm putts that were made with the hole sensor. When all putts are completed, the system controller 114 utilizes surface modification elements 106 of playing surface 104 so that balls are funneled into the hole being used. In some embodiments, one or more surface modification elements 106 (e.g., one or more linear actuators) are configured to lower the playing surface 104 to provide this funnel.

[0270] FIGS. 6A and 6B are flowcharts showing assembly, testing, and calibration of a sport simulation system, in accordance with some embodiments. In some embodiments, the system includes three primary sub-systems, including the mechanical structure and deck assembly (as shown and described in relation to FIGS. 1B, 2A-3B above), the computer, electronics, virtual-reality, and player interface systems (as shown and described above in relation to FIGS. 1A-1C above, and 6B onward, below), and the guidance and camera systems (as shown and described in relation to FIGS. 1A-1C above, and 6B onward, below). The setup method 600 initially includes manufacturing, assembling, and/or setting-up and connecting the mechanical structure and deck assembly at 602, the computer, electronics, virtual-reality, and player interface systems at 604, and the and the guidance and camera systems at 606. The mechanical structure and deck assembly, and the computer, electronics, virtual-reality, and player interface system sub-systems are then tested to confirm that they are operating and interacting correctly at 608. The guidance and camera systems are then tested to confirm that they are operating and interacting correctly at 610. The various different operational modes and games, as well as all other operations are then tested at 612. In other embodiments, the testing is conducted in a different order.

[0271] In some embodiments, each time that the system is initiated, such as once a day or before each paying session or game, the system is calibrated at 614. In some embodiments, calibration includes calibration of the guidance system and/or the surface topography of the playing surface. In some embodiments, surface topography calibration is achieved by optically scanning the playing surface using a laser or infrared scanner and if the playing surface is not level, sending commands to the actuators to flatten the playing surface or otherwise position it in an initial state. In some embodiments, guidance system calibration is achieved by illuminating the playing surface using a calibration template (e.g., a grid or one or more straight lines) and then detecting (e.g., using the video camera system and appropriate identification software) whether the illuminated calibration template is properly aligned and/or displayed on the surface.

[0272] In some embodiments, the system then goes through a burn-in test at 616 to ensure that all bugs in the operation are identified and addressed. In some embodiments, the burn-in test comprises running the software through all possible operations, in all possible sequences, repeating such testing multiple times, and/or logging any faults or errors detected.

[0273] FIG. 6B is a more detailed flow chart of a sport simulation system setup method 640. Initially, the mechanical structure and deck is provided by: manufacturing or providing the frame assembly (described above) at 641; integrating one or more actuators (described above) to the frame assembly at 642; connecting cabling (in the case of electrically powered actuators) or pipes/tubing (in the case of hydraulic or pneumatic actuators) at 643; the ball delivery system (e.g., as described in FIGS. 4 and 5) is connected to the frame assembly at 644; an optional sub-floor (see FIG. 1B) assembly is connected above of the actuators at 645; the optional compliance surface or layer (see FIG. 1B) is then provided above the sub-floor (if one exists) at 646/647; the playing surface or green (see FIG. 1B) (if one exists) is then connected above the compliance surface or layer (if one exists) at 648; the hole assembly is finalized if necessary at 649 (e.g., by ensuring that the hole cup is aligned with the top of the playing surface); the ball delivery system is finalized if necessary at 650; and after which the deck assembly is ready at 662.

[0274] The computer, electronics, virtual reality, and player interface systems are provided by: manufacturing or providing the computer assembly (described above) at 651; loading and testing the requisite software at 652; integrating, connecting, and/or testing or verifying the actuator electronics (e.g., any required relays, transformers; valves etc.) at 653; integrating, connecting, and/or testing or verifying the camera interface at 654; integrating, connecting, and/or testing or verifying the guidance and visual systems at 655; integrating, connecting, and/or testing or verifying the guidance audio system at 656; integrating, connecting, and/or testing or verifying the virtual reality interface at 657; integrating, connecting, and/or testing or verifying the payer interface at 658; and after which the computer, electronics, virtual reality, and player interface systems are ready at 659.

[0275] The computer, electronics, virtual reality, and player interface systems are provided by: manufacturing or providing the computer assembly (described above) at 651; loading and testing the requisite software at 652; integrating, connecting, and/or testing or verifying the actuator electronics (e.g., any required relays, transformers; valves etc.) at 653; integrating, connecting, and/or testing or verifying the camera interface at 654; integrating, connecting, and/or testing or verifying the guidance and visual systems at 655; integrating, connecting, and/or testing or verifying the guidance audio system at 656; integrating, connecting, and/or testing or verifying the virtual reality interface at 657; integrating, connecting, and/or testing or verifying the payer interface at 658; and after which the computer, electronics, virtual reality, and player interface systems are ready at 659.

[0276] The guidance system integration is provided by: integrating, connecting, and/or testing or verifying the camera assembly with the other systems, like the deck assembly, at 650; integrating, connecting, and/or testing or verifying the light and/or laser system at 661; integrating, connecting, and/or testing or verifying the audio system at 662; integrating, connecting, and/or testing or verifying the wireless communications systems (e.g., wifi routers, RFID chips in balls and accompanying sensors, etc.) at 663; and after which the guidance system is ready at 664.

[0277] The system is then tested to ensure that it is functional and all modes of operation are verified by: testing and/or verifying that the computer system works with and can control the deck assembly at 665; testing and/or verifying that all cabling is connected at 666; testing and/or verifying that all actuators are functional at 667; testing and/or verifying that the hole actuator functions at 668; testing and/or verifying that the ball delivery system functions at 669; aligning the camera system at 670; aligning, testing and/or verifying the camera system at 671; testing and/or verifying the lighting and/or laser systems at 672; testing and/or verifying audio system at 673; testing and/or verifying the feedback (or loop) system for the variable topography surface at 674; testing, verifying, and/or eliminating the feedback (or loop) system for the audio system at 675; testing and/or verifying all operational modes of the system at 676; and after which the functions of the system are ready at 677.

[0278] Finally, the system's performance is determined, calibration is performed, and burn-in testing is performed by: calibrating the algorithm for the visual guidance system at 678; calibrating the algorithm for the audio guidance system at 679; performing exhaustive mode testing of the system at 680; performing burin-in testing at 681; and after which the entire system is ready for operation at 682.

[0279] It should be appreciated that any of the above steps described in relation to FIG. 6B may be eliminated, or replaced by other steps. Also, any of these steps may be performed in any appropriate order.

[0280] In some embodiments, a sport simulation system (e.g., sport simulation system 100) recreates putting greens from actual golf courses. Other than the putting part of the game, a traditional virtual reality simulator golf game is provided. Unlike, traditional virtual reality simulator golf games, identical (or similar) topography is provided for the greens. In some embodiments, traditional scoring is used: to par or match play against others (in-person or online). In other embodiments, skills challenges/big break situational games can be played on the system 100. In other embodiments, only par 3s or approach shots from a variety of different/famous holes may be played. In yet other embodiments, duck hunt/darts using punch shots may be played. In other embodiments, long and straight drive contests; accuracy with iron contests may be played, or even putting-only legends games. Some embodiments, allow for the recreation of the top famous putts of all time (Jack at 17 Augusta, Tiger at 18 Torrey, Leonard at Brookline). In yet other embodiments, six-ball challenge, with skeeball scoring may be played. Some embodiments allow for a kids-friendly mode where the topography changes to funnel the ball towards the hole. Other embodiments, allow for simultaneous candy crush and miniature golf to be played. Even other embodiments allow for 15-inch or other predetermined sizes of holes for easier scoring.

[0281] In some embodiments, these games provide positive feedback loops to drive engagement; allow players to level up their games, i.e., tracks long-term improvement; provide modern-handicapping (e.g., using artificial intelligence to adjust game play to each user's skill sets, allowing players of varying skill levels to play together competitively and at same time/rate).

[0282] In addition to game play, other modes of operation may be played, e.g., 18 hole multiple players70-80 different courses; regular golf with putting assistance; regular golf without putting assistance; non-golfer challenge game (using concentric circles); 9 hole multiple players70-80 different courses; regular golf with putting assistance; regular golf without putting assistance; non-golfer challenge game (using concentric circles); 9 hole practice puttingmultiple tries from one spot with ball return.

[0283] One exemplary game is practice makes perfect and can be played using the systems and methods described herein.

[0284] FIG. 7 is a flowchart of a method of managing a practice makes perfect game at a sport simulation system, in accordance with some embodiments. Practice Makes Perfect is a game that allows players to (i) get accustomed to the system, and (ii) continuously increase their skill level at putting in general using the system's guidance and teaching abilities. The new combination of technologies include the guidance system, the undulating green, the scoring, game-flow, and sound enhancements. The players skill at putting will increase due to the coaching the system will do, which includes displaying and reviewing gradients of the green, planning the direction of the putt to maximize the likelihood that the putt is made, guidance on how far back to swing the putter for optimal velocity towards the hole, and continuous feedback of how the player is doing, including instant replay and commentaries on what the player did right or wrong during each putt. Players may use Practice Makes Perfect on their own, but ideally will play with a group to enhance team spirit and positive reinforcement.

[0285] In some embodiments, a method 700 is performed by an electronic device (e.g., system controller 114, FIG. 1A) and/or one or more components of the electronic device (e.g., an I/O subsystem, an operating system, etc.). In some embodiments, the method 700 is governed by instructions that are stored in a non-transitory computer-readable storage medium and that are executed by one or more processors of a device, such as the one or more processors 202-1 of a management module 114-1 (FIG. 1C). For ease of explanation, the following description of method 700 is performed by the system controller 114 (also referred to simply as the device). In some embodiments, with reference to FIG. 1C, the operations of method 700 are performed by or use, at least in part, a surface modification module (e.g., surface modification module 222), a ball color detection module (e.g., ball color detection module 224), a ball path determining module (e.g., ball path determining module 226), a graphics rendering module (e.g., graphics rendering module 228), a game selection module (e.g., game selection module 230), a ball delivery module (e.g., ball delivery module 232), a ball retrieval module (e.g., ball retrieval module 234), user-specific putting data (e.g., user-specific putting data 218), and a leaderboard updating module (e.g., leaderboard updating module 220). Some operations in method 700 are, optionally, combined and/or the order of some operations is, optionally, changed.

[0286] In some embodiments, a practice makes perfect game play method 700 initiates when the players are on the green at 702. In some embodiments, the game can be played with anywhere from 1-10 players, while in preferred embodiments, the game can be played with 8 or fewer players. In some embodiments, once the players walk onto the dynamic green, they can touch a player interface control screen to initiate game play. In some embodiments, the first part of the game is designed to allow players new to the system to get comfortable with the variables of the game, including the physical undulating green, the laser guidance system, and the unique synthesis of the mechanical, electrical, and audio systems as applied to golf. In some embodiments, the players are first presented (visually and/or audibly) with an introduction to a practice mode at 704. If a player is new to the system and/or the game, i.e., has never played the game before, (706Yes), the game and the system is described to the player (again, visually and/or audibly) at 708. If a player is not new to the system and/or the game, i.e., has played the game before, (706No), or after the game and the system is described to the player at 708, the player chooses their skill level at 710. In some embodiments, if the players are all seasoned players, they may choose to skip this introduction.

[0287] In some embodiments, the player selects their skill level at 712, while in other embodiments, the player's skill level is stored on the system from prior games and is recalled at 714. For example, the player interface control screen allows the player to select their skill level, either manually or the game system can determine the skill level based on past performance by each player, according to the system's elaborate and extensive database of individual player's history and statistics. In some embodiment, the game is progressive, which means that as the player's abilities improve, both during a game, and among multiple sessions, they are challenged with increasingly difficult putts so that their level of play improves overall.

[0288] In some embodiments, the green will set itself up for the game, according to the current level (see outline below for a detailed description of each level). In other words, the system sets up the green for the first level in the game at 716. In some embodiments, the green is also setup for a particular player's skill level. In some embodiments, this includes changing the topography of the playing surface, selecting the difficulty of the challenges in the game, etc.

[0289] In some embodiments, the guidance system then projects onto the playing surface or green the gradients or slopes to visually depict the topography of the green at 718. In some embodiments, this is based on the particular skill level of the particular player. In some embodiments, the guidance system is enabled with an emphasis on a pendulum guidance mechanism that displays on the green with a laser or projector gradients and past ball tracks. For example, the guidance system displays on the green using a laser or projector, the length and speed of the putting motion via a pendulum animation. In some embodiments, it also displays the best fit path to the hole and a target point where to hit the ball.

[0290] In some embodiments, the ball delivery system delivers to the player one or more balls 720. In other embodiments, the player or a caddy selects and places the balls manually on the green. In some embodiments, the player receives audible instructions of where and/or how to place and address the ball with their golf club (typically a putter). At 720. In some embodiments, the guidance system projects one or more visual guides onto the playing surface at 724. For example, displaying a moving pendulum to show the player the proposed putting stroke and speed; prior puts ball tracks (i.e., the lines or loci that prior balls took); a best fit line (i.e., an ideal line that the ball should follow to be sunk in the hole); a target of one or more concentric rings around the hole; a marker showing a line where the player should swing back their club to and another marker showing a line where they should follow the club through to; etc. In some embodiments, the audio system also coaches the player on how to sink the put. In some embodiments, the player does not receive any audio assistance if they have a high skill level.

[0291] Once the player has made their put towards the hole, a sensor registers whether the player's ball was received in the hole. In some embodiments, another sensor (e.g., a ceiling mounted camera and associated hardware and software) determines whether the ball stopped within one of the concentric circles surrounding the hole. In some embodiments, the highest point is awarded if the ball was sunk in the hole, the next highest points if it stops close to the hole; and lower points the further away from the hole that the ball stopped. The score board then (or constantly, i.e., in real time) displays the statistics including the player's score and skill level on a score- or leader-board at 728. In some embodiments, after each additional balls (e.g., 5 balls) are delivered up to the maximum for each skill level. For example for Level 1: Training mode to learn about pendulum putting; the green surface is flat and all putts are straight to the hole; a short putt is provided with 5 balls; a medium putt, 10 balls; and a long putt, 15 balls.

[0292] In some embodiments, the camera/tracking system tracks each shot; shows a replay, if desired; records statistics for each player; and stores them across multiple sessions. In some embodiments, the player then plays his next shot and repeats steps 720-728 until his round is complete (730Yes) and the player is done at 732.

[0293] If all of the players are not done playing (734No), the next player takes their turn at 736 and steps 718-734 are repeated until all players in that group have completed their puts (734Yes).

[0294] For example, once the first player has completed all the putts for Level 1, play will proceed to the next player until all players have completed Level 1; after all players have completed Level 1, the game will proceed to the next level. In some embodiments, a Klaxxon (or red alert siren) will sound for all players to exit the green at the end of a level, and once they have left the green, the green will change shape for the next level. In other embodiments, the green changes while the users are standing on it. Continuing with the example, the emphasis of Level 2 is to learn about the Target Point, or the best place for the putter to strike the ball. The skill level is increased by having the level start with a medium-length shot, progressing to long shots, and finally a mild gradient is introduced to the green. In other words, Level 2: Training to learn about Target Point; a straight shot, medium length, 5 balls; a straight shot, long length, 10 balls; end of the green is raised or lowered to create a gradient, 15 balls. Continuing with the example, once Level 2 is complete, the play proceeds to Level 3. In Level 3 the green is banked and the emphasis is on the Best Fit Line, which shows the path the ball should travel to the hole. In other words, for Level 3: Banked Surface, Training to learn how to use the Best Fit Line; short putt, 5 balls; medium putt, 10 balls; and long putt, 15 balls. In this example, Level 4 is the final level and concentrates on using all the skills learned on more advanced shots, which include multiple peaks and valleys on the green and putts of varying lengths. Here, Level 4: Peaks and Valleys, utilizing all skills learned; 5 balls; followed by 10 balls; followed by 15 balls.

[0295] If there are further levels in the game to play (738No), then the green is set for that next level in the game at 716 (and in some embodiments the player's skill level), and steps 716-738 are completed until all levels have been completed by all players (738Yes). Once the game has been completed at 740, the score- or leader-board displays the final scores (and in some embodiments other statistics, like time spent per put, etc.) at 742. In some embodiments, the score- or leader-board (with accompanying audio) displays the winner, e.g., displaying the winner's avatar, and everyone's final score. In some embodiments, the score- or leader-board displays the total experience points (and/or incentives like, for example, an incentive to play another round to get bonus points) that the player has accumulated while playing on the system at 744. In some embodiments, the experience score or points are used to handicap the user for future games, i.e., to set the user's skill level. In some embodiments, this experience score is similar to a handicap in regular golf. In some embodiments, the experience score also takes into account scores from games played on other remote similar systems (e.g., at other systems located in other cities etc.). The practice make perfect game is then complete at 746.

[0296] It should be understood that the particular order in which the operations in FIG. 7 have been described is merely exemplary and is not intended to indicate that the described order is the only order in which the operations could be performed. One of ordinary skill in the art would recognize various ways to reorder the operations described herein.

[0297] Attention is now directed to FIG. 8, illustrating another game available at the sport simulation system 100. More specifically, FIG. 8 is a flowchart of a method of managing a round of golf at a sport simulation system, in accordance with some embodiments.

[0298] In some embodiments, an 18-with-a-bullet game play method 800 is performed by an electronic device (e.g., system controller 114, FIG. 1A) and/or one or more components of the electronic device (e.g., an I/O subsystem, an operating system, etc.). In some embodiments, the method 800 is governed by instructions that are stored in a non-transitory computer-readable storage medium and that are executed by one or more processors of a device, such as the one or more processors 202-1 of a management module 114-1 (FIG. 1C). For ease of explanation, the following description of method 800 is performed by the system controller 114 (also referred to simply as the device). In some embodiments, with reference to FIG. 1C, the operations of method 800 are performed by or use, at least in part, a surface modification module (e.g., surface modification module 222), a ball color detection module (e.g., ball color detection module 224), a ball path determining module (e.g., ball path determining module 226), a graphics rendering module (e.g., graphics rendering module 228), a game selection module (e.g., game selection module 230), a ball delivery module (e.g., ball delivery module 232), a ball retrieval module (e.g., ball retrieval module 234), user-specific putting data (e.g., user-specific putting data 218), and a leaderboard updating module (e.g., leaderboard updating module 220). Some operations in method 800 are, optionally, combined and/or the order of some operations is, optionally, changed.

[0299] In some embodiments, the method 800 beings when a set of players from 1-8 walk to game bay (e.g., to a sport simulation system 100, FIG. 1A) and touch a player interface control screen (802, 804). The player interface control screen allows players to select what type of game they would like to play, order beverages, evaluate their player statistics, or view the golf merchandise and get prizes (806, see exemplary user interfaces provided in FIGS. 13A-13C). In this example, input is received selecting a Traditional Virtual Reality Simulator golf game on an 18 hole course or 18 with a bullet (808C). Selectable options are also provided for practice makes perfect (808A), kid's corner (808B), and 6-pack challenge (808D). The players are also able to select a course to be played of countless available courses (810) and if they would like putting guidance system enabled or disabled (812). Each player has their golf bag conveniently located in their respective bay and will start to drive at hole 1 on a Virtual Reality (VR) simulator (814).

[0300] Each player drives on the VR simulator on fairway number 1. While the players are driving and getting to their green approach shot, a playing surface 104 (FIG. 1A) has been controlled using surface actuators (e.g., surface modification elements 106, FIG. 1A) that are driven based on instructions from a system controller 114 (FIG. 1A) (816-818). In some embodiments, this step does not occur until the system determines there are no players on the green surface (820). Based upon the green surface for hole 1, a control computer (e.g., system controller 114) drives a correct hole actuator for that green and retracts a prior hole (822). In some embodiments, a database associated with the system controller 114 houses green configurations for every available green. Based upon a pre-calibrated position table for the system, the system controller 114 drives surface actuators to correct positions for the green surface using linear position sensors as feedback to confirm accurate location of each actuator (824). Once a player hits an approach shot on the green for the VR simulator, a ball delivery system drops a ball on the actual green ready for that player to play. The ball delivery system will deliver the ball where it landed from the approach shot on the VR simulator (as discussed above in reference to FIGS. 4 and 5. The ball is delivered from a possibility of a number of locations around the putting green deck. Based upon where the ball landed on the VR screen the control computer will select the ball delivery number, adjust the speed of rotation, allow a ball to proceed through the mechanism, and then confirm delivery and location with the camera system (830). The approach shot and ball delivery process will occur for all the players, until they all are on the green (834).

[0301] The players can now proceed to putt out on the green just as you do on a real golf course green. All the balls for each player are located on the green and the system controller 114 using cameras 103 to determine exactly what ball is associated with what player. The green can support all players on the putting green surface at once, e.g., up to 6, 7, 8, 9, 10, or other number of adults. The system knows the longest ball (838), and in training mode, guidance information is controlled to enable putting training features. The system shines on the first putter based upon the distance to the hole (836). The system controller 114 knows exact topography of the playing surface, ball location, and location of a hole on the playing surface 104 (or other target besides a hole). The target algorithms take this information into account and then control the guidance system. The visual guidance system is a combination of projected images and laser lights. The first projection on the green is a grid that shows the contour or peaks and valleys of the green to help the player read the green to familiarize themselves with why the ball would go certain ways due to surface topography (840). Based upon that grid a guidance system then puts a best fit line on the playing surface 104 and shows a player where the ball would go under an accurate putt given the surface topography (844). The next image from the guidance system is to put an X on the surface to identify the target point the player should hit the ball to, taking into account the surface topography, that if hit, would follow the best fit line (846). The final image from the guidance system is to put the pendulum start and finish by the player to help them determine how hard to hit the putt to achieve the putting distance (848). The system will keep track of the results of the putt via ball tracking and keep results as to the success of the putt and provide feedback as to what might of went wrong under a missed putt. The system will calibrate per player the pendulum distance to enhance their ability to make putts based upon the player's typical force. The player will then hit the putt (850) and this will engage the audio system for player input and feedback and the ball tracking system to monitor the putt as it proceeds to the hole (854). Once the player hits the ball, the audio system will cheer and coach the approaching shot (852). All audio and visual putting guidance interaction is via closed loop image processing using the camera system above the green, the visual guidance system, and the audio guidance and feedback system. The system will provide putting tips via the camera system tracking through the entire putt. The control computer will determine on that putt if the player should putt out (856) or put a marker (858) so the next player can putt. The system will determine if that was the last player to putt or not or to move onto the next longest putt (860, 864). This is determined using the camera system and the control computer as the players proceed through the putting process.

[0302] When all of the players have putted out on that current green they will proceed off the green surface (862) and back to the VR simulator for the next hole drive and approach shots (866, 868). The system will keep track of each player's drives, approach, and putts and maintain keeping the score and displaying that at the bay display monitor. The VR simulator will transition to the next hole to be prepared for the players to start driving again.

[0303] It should be understood that the particular order in which the operations in FIG. 8 have been described is merely exemplary and is not intended to indicate that the described order is the only order in which the operations could be performed. One of ordinary skill in the art would recognize various ways to reorder the operations described herein.

[0304] Attention is now directed to FIG. 9A, illustrating another game available at the sport simulation system 100. More specifically, FIG. 9A is a flowchart of a method of managing a 6-pack putting challenge game at a sport simulation system, in accordance with some embodiments.

[0305] In some embodiments, a 6-pack putting challenge game play method 900 is performed by an electronic device (e.g., system controller 114, FIG. 1A) and/or one or more components of the electronic device (e.g., an I/O subsystem, an operating system, etc.). In some embodiments, the method 900 is governed by instructions that are stored in a non-transitory computer-readable storage medium and that are executed by one or more processors of a device, such as the one or more processors 202-1 of a management module 114-1 (FIG. 1C). For ease of explanation, the following description of method 900 is performed by the system controller 114 (also referred to simply as the device). In some embodiments, with reference to FIG. 1C, the operations of method 900 are performed by or use, at least in part, a surface modification module (e.g., surface modification module 222), a ball color detection module (e.g., ball color detection module 224), a ball path determining module (e.g., ball path determining module 226), a graphics rendering module (e.g., graphics rendering module 228), a game selection module (e.g., game selection module 230), a ball delivery module (e.g., ball delivery module 232), a ball retrieval module (e.g., ball retrieval module 234), user-specific putting data (e.g., user-specific putting data 218), and a leaderboard updating module (e.g., leaderboard updating module 220). Some operations in method 900 are, optionally, combined and/or the order of some operations is, optionally, changed.

[0306] The method 900 begins when a set of players from 1-8 walk onto a playing surface (e.g., playing surface 104, FIG. 1A) at 902 and touch a player interface control screen (e.g., one of the user interfaces shown in FIGS. 13A-13C) at 904. In some embodiments, the player interface control screen allows the player to select what type of game they would like to play (904), order beverages, evaluate their player statistics, view the golf merchandise and get prizes (see FIGS. 13A-13C for exemplary user interfaces), and select whether guidance should be provided or not (904). 6-pack challenge is a fast-paced golf putting game (e.g., takes 2 players about 10 minutes to complete) with 3 rounds of 6 balls each, and can be selected using the provided user interfaces. In response to a selection of 6-pack challenge, the players are then able to select a difficulty level (Easy, Intermediate, Mission Impossible), a golf course, and if they would like putting guidance to be enabled or disabled (904). Each player has their putter and will take turns putting 6 balls at a time per round, with 3 rounds total. Depending on the round, the green is either flat, raised, or banked (using either a tilting surface, FIG. 3A-3B, or a deformable surface, FIG. 3C) at 906.

[0307] In some embodiments, scoring for each round increases, thereby allowing for an upset (i.e., underdog reversal) as late as the final putt. The scoring is 1, 2, 3 depending on the round. The projectors 105 (FIG. 1A) are instructed (by the system controller 114) to project a circular target around the hole and trajectories for each shot (908), if a player's ball winds up in the target area, they will score one point. If the ball goes in the hole, they will score 2 points. If they don't make the Sally Line (Never up, Never in), 1 point is subtracted from their score. The scoring is progressive based on the difficulty of the shot and the difficulty of the playing surface 104 (which changes with each round). Scores for each respective shot are determined at 910. An audio system (e.g., audio system 112, FIG. 1A) moves gameplay along and also conveys positive and negative feedback based on the success of the shots (912). The player may stand on the playing surface 104 and a ball delivery system (e.g., ball delivery system 108, FIG. 1A) delivers a first set of balls for a first part of round 1. The player will now begin to putt (952). The game flow is as follows for Round 1 with flat surface, 1 scoring: [0308] a) First 3 balls (e.g., balls having a white color) are delivered and the shot is a straight line to the hole (e.g., playing surface 104 is configured to be flat) at 910. Depending on guidance settings, a white line is drawn on the green from the ball to the hole (e.g., a best fit line, as discussed in more detail in reference to FIGS. 15A-15K). In some embodiments, cameras 103 (FIG. 1A) track all shots (914) and a leaderboard is updated to display points for each participant (916). Points are provided as follows: [0309] In hole: 2 pts [0310] In target: 1 pt [0311] Never-up Never in: 1 pt [0312] b) Next 2 balls (orange) are delivered, the shot is along a more challenging path to the hole, and shots are tracked using cameras 103 (918-922). Depending on the guidance settings, an orange line is drawn on the green from the ball to the hole for the optimal trajectory (e.g., a best fit line having a second color distinct from the first provided white color). Points are provided as follows (920): [0313] In hole: 4 pts [0314] In target: 2 pts [0315] Never-up Never in: 2 pts [0316] c) Next 1 ball (yellow) is delivered (926) and a shot is along an even more challenging path to the hole (928, 930, 932). Depending on the guidance settings, a yellow line is drawn on the green from the ball to the hole for the optimal trajectory. Point values are provided as follows [0317] In hole: 6 pts [0318] In target: 3 pts [0319] Never-up Never in: 3 pts

[0320] The camera/tracking system will track each shot and compute the scoring (932, 924, 914). The score is displayed/updated on a large scoreboard after each shot (8). Once the first player has completed all 6 putts from round one, play will proceed to the next player until all players have completed round 1. After all players have completed round 1, the game will proceed to the next round (938-940). A Klaxxon (or red alert) will sound for all players to exit the green (940). Once they have, the green will change shape for the next round (906).

[0321] The game play is as follows for Round 2 with a Raised Surface, 2 scoring: [0322] a) First 3 balls (white) are delivered and the shot is a straight line to the hole (910). Depending on the guidance settings, a white line is drawn on the green from the ball to the hole. [0323] In hole: 4 pts [0324] In target: 2 pt [0325] Never-up Never in: 2 pt [0326] b) Next 2 balls (orange) are delivered and the shot is along a more challenging path to the hole (918-922). Depending on the guidance settings, an orange line is drawn on the green from the ball to the hole for the optimal trajectory. [0327] In hole: 8 pts [0328] In target: 4 pts [0329] Never-up Never in: 4 pts [0330] c) Next 1 ball (yellow) is delivered (926) and the shot is along an even more challenging path to the hole (928, 930, 932). Depending on the guidance settings, a yellow line is drawn on the green from the ball to the hole for the optimal trajectory. [0331] In hole: 12 pts [0332] In target: 6 pts [0333] Never-up Never in: 6 pts

[0334] Once Round 2 is complete, the play proceeds to Round 3. Round 3 has a slight variation at the end as described in the following outline, to keep the game exciting right to the last putt. Round 3 uses a Banked Surface, 3 score, and proceeds as follows: [0335] a) One white ball is delivered and the participant putts one white ball (952). Depending on the guidance settings, a white line is drawn on the green from the ball to the hole (914). [0336] In hole: 6 pts [0337] In target: 3 pt [0338] Never-up Never in: 3 pt [0339] b) Next 2 balls (orange) are delivered (920) and the shot is along a more challenging path to the hole (918-922). Depending on the guidance settings, an orange line is drawn on the green from the ball to the hole for the optimal trajectory. [0340] In hole: 12 pts [0341] In target: 6 pts [0342] Never-up Never in: 6 pts [0343] c) Next 3 yellow balls are delivered (926) for players choice (950). The more challenging shot the player chooses, the more he/she is rewarded for a successful putt. The choices are the same white, orange, and yellow shots and are scored accordingly: [0344] In hole: 6, 12 or 18 pts [0345] In target: 3, 6, or 9 pts [0346] Never-up Never in: 3, 6, or 9 pts

[0347] After the 3rd round, the game is over (934Yes, 942). The leaderboard and audio system announce the winner in a flashy manner (944), displaying the winner's avatar and everyone's final score. Experience points and other incentives are displayed (946) and game is complete (948).

[0348] It should be understood that the particular order in which the operations in FIG. 9A have been described is merely exemplary and is not intended to indicate that the described order is the only order in which the operations could be performed. One of ordinary skill in the art would recognize various ways to reorder the operations described herein.

[0349] FIG. 9B is a flowchart of a method 950 of playing 6-Pack Challenge game at a sport simulation system, in accordance with some embodiments.

[0350] In some embodiments, the method 950 is performed by an electronic device (e.g., system controller 114, FIG. 1A) and/or one or more components of the electronic device (e.g., an I/O subsystem, an operating system, etc.). In some embodiments, the method 950 is governed by instructions that are stored in a non-transitory computer-readable storage medium and that are executed by one or more processors of a device, such as the one or more processors 202-1 of a management module 114-1 (FIG. 1C). For ease of explanation, the following describes method 950 as performed by the system controller 114 (also referred to simply as the device). In some embodiments, with reference to FIG. 1C, the operations of method 950 are performed by or use, at least in part, a surface modification module (e.g., surface modification module 222), a ball color detection module (e.g., ball color detection module 224), a ball path determining module (e.g., ball path determining module 226), a graphics rendering module (e.g., graphics rendering module 228), a game selection module (e.g., game selection module 230), a ball delivery module (e.g., ball delivery module 232), a ball retrieval module (e.g., ball retrieval module 234), user-specific putting data (e.g., user-specific putting data 218), and a leaderboard updating module (e.g., leaderboard updating module 220). Some operations in method 950 are, optionally, combined and/or the order of some operations is, optionally, changed.

[0351] In some embodiments, a method 950 includes: delivering (902), to a putting green surface, a first plurality of golf balls having a first color. In some embodiments each golf ball is delivered to a predetermined position on the putting green surface (using ball delivery system 108, as explained above in reference to FIGS. 4 and 5). For example, when a game of 6-pack challenge is played at a sport simulation system, the ball delivery system 108 is instructed, via ball delivery interface 210, to deliver the first plurality of golf balls to predetermined positions that are determined based on graphics projected on the putting green surface (e.g., circular graphics projected on the putting green surface by one or more projectors 105). In some embodiments, each golf ball in the first plurality of golf balls have a white color.

[0352] The method also includes: configuring (954) the putting green surface to have a first topography. In some embodiments, the first topography is a substantially flat topography, such that the putting green surface is initially configured in a substantially flat configuration (e.g., without any tilt or contouring of the surface). Topography of the putting green surface is configured by sending instructions (e.g., by system controller 114 using playing surface interface 212 and surface modification module 224, FIG. 1C) to one or more surface modification elements 106 (FIG. 1A).

[0353] For each respective golf ball in the first plurality of golf balls (956), the method includes at least five operations (e.g., operations 958-968).

[0354] The first operation includes determining (958) that a first game participant is addressing a respective golf ball of the first plurality of golf balls. In some embodiments, this determination is made by using one or more cameras 103 (FIG. 1A) to determine that a user has placed their feet within a feet placement graphic projected on the putting green surface. In some embodiments, this determination is made by using the one or more cameras 103 to determine that a user is within a predetermined distance of a respective golf ball and/or determine that the user has placed their putter behind the golf ball.

[0355] The second operation includes determining (960), based on the first topography of the putting green surface, a best path from the respective golf ball to a target on the putting green surface. In some embodiments, ball path determining module 226 (FIG. 1C) in conjunction with tracking system interface 204 are used to perform this operation. As discussed above, in some embodiments, the target is a hole extending through the putting green surface and the hole may be of varying sizes depending on skill level of the first game participant. In other embodiments, the target is a graphic project on the putting green surface (such as a gopher moving around the surface, a bulls-eye, and the like). Determining the best path is discussed in detail below in reference to FIGS. 15A-15K.

[0356] The third operation includes sending (962), to a projecting device that is distinct from the system controller 114 (the device performing the method 950), instructions to (a) render a representation of the best path on the putting green surface and (b) render a substantially circular graphic around the target, the substantially circular graphic having a size that is determined based on a skill level associated with the respective game participant. In some embodiments, instead of a substantially circular graphic a graphic of any suitable shape is rendered (e.g., a graphic shaped like a country, substantially rectangular shape, etc.) In some embodiments, the rendered graphics are smaller for more advanced players (since they are expected to be better players) and are larger for beginners, thus evening the playing field for beginner players at least slightly. In this way, beginners learn to enjoy the game of golf and are encouraged to continue practicing and playing games at sport simulation system 100.

[0357] The fourth operation includes monitoring (964) a path of the respective golf ball after the first game participant putts the respective golf ball (e.g., via information received from cameras 103 via the tracking system interface 204, FIG. 1C).

[0358] The fifth operation is based on whether the first game participant's putt causes the respective golf ball to hit the target (e.g., to fall through a hole on the putting green surface). In accordance with a determination that the first game participant's putt causes the respective golf ball to hit the target, the fifth operation includes assigning (966) a first point value to the first game participant. In accordance with a determination that the first game participant's putt causes the respective golf to come to a stop within the substantially circular graphic, assigning (968) a second point value to the first game participant (e.g., the second point value is smaller than the first point value but still awards points for getting close to the target). In accordance with a determination that the first game participant's putt does not cause the respective golf ball to come to a stop within the substantially circular graphic and does not cause the respective golf ball to hit the target, assigning a third point value to the first game participant (e.g., zero points or possibly negative points).

[0359] Additional operations of method 950 may be interchanged or added to include operations discussed above with respected to FIG. 9A. For example, topography of the putting green surface is re-configured to be more challenging and the method 950 outlined above is repeated for a second plurality of golf balls have a second color. As another example, a new plurality of golf balls having the first color are delivered and a second game participant interacts with the method 950 outlined above. As discussed above, some embodiments, allow for multiple players and multiple rounds of game play.

[0360] It should be understood that the particular order in which the operations in FIG. 9B have been described is merely exemplary and is not intended to indicate that the described order is the only order in which the operations could be performed. One of ordinary skill in the art would recognize various ways to reorder the operations described herein.

[0361] Attention is now directed to FIG. 10, illustrating another game available at the sport simulation system 100. More specifically, FIG. 10 is a flowchart of a method of managing a laser crunch putting challenge game at a sport simulation system, in accordance with some embodiments.

[0362] In some embodiments, a laser crunch game play method 1000 is performed by an electronic device (e.g., system controller 114, FIG. 1A) and/or one or more components of the electronic device (e.g., an I/O subsystem, an operating system, etc.). In some embodiments, the method 1000 is governed by instructions that are stored in a non-transitory computer-readable storage medium and that are executed by one or more processors of a device, such as the one or more processors 202-1 of a management module 114-1 (FIG. 1C). For ease of explanation, the following description of method 1000 is performed by the system controller 114 (also referred to simply as the device). In some embodiments, with reference to FIG. 1C, the operations of method 1000 are performed by or use, at least in part, a surface modification module (e.g., surface modification module 222), a ball color detection module (e.g., ball color detection module 224), a ball path determining module (e.g., ball path determining module 226), a graphics rendering module (e.g., graphics rendering module 228), a game selection module (e.g., game selection module 230), a ball delivery module (e.g., ball delivery module 232), a ball retrieval module (e.g., ball retrieval module 234), user-specific putting data (e.g., user-specific putting data 218), and a leaderboard updating module (e.g., leaderboard updating module 220). Some operations in method 1000 are, optionally, combined and/or the order of some operations is, optionally, changed.

[0363] FIG. 10 is a flowchart of a method of managing a Laser Crunch (also referred to as Odd Ball and Kids' Corner) game at a sport simulation system, in accordance with some embodiments. An exemplary game in a series of short games under the umbrella title Odd Balls is described in the foregoing paragraphs. Odd Balls is a collection of short, fast-paced, exciting games meant for children or people who are not entirely familiar with golf. In addition, some players may just want a fun, unique, video-game-like interface to experience the new technologies that have been developed with the laser guidance system, the undulating green, the scoring, game-flow and sound enhancements.

[0364] In Laser Crunch, the laser guidance system draws several small animals on the ground (e.g., ducks, also referred to as targets that are projected on the surface of a playing surface). The objective is for the player who is putting to shoot the golf balls and intersect the animals, thereby eliminating them. The sound system will indicated progress with blips for hitting the animals. A background sound/musical score will increase in intensity/speed as play progresses. The animals will disappear when hit. The more the player hits, the better they score. Multiple animals can be hit with a single stroke of the putter. The animals may be slowly moving, or may be static, depending on the game setting. If all animals are eliminated within the number of balls allowed, the large children's hole will open up on the green and they have a chance to score more points if they can hit the ball into the hole.

[0365] In some embodiments, method 1000 beings when a set of players from 1-8 will walk onto a dynamic green and touch the player interface control screen (1002). The player interface control screen will allow the player to select what type of game they would like to play, order beverages, evaluate their player statistics, or view the golf merchandise and get prizes (1004, see exemplary user interfaces shown in FIGS. 13A-13C). In this exemplary game, the players have selected Odd Balls under the Games menu, and then Laser Crunch. They will also be asked to choose a number of shapes/animals to eliminate (5, 10, or 20), and whether they want the shapes/animals to be stationary or moving. The green will set itself up for the game: flat with no holes (1006). The guidance system is designed to display the animals for the user to shoot with the golf ball and they will either be moving slowly or stationary depending on the settings (1008). The automatic ball delivery system sets up 5 balls for the player to shoot (1010). The audio system tells the player to begin shooting (1012). The player shoots the balls towards the targets. As the targets are hit by the balls, they are eliminated (1014) and the audio system emits a pinging sound (1016).

[0366] As more targets are eliminated, the background music increases in intensity and speed to keep the game exciting and moving along (1016). The scoreboard is constantly updated, displaying the number and pictures of the targets that were eliminated. The number remaining is also displayed (1018). One shot can score multiple targets. After all 5 shots are taken, and there are still remaining targets, play will continue if the total number of balls shot so far is less than the number of targets chosen (1020No and 1030Yes) per the following schedule: 5 balls total for 5 targets, 10 balls total for 10 targets, 15 balls total for 20 targets. The 5 balls are eliminated from the game by the ball retrieval system. If the player has more shots, 5 more balls are delivered and play resumes. If all the targets are eliminated (1020Yes), the large hole on the green will open up and be lit with arrows pointing to it by the guidance system. The player will score bonus points if they can shoot their remaining balls into the hole (1024Yes). They also get bonus points if they do not require all the balls from the next round (e.g., they eliminate 10 targets with 5 balls or less). The player's turn is complete (1028) and play is passed on to the next player, who repeats the entire game play described above. This process is repeated until all players have completed the game, and the game is then over (1034Yes, 1038). The Scoreboard and audio display will announce the winner in a flashy manner, displaying the winner's avatar (1040) and everyone's final score. Experience points and other incentives is displayed (1042) and game is complete (1044).

[0367] It should be understood that the particular order in which the operations in FIG. 10 have been described is merely exemplary and is not intended to indicate that the described order is the only order in which the operations could be performed. One of ordinary skill in the art would recognize various ways to reorder the operations described herein.

[0368] Attention is now directed to FIG. 11, depicting aspects of a tracking system (e.g., tracking system 102, FIG. 1A). In particular, FIG. 11 is a flowchart that illustrates a method of activating various components of a sport simulation system (e.g., sport simulation system 100, FIG. 1A) in order to use a tracking system with the sport simulation system, in accordance with some embodiments. Stated another way, FIG. 11 identifies the hardware and software process required for the tracking function during a typical game played at a sport simulation system.

[0369] In some embodiments, the tracking system utilizes several high resolution, high speed digital still frame cameras to track the ball position on the green. In some embodiments, the tracking system also has preset knowledge of all possible hole locations and sizes. In some embodiments, a purpose of the tracking system is to: 1) locate balls on the green, 2) track ball motion during each putt, 3) determine ball location over time relative to the hole or other objects (laser targets), 4) associate balls to different players, and 5) interface with other systems as required to support all required gaming modes. In some embodiments, each camera is directly connected to a computer, which contains state-of-the-art machine vision software (for simultaneous, precision tracking of multiple golf balls on a green). In some embodiments, the tracking system interfaces to the other system via a TCP/IP switch and communicates via network sockets.

[0370] In some embodiments, the tracking system requires multiple cameras (e.g., cameras 103, FIG. 1A) to handle situations in which any given single camera may not have an adequate field of view to locate and/or track moving balls as required. Examples of this include cameras being blocked by players, balls being too close to each other to allow for a single camera to separate them, and distance related effects that make closer balls appear larger while further balls appear smaller. In some embodiments, the tracking system also needs to be able to handle spurious objects detected that appear to be balls and filter them out, as well as handling cases where individual cameras do not see every ball in every frame.

[0371] In some embodiments, the tracking system needs to be able to determine the distance between any two points on the green with a fair degree of accuracy (1). As part of the camera and tracking system configuration, a 3-D pixel level mapping of the green is created for each camera using detailed models of the undulating green and its actuators. In some embodiments, the precise location of holes and their state (i.e. open or closed) is known by the tracking system as well as any features being drawn on the green by the projection system. In addition to traditional playing modes, this also allows for virtual reality features to be incorporated into a game via the projection system (e.g., shooting at animal shapes in the Laser Crunch game). In some embodiments, for each camera, any given location on the green will appear in a different location on the image. The tracking system will then be able to understand that a ball that appears in different positions in each camera image is actually the same ball. The distance from any given ball center to any other object (ball, laser target, etc.) is determined using pixel locations in the camera images. In some embodiments, the pixel distance between objects from various camera locations will vary based on the camera distance from the objects. For example, a camera that is close to two adjacent ball might show 300 pixel separation, while a camera much further away at a different angle might only show a 100 pixel separation. Both of these then need to be able to compute the same physical distance of separation. In some embodiments, this information is provided to a system controller (e.g., system controller 114) during a camera integration process.

[0372] Several of the games described herein require the tracking system to maintain knowledge of the distance of the ball to the hole over the duration of the putt to facilitate scoring. And one of the games requires the tracking system to maintain distance knowledge of the ball to a set of moving targets during the putt. In order to accomplish this, in some embodiments, the tracking system takes multiple images per second of the green using each of the cameras. In some embodiments, the image processing software uses a ball discrimination algorithm to locate the balls in each frame. In order to be able to fulfill all requirements related to tracking balls, associating balls with players, and scoring based on ball position relative to various objects, the tracking system needs to have a high level understanding of what is taking place on the green.

[0373] In some embodiments, a state machine is used by the tracking system to monitor and keep track of the flow of the game while players are putting on the green. For example, while all the balls on the green are stationary, there is no scoring taking place, and the tracking system is just waiting for one of the players to initiate a putt. Once a player putts a ball, the tracking system detects this and enters a new state where ball motion is tracked, and required interaction with other systems (audio, guidance) takes place. When the ball comes to a stop, either due to missing the hole or the putt being sunk, the tracking system will perform any required scoring, ball distance computation, and then transition to a state where it is waiting for the next longest putt to take place.

[0374] In some embodiments, the tracking system interacts with almost all the other systems at one point or another during game play.

[0375] In some embodiments, the tracking system receive inputs from a virtual reality system and the ball management system when the players transition from the VR simulator to the putting green. A cue from the ball management system will inform the tracking system to capture and record initial ball locations. In some embodiments, the VR system will inform the tracking system which ball belongs to which player, and the total stroke count to that point in the hole play. In some embodiments, the tracking system will then compute the ball to hole distances for each players ball and determine which player is first to putt. During putting, the tracking system maintains ball to player association, and keeps track of total putts for scoring. When putting is complete, the tracking system checks for the green being clear. When the green is clear, the tracking system informs the VR simulator to proceed to the next hole.

[0376] In some embodiments, the tracking system will receive cues to begin tracking from the state machine, which is the driver that moves the game along sequentially. Player's progress in putting is sent to the state machine for updates as the play moves forward.

[0377] In some embodiments, the tracking system maintains knowledge of putt count during green putting, and communicates total putts to the control/display system so that player scores can be updated in real time after each putt. Each player is associated with a specific ball, so the tracking system informs the display system of who is putting so that players name can be displayed along with any other relevant information concerning the player.

[0378] In some embodiments, the tracking system is the primary source of information being provided to the audio system. Player name announcements and simulated crowd or other noises based on moving ball location or putt finish location are based on information that the tracking system sends to the audio system. Information concerning the hitting of guidance based targets are communicated from the guidance system directly to the audio system.

[0379] In some embodiments, the tracking system interface to the guidance system is primarily used to inform the guidance system of ball location, both when stationary and while the ball is moving. The tracking system cues the guidance system concerning the ball location when a player is about to initiate a putt. Based on this information, the guidance system knows where the ball is, and can begin the process of computing trajectory, speed and backstroke required to make the optimum putt and displaying the required guidance on the green.

[0380] If the guidance system is displaying stationary or moving laser targets, information from the tracking system concerning ball movement over time is passed to the guidance system to allow it to remove or further manipulate objects that the ball hits on its way to the hole. The guidance system will then need to forward that information to the audio system if any additional sound effects are required based on the projected objects being hit. Scoring based on these events also needs to be forwarded by the guidance system to the display system so real time scoring can be displayed.

[0381] In some embodiments, after configuring a playing surface with a desired shape (1102), delivering balls to desired locations (1104), and projecting any targets onto the playing surface (1106), a signal is sent to each camera control computer (e.g., controllers for each camera 103) via TCP/IP sockets to activate the targeting system (1108) by turning on the digital cameras and taking images at 30 frames per second (fps) (1110). Each computer uses machine vision software to locate balls on the green (1112). Once a player steps up to putt a respective ball (1114), the cameras (1116, 1118) associate a ball with that player. The guidance system computes the optimal trajectory path, target point, and pendulum swing for the player (1128) and the projection system displays results on the playing surface (1130). The player makes his putt based on the provided guidance information that was projected on the playing surface (1132). The tracking system cameras record (1134) and the machine vision algorithms track ball trajectory and where each ball comes to rest (1136). If the putt is not made, the process begins again (1120No). If the putt is made (1120Yes), then the shot is complete and the ball management system retrieves the balls from the hole (1122). Data collected during movement of the ball on the playing surface is used to assess the putt and give the player feedback (1124). An instant replay of the putt is also available for viewing, and extra-good putts are stored for the long-term into a player database which may be retrieved later. The tracking system activation method 1100 then ends (1126) and waits for an indication from a system controller (e.g., system controller 114) as to when activation is again required.

[0382] Additional details regarding guidance and tracking systems are also provided above in reference to FIG. 1A-1C.

[0383] FIG. 12 is a flowchart of hardware and software processes required for using guidance functions during an 18 with a bullet game mode selected at a sport simulation system (e.g., sport simulation system 100, FIG. 1A), in accordance with some embodiments. In this example, players have completed shots at a VR simulator (1202) and all approach shots have ended up on the green at their locations and have been delivered to a playing surface via a ball management system (1204). The longest putt player is ready to start putting and ready for the guidance feedback (1204). The control software collects camera images from the deck with the players in place and determines the longest ball and the first player to putt. The software determines that the image is viable and can be used for future processing and commanding (1204). If it is not viable, then more images are taken from the second camera and then verified viable for use without obstructions. The control system will read all actuator positions from the sensors and confirm that the topography is correct for that hole on the specific course and that all actuators have reached the correct location (1208). In some embodiments, a tilt of the playing surface is verified (e.g., in embodiments in which a tilt-able playing surface is utilized, as discussed in reference to FIGS. 3A-3B. A calculation of ball to hole trajectory is then made and an estimation is ready to apply influence functions. The guidance usage method 1200 has distance factored in to also apply influence functions that have that as a variable (1210). The method reviews mechanical influence functions for a current topography of the playing surface and applies a correction based upon the ball location and the estimated trajectory to the hole (1212). These mechanical influence functions are determined based upon an elaborate calculation that takes into account actuator locations (or tilt of the playing surface) and the mechanical properties of the deck assembly (1212). Based upon the deck topography and the ball location with respect to the hole, there is a gravity influence function calculated and applied based upon the peaks and valleys over which the ball has to transverse (1214). The gravity influence is calibrated for every deck topography for every course on every green and applied based upon ball location with respect to the hole (1214). Based upon the distance of the ball to the hole, the method 1200 has access to a green surface friction influence function that will take into account the surface roughness and will have to be calibrated also (1216).

[0384] In some embodiments, the system controller 114 sends instructions to the guidance system to output a gradient across the playing surface that will help a respective participant identify green topography and help that respective participant read the green as they proceed through the putt (1218). The visual guidance can be complemented with audio guidance to identify and point out specific putting/aiming or speed advice (1220). In some embodiments, the system controller 114 sends instructions to the guidance system to output a best fit line across the playing surface showing a respective participant, based upon all the surface conditions, that will allow the respective participant to hit the ball with a correct trajectory to the hole that will allow for making the putt (1222). In some embodiments, the visual guidance can be complemented with audio guidance to identify and point out additional guidance and instructional advice (1226, 1230). In some embodiments, the system controller 114 commands the guidance system to output a the target point on the playing surface (1224). The target point is the point on the green surface that if you hit the golf ball towards that location it would follow along the best fit line and sink the putt (1224).

[0385] All of the guidance commanding is based upon the player's ball location with respect to the hole, factoring all the physical and material influences on the ball as it would proceed. The system controller 114 commands the guidance system to output a putting pendulum back and forth putter motion graphic (1228). The system will accept a calibration for each player to identify an influence that has a factor related to how hard the individual hits with the selected putter (1232) for that game (1228). The player makes the putt and the camera system records the whole time the actual motion of the ball (1234). In some embodiments, the system controller 114 commands audio output based upon the progress as the put is taking place (1236). The system controller 114 commands the guidance system to output an actual ball path on the playing surface so that the player can see where the ball went with respect to the best fit line so that putting adjustments can be done in the future (1238 and in some embodiments, audio is also provided 1240 that indicates how the player could improve their short in order to more closely follow the best fit line). In some embodiments, player putting statistics are recorded and sent to a scoreboard and retained over the entire game. In some embodiments, the player putting experience is recorded during the entire game and can be retrieved any time and sent to remote computing device for future playback. The system controller 114 then identifies the next longest putt (i.e., furthest away from the hole) and the visual and audio guidance will proceed as described above for method 1200 with respect to the next longest putt.

[0386] In some embodiments, the guidance system also includes a large touch-sensitive display that the game and guidance systems can use to display video capture of the putt, video display of a proper motion, scores from either games or putting evaluations. The information in the large display is captured and recorded using the data from the camera images (e.g., data captured by the cameras 103, FIG. 1A).

[0387] FIGS. 13A-13C show exemplary user interfaces for selecting various games or modes. As shown in FIGS. 13A-13C, gaming user interfaces 1302, 1304, and 1306 allow participants of games at a sport simulation system (e.g., sport simulation system 100, FIG. 1A) to make game selections, order food and beverages, and view player cards and other user-specific information. In some embodiments, the gaming interfaces 1302, 1304, and 1306 are available using devices that are provided with the sport simulation system 100, while in other embodiments, the gaming interfaces are accessible through any mobile device.

[0388] FIG. 14 is a flowchart of a method 1400 of projecting guidance lines onto a playing surface, in accordance with some embodiments.

[0389] In some embodiments, the method 1400 is performed by an electronic device (e.g., system controller 114, FIG. 1A) and/or one or more components of the electronic device (e.g., an I/O subsystem, an operating system, etc.). In some embodiments, the method 1400 is governed by instructions that are stored in a non-transitory computer-readable storage medium and that are executed by one or more processors of a device, such as the one or more processors 202-1 of a management module 114-1 (FIG. 1C). For ease of explanation, the following describes method 1400 as performed by the system controller 114 (also referred to simply as the device). In some embodiments, with reference to FIG. 1C, the operations of method 1400 are performed by or use, at least in part, a surface modification module (e.g., surface modification module 222), a ball color detection module (e.g., ball color detection module 224), a ball path determining module (e.g., ball path determining module 226), a graphics rendering module (e.g., graphics rendering module 228), a game selection module (e.g., game selection module 230), a ball delivery module (e.g., ball delivery module 232), a ball retrieval module (e.g., ball retrieval module 234), user-specific putting data (e.g., user-specific putting data 218), and a leaderboard updating module (e.g., leaderboard updating module 220). Some operations in method 1400 are, optionally, combined and/or the order of some operations is, optionally, changed.

[0390] In some embodiments, a method 1400 optionally includes: retrieving (1402) information identifying putting characteristics associated with a first user. In some embodiments, this information is stored in a data structure on the system controller 114 (or could be stored at a server remotely location from the system controller 114), such as user-specific putting data 218 (FIG. 1C). In some embodiments, the user-specific putting data 218 includes information about swing data associated with a plurality of users of a sport simulation system, the plurality include the first user. The swing data includes impact speed, impact angle, backswing distance, follow-through distance, difficulty levels for each shot, and the like.

[0391] The method 1400 includes identifying (1404) a current topography of a putting green and a current position of a ball on the putting green, the ball being associated with the first user. In some embodiments, a tracking/guidance system 102 provides information to system controller 114 in order to allow the system controller 114 to identify the current topography and the current position of the ball.

[0392] The method 1400 also includes: determining (1406), based on the current topography of the putting green, a best path from the current position of the first user's ball on the putting green to a target on the putting green. In some embodiments, the best path is further based on the putting characteristics associated with the first user (1414). Best paths are discussed in more detail in reference to FIGS. 15A-15K.

[0393] In some embodiments, the method 1400 optionally includes determining (1408) a backswing distance and a corresponding follow-through distance that will allow the first user to hit the ball along the best path. Backswing and follow-through distance determinations are also discussed below in reference to FIGS. 15A-15K.

[0394] The method 1400 additionally includes sending (1410), to a projecting device that is distinct from the electronic device, instructions to render a representation of the best path on the putting green. In some embodiments, the method 1400 optionally includes: sending (1412), to the projecting device, instructions to render a first graphic at a first position on the putting green that corresponds to the backswing distance relative to the current position of the golf ball, and render a second graphic at a second position distinct from the first position on the putting green that corresponds to the follow-through distance relative to the current position of the golf ball.

[0395] It should be understood that the particular order in which the operations in FIG. 14 have been described is merely exemplary and is not intended to indicate that the described order is the only order in which the operations could be performed. One of ordinary skill in the art would recognize various ways to reorder the operations described herein.

[0396] FIGS. 15A-15B are schematics of a best fit line and associated guidance graphics projected onto a playing surface of a sport simulation system, in accordance with some embodiments. As shown in FIG. 15A, the best fit line and associated guidance graphics may include a backswing graphic 1518, a ball alignment graphic 1510, a follow-through graphic 1506, and a best path graphic 1504 (FIGS. 15A and 15B). The backswing graphic 1518 and follow-through graphic 1506 instruct a respective game participant as to the appropriate speed at which to strike a ball in order to allow the ball to follow the best fit line and make a respective putt. In some embodiments, the associated guidance graphics also include a target alignment graphic 1502 (FIG. 15B) that provides the respective game participant with a visual indication as to where they should target their putt in order to make it into the hole (or to hit some other suitable target). Additional details regarding determining appropriate guidance graphics are provided throughout this description and, in particular, in reference to FIGS. 15C-15K below.

[0397] FIGS. 15C-15K are coordinate frame diagrams of a playing surface used to illustrate proper determination of a best fit line, in accordance with some embodiments.

[0398] In some embodiments, a physics-based computation is utilized to determine a best path to a hole. In some embodiments, the optimal trajectory for a successful golf putt depends on the shape and speed of the green, the initial interaction between the ball and putter, and the properties of the ball. Assuming a standard golf ball and ignoring the nuances introduced by various ball manufacturers, as these are more relevant to driving than putting, we focus on the green and the putt. Our goal is to determine the best path for an arbitrarily shaped green of known speed with a player who putts using a pendulum swing of known arc length and timing. This is accomplished using a physics model for the motion of a ball on a putting green of arbitrary shape. In some embodiments, the determination of the best path breaks down as follows: (1) Determine the shape of the green; (2) Find the initial velocity vector for the ball that will produce a successful putt; (3) Map this onto a specific swing direction and pendulum arc for the player.

[0399] In some embodiments, surface modeling techniques are utilized to determine a best path for any arbitrary green surface with the ball located at any point with respect to the hole. This first requires computing the shape of the surface with sufficiently high fidelity to model the physics interaction between the moving ball and the undulated green. In some embodiments, cameras and stereoscopic vision are utilized. A stereo camera rig will be located above the green surface and will estimate its shape every time it changes. The cameras are calibrated into a common coordinate frame with the green. This allows the recovered surface information to be transformed into a uniformly gridded height map. Thus, topographic data consists of an elevation model with respect to the zero position of a playing surface (lowest and flattest). In some embodiments, stereo algorithms are utilized that are able to use either these or commercial off-the-shelf stereo packages, provided the off-the-shelf packages produce a point cloud in some standard format.

[0400] For a flat, tilted green, the acceleration profile for a ball is determined to first order by the velocity imparted by the putt, the tilt of the green, and the speed of the green. In some embodiments, the ball is assumed to be a uniform solid sphere. An initial coordinate frame is established with y defined by the line from the ball to the hole and x by its perpendicular (as shown in FIG. 15C). Let (, ) be the tilt angles of the plane with respect to x and y, i.e., the pitch and the roll of the green (as shown in FIGS. 15D-15E) with respect to the instantaneous direction of travel of the ball. Then the acceleration vector a=(a.sub.x, a.sub.y) for a ball in motion is given by the following:

a.sub.x=g.Math.sin()f/m.Math.sin()

a.sub.yg.Math.cos()sin()f/m.Math.cos() [0401] where, [0402] g=acceleration due to gravity

=a tan 2(r.sub.g.Math.cos()cos()sin()I.sub.b.Math.sin(),r.sub.g.Math.cos()cos()cos()I.sub.b.Math.cos()sin())

m.Math.f=(r.sub.g.Math.cos()cos()cos()I.sub.b.Math.cos()sin())/((1+I.sub.b)cos()).Math.g [0403] with r.sub.g=speed of green, I.sub.b=moment of inertia of ball normalized by (m R.sup.2)=2/5 for a solid sphere, =yaw angle between line of putt and line to hole (as shown in FIG. 15F), and m=mass of ball.

[0404] Given the above retarding acceleration and an initial position and velocity, the path of the ball can be predicted. However, this applies only to a flat green. For an arbitrary surface, the above set of equations are applied locally, in a piecewise fashion. For any instant in time, the surface model described above gives (, ) for the plane tangent to the surface at its point of contact with the ball. If the ball's velocity at that time is known, the acceleration profile above can be applied for a short extent over which the green is nearly planar. This extent is a tunable parameter in our system. Once the ball has traveled beyond this planar patch, the process is repeated with its new velocity determined from the previous patch and the local tilt of the green determined from the surface model. Observe that at every patch, (x, y, , , ) all change. In other words, there is a moving coordinate frame defined by the location of the ball and it's line to the hole. Integrating the ball's motion is continued in this manner until one of three conditions is met: (1) the ball's velocity and acceleration drop beyond a threshold, at which points it is declared to have stopped, (2) the ball is captured by the hole, or (3) the ball leaves the green area.

[0405] If the ball's path crosses the hole, a decision must be made as to whether it is captured by the hole. If R.sub.h is the radius of the hole and R is the radius of the ball, the first requirement for capture is that the distance from the center of the ball to the hole is less than R.sub.h somewhere along its trajectory. However, this is not sufficient for capture as the ball may skip over the hole or skirt it. This is captured as follows:

vel_=1/sqrt(1cos()sin()).Math.(sqrt(R.sub.h.sup.2.sup.2)+sqrt((R.sub.hR).sup.2.sup.2)).Math.sqrt(g/(2R))

vel_=1/sqrt(1+sin()sin()).Math.(sqrt(R.sub.h.sup.2.sup.2)+sqrt((R.sub.hR).sup.2.sup.2)).Math.sqrt(g/(2R))

[0406] If the ball's velocity v has norm less than vel_ and vel_, it will be captured. In the above, is the component of the ball's straight-line trajectory not in line with the direction to the hole.

[0407] The physics model described thus far can predict a path and a capture condition given the initial position and velocity of the ball. However, velocity is not a concept that can easily be presented to a player. Instead, a player is shown an arc length for a pendulum swing that will result in a given velocity. It is the case that the initial velocity of the ball depends not only on the length of the arc but on the speed of the swing and the weight of the club. However, averages are assumed for a new player and data can be collected on individual players to better determine specific mappings between swing and ball velocity. This amounts to generating a lookup table for each player/club combination. The table starts with a pre-defined default for new players and adapts as the player's data becomes known to the system.

[0408] The above has described a forward modeling problem. Given the shape and properties of the surface and an initial velocity (i.e., known pendulum swing), a determination of the path of the ball and the capture condition can be approximated. In some embodiments, what is desired is to suggest a swing that will result in a captured ball. In some embodiments, a combination of good initialization through heuristics and a non-differential optimization via the Nelder-Mead algorithm is utilized. The Nelder-Mead approach mitigates the complexity of the physics model, since it is a search strategy rather than a gradient based algorithm. However, such approaches benefit tremendously from good initialization.

[0409] In some embodiments, initialization is as follows: Given, a green shape and the ball and hole location, we will determine a set of putts at some standard ball velocity and a set of angles sweeping the approximate direction to the hole. We will then determine the two paths L.sub.hole and R.sub.hole that pass the hole closest to either side. Let B be the line segment from L.sub.hole to R.sub.hole passing through the hole that is perpendicular to both L.sub.hole and R.sub.hole. Let B.sub.l be the vector from the hole to L.sub.hole along B and let B.sub.r be the vector from the hole to R.sub.hole along B (as shown in FIG. 15G). Then w.sub.l=|Bl|/|B| and w.sub.r=|Br|/|B| are weighted distances between the hole and L.sub.hole and R.sub.hole, respectively. If the velocity vectors associated with the putts resulting in L.sub.hole and R.sub.hole are v.sub.l and v.sub.r, respectively, we set the initial velocity for the optimizer at;

v.sub.init=w.sub.l.Math.v.sub.l+w.sub.r.Math.v.sub.r

[0410] The more finely sampled the initial sweeping angles, the better the approximation vinit will be at the expense of more computational time.

[0411] The model described thus far of a ball as a solid uniform sphere with a mass m and a moment of inertia I.sub.b is an approximation that holds because it has little impact on the putting game. However, a golf ball is not a uniform sphere but consists of a sequence of layers, usually with a dense metallic inner core and a soft lighter outer layer. The materials and layers of the ball, and thus m and I.sub.b, vary depending on the ball manufacturer, but these are minor effects for the moderate speed rolling motion that occurs during a putt. Likewise, in reality the speed of the green r.sub.g is affected by environmental conditions such as heat and humidity. Indoors, on an artificial surface, these effects are expected to be minimal as well. Finally, the physics model holds for a ball in motion but is not valid at the moment of impact with the putter or when the ball approaches zero velocity. The former is not an issue because our mapping from swing to velocity just after impact amounts to a lookup table rather than a physics model. The low velocity case is only relevant if the velocity approaches zero just as the ball approaches the hole. However, this is of little concern, since the typical predicted best path will not have near-zero velocity near the hole.

[0412] To project a best-fit guidance line on a playing surface, the software system must compute the shape of the green surface, iteratively execute the physics model until a path to the hole is found, fit a curve that passes through the set of x,y coordinates along this path, and project this curve onto the green. Additionally, these activities should be completed within a sufficiently short period of time as to not negatively impact player experience. This performance goal is met by parallelizing both the surface topology and iterative physics model computation through calls to a general-purpose computing on graphics processing units (GPGPU) API. In some embodiments, a physics model is implemented in Numerical Python (NumPy) and uses NVIDIA Compute Unified Device Architecture (CUDA) extensions to achieve GPGPU parallelization. The guidance path is drawn in the Unity game engine, which finds a spline that passes through each x,y coordinate pair that is returned from the physics model when a path to the hole is found. Unity also stitches the light fields of multiple projectors together and draws this spline curve onto the green.

[0413] During gameplay at a sport simulation system, player data is collected in real time and sent to a cloud-based distributed database through a Representational State Transfer (REST) API. These metrics consist of, but are not limited to, initial putt velocity, width of pendulum swing, and putt angle. A server-side streaming analytics layer leverages this data to compute derived statistics such as a guidance line tailored to a player's swing speed, putter choice, and handedness. As with the physics model output, derived statistics directly affect player experience. Therefore, the time between acquisition of player data and delivery of derived statistics must be sufficiently short as to avoid noticeable delays in gameplay. The query component of this timeliness requirement is met by defining all query types when the database is created. This design guarantees that data is optimally indexed for each query. The streaming analytics layer gracefully handles various levels of load by claiming and relinquishing compute nodes as neededa property that ensures a consistent level of performance.

[0414] The trajectory of the ball along an arbitrary surface depends on the initial velocity and direction of the ball. When the ball is on the green the trajectory is initiated by a pendulum swing that transfers energy from the player to the ball, exerting a force that drives the ball in a given direction at a given velocity. The energy imparted to the ball is a function of the amplitude of the pendulum swing and of the strength of the stroke. Although there is a relationship between these two quantities, this relationship is not fixed, unless the player adds no strength to the swing. This leads to two different techniques of putting a ball: 1) Adding no force to the putting swing and letting the width of the pendulum determine the force to the ball. This technique is consequent with the tendency of experienced players to hit the ball with the same swing and adjusting the desired reach distance by varying the club instead of the swing. In this case, the initial velocity of the ball is mainly determined by the width of the pendulum swing; and 2) Keeping the pendulum swing constant and varying the reach distance by adding force to the swing. In this case, the initial velocity of the ball is mainly determined by the force added to the swing, not by the width of the pendulum swing.

[0415] In practice, some players consistently prefer to vary the width of the pendulum swings while others prefer to always use tight swings, regardless of the distance to the hole. Players that do not add force to the swing do muscle the stroke but simply adjust the width of the pendulum to add force to the stroke; this is advantageous to strike the ball a given distance since there is only one variable to control: the width of the swing; however, wider swings give opportunity to the face of the putter to open or close, with increase the error in the initial direction of the ball. On the other hand, players with a consistently tight swing need to compensate for the lack of amplitude of the pendulum by adding force to the stroke; however, a tight stroke gives little opportunity to the face of the putter to open or close so there is little error in the initial direction of the ball. The actual putting game of a player usually lies between these two extremes.

[0416] In some embodiments, both the force of the stroke and the initial direction of motion of a stroke is measured. The length of the pendulum swing as a function to the distance to the hole is determined from the imagery, while the strength of the stroke is determined from the initial velocity of the ball, which is proportional to its kinetic energy, which in turn is proportional to the contact force. The second factor in the determination of the trajectory of the ball, the initial direction of motion, is also determined from the imagery.

[0417] The statistical data that describes the initial direction of motion of the ball and its initial velocity is used to determine the characteristics of the putting game of the player. These characteristics are stored in a database. From this statistical data we can determine variations in the initial velocity and in the initial direction of motion. Integrating the initial velocity we obtain the initial acceleration (a.sub.x, a.sub.y), which is proportional to the force F used to strike the ball. Hence, variations on the initial velocity yield variations F. Likewise, variations in the initial direction of motion lead to left and right variations of the yaw angle, i.e., .sub.L and .sub.R. These variations add uncertainty to the path that the ball will follow so the cone widens (as shown in FIG. 15I). The width of the cone of a player at the height of the hole, W.sub.player, is a measure of goodness of the player; inexperienced players will have a large W.sub.player while experienced ones will have one that approaches W.sub.green, which is the smallest possible width at the height of the hole.

[0418] The system displays a trajectory consistent with the statistical length of the pendulum swing for putts of the given distance tailored for each particular player. Trajectories of players that tend to strike the ball well, with appropriate contact forces and hit the ball true have narrow cones, while those of players who hit the ball with incorrect contact forces, or that hit the ball opening or closing the face of the putter, will have wide error cone (as shown in FIG. 15J).

[0419] Independent of the player, the minimum width of the cone is defined by the uncertainties of the green and the ball, i.e., trajectories over an ideal smooth surface with an ideal ball have an cone with a width of Whole at the height of the hole; any non-ideal terrain has a roughness that creates an uncertainty over the trajectory that increases with distance. The characteristics of the game of a player always add uncertainty to the ideal trajectories, never subtract from them. The best line is modified to fit the particular game preferences of the player. For example, consider a player that hits the ball with a consistent contact force. For this player, a modification is made to the best line that minimizes the width of the cone at the hole among the trajectories produced by this particular contact force (as shown in FIG. 15K). Although the generated trajectory with this particular contact force is not an ideal trajectory (in the sense that there exist trajectories with narrower cones), it is generated with a stoke with which the player feels comfortable as it follows that statistical way in which the player putts. A similar adjustment can be done considering the length of the pendulum swing instead of the strike force.

[0420] In some embodiments, learning is also based on other inputs (e.g., cameras and 3D sensors). The selection of the trajectory is used as a pedagogical tool that instead of adapting the suggested stroke to the style of the player, it instead indicates ways in which the player can minimize the cone by modifying his/her game. The system can purposefully display a trajectory with a small error cone that the player can follow by altering one or more of his putting characteristics. For example, if the system determines that a source of inconsistency of the player putts is that the initial direction of motion is erratic, then the likely cause is that the player is opening and/or closing the face of the putt at the moment of contact with the ball. In this case, the system will display a trajectory with a tight pendulum swing that minimizes the opportunity of the face of the putt to open or close. To compensate for the tight pendulum swing, the player is forced to increase the strength of his/her stroke.

[0421] In some embodiments, length of the pendulum swing and the initial direction of the ball can be determined by imagery of the ball. Additional information about the swing, that also affects the ball trajectory, is obtained by analyzing the player instead of the ball. Imagery of the player can show errors in the putt that explain wider cones in the trajectory. For example, bobbing of the head, incorrect shoulder height, and body motion, can all be detected directly from images of the player and are used to suggest courses of action that are likely to reduce the cones of his/her trajectories. Hence, the W.sub.player of the resulting trajectories becomes the measure against which we verify whether the suggestions to the player are taking effect, and their change over time is displayed as guidance and motivation. The statistically obtained W.sub.player value is a measure that summarizes the quality of the putting game of a player at any given time.

[0422] FIG. 16 is a flowchart of a method of managing a game of hand golf at a sport simulation system, in accordance with some embodiments.

[0423] In some embodiments, the method 1600 is performed by an electronic device (e.g., system controller 114, FIG. 1A) and/or one or more components of the electronic device (e.g., an I/O subsystem, an operating system, etc.). In some embodiments, the method 1600 is governed by instructions that are stored in a non-transitory computer-readable storage medium and that are executed by one or more processors of a device, such as the one or more processors 202-1 of a management module 114-1 (FIG. 1C). For ease of explanation, the following describes method 1600 as performed by the system controller 114 (also referred to simply as the device). In some embodiments, with reference to FIG. 1C, the operations of method 1600 are performed by or use, at least in part, a surface modification module (e.g., surface modification module 222), a ball color detection module (e.g., ball color detection module 224), a ball path determining module (e.g., ball path determining module 226), a graphics rendering module (e.g., graphics rendering module 228), a game selection module (e.g., game selection module 230), a ball delivery module (e.g., ball delivery module 232), a ball retrieval module (e.g., ball retrieval module 234), user-specific putting data (e.g., user-specific putting data 218), and a leaderboard updating module (e.g., leaderboard updating module 220). Some operations in method 1600 are, optionally, combined and/or the order of some operations is, optionally, changed.

[0424] In some embodiments, a method 1600 includes: providing (1602) a plurality of golf balls, each visually identifiable as belonging to either a first plurality of golf balls associated with a first participant or a second plurality of golf balls associated with a second participant. Each golf ball may be identifiable based on color, shape, size, hash-shading, or some other distinguishing characteristics. In other embodiments, golf balls are associated with participants using only data obtained from visual sensors and without requiring the golf balls to have distinguishing characteristics.

[0425] The method 1600 further includes: monitoring (1604), using one or more sensors communicably coupled with the electronic device (e.g., one or more cameras 103, FIG. 1A), continuous throws of golf balls in the plurality of golf balls towards a hole extending through a surface of a playing surface.

[0426] The method 1600 also includes: detecting (1606), by at least one detection sensor communicably coupled with the electronic device (e.g., one or more color detection sensors, whenever a golf ball of the plurality of golf balls has passed into the hole. In other embodiments, targets are instead utilized and holes are not utilized. For example, one target is a gopher or bulls-eye moving around a playing surface. In some embodiments, a combination of targets and holes are utilized to provide an exciting game with various scoring values depending on the target or hole that is hit.

[0427] The method 1608 additionally includes: determining (1608) whether each golf ball of the plurality of golf balls that has passed into the hole is associated with the first plurality of golf balls or the second plurality of golf balls.

[0428] In some embodiments, the method 1600 further includes: assigning (1610) a first predetermined point value to the first game participant for any golf balls that passed into the hole that are associated with the first plurality of golf balls and assigning the first predetermined point value to the second game participant for any golf balls that passed into the hole (or, in other embodiments, golf balls that hit some other suitable target) that are associated with the second plurality of golf balls.

[0429] In some embodiments, the method 1600 also includes: assigning (1612) a second predetermined point value to the first game participant for any golf balls that: (i) are associated with the first plurality of golf balls, and (ii) are determined, based on the monitoring of the continuous throws, to have passed into the hole without first bouncing on the playing surface and assigning the second predetermined point value to the second game participant for any golf balls that: (i) are associated with the second plurality of golf balls, and (ii) are determined, based on the monitoring of the continuous throws, to have passed into the hole without first bouncing on the playing surface. In this way, additional points are provided to users for hitting the target or getting a ball into the hole without having to bounce or roll the ball.

[0430] In some embodiments, the second predetermined point value is optionally larger than the first predetermined point value (1614). In other embodiments, the second predetermined point value is a bonus point value that is added on to the first predetermined point value that provides additional points for more difficult balls that hit a target or make it into a hole.

[0431] In some embodiments, after throwing all balls in the first and second pluralities, the system controller 114 also determines which balls are within a predetermine distance of the target or hole (e.g., within 1 foot) and then assigns additional points based on distance from the target or hole.

[0432] It should be understood that the particular order in which the operations in FIG. 16 have been described is merely exemplary and is not intended to indicate that the described order is the only order in which the operations could be performed. One of ordinary skill in the art would recognize various ways to reorder the operations described herein.

[0433] FIG. 17 is a schematic of a putting green surface used to illustrate a robotic assistant (e.g., game play assistant graphic 1702) projected on the putting green surface, in accordance with some embodiments. In some embodiments, after a particular game is completed (or a respective round of a particular game), the system controller 114 sends instructions to the tracking/guidance system 102 (FIG. 1A) that allow the system 102 to render the game play assistant graphic 1702 and to render a line that corresponds to a path followed by a particular user's golf shot. In some embodiments, the graphic 1702 is coordinated with the ball delivery system 108 (FIG. 1A) so that balls are cleared from the playing surface 104 at the same time that the graphic 1702 passes the balls.

[0434] FIGS. 18A-18B are schematics of a putting green surface used to illustrate a graphic projected on the surface of the putting green surface that illustrates changing height positions for linear actuators, in accordance with some embodiments. As shown in FIG. 18A, representations of surface modification elements 106 (FIG. 1A) are projected on to a playing surface 104 as surface modification graphics 1802. In some embodiments, as the playing surface 104 is configured to have a different contour or a different tilt (see discussed in FIGS. 3A-3B and 3C), the surface modification graphics 1802 are lowered or raised to provide an indication that the surface modification elements 106 beneath the surface are being actuated. Initially, the surface modification graphics 1802 may be positioned at uniform height levels (FIG. 18A) and then each individual surface modification graphic 1806 is modified during contouring/tilting of the playing surface 104 to raise and lower height of the graphics (as shown with exemplary raised graphic 1804 and lowered graphic 1806).

[0435] Attention is now directed to FIG. 19A, which is a schematic diagram showing a physical gaming suite with anamorphic images projected therein, in accordance with some embodiments. As shown in FIG. 19A, embodiments disclosed herein are capable of projecting anamorphic images on one or more surfaces within the physical gaming suite 1900, including a bottom surface 1904 (e.g., a deformable surface, such as that described in reference to FIG. 19C) and/or a back surface 1902 (e.g., a projection screen that is included within the physical gaming suite). In some embodiments, the bottom surface 1904 is substantially perpendicular to the back surface 1902. For example, the bottom surface 1904 is a playing surface (e.g., playing surface 104, FIGS. 1A-1B) on which a game is played by game participants (e.g., a putting game).

[0436] Some embodiments also project images on or over side surfaces, ceiling surfaces, and other objects within the physical gaming suite such as chairs, hula hoops, golf clubs, and others. As explained in more detail below, anamorphic images are presented within the physical gaming suite as 2D images (as shown in FIG. 19A that are viewable in 3D by game participants within the physical gaming suite 1900 (examples of 3D viewing of the images are shown in FIGS. 19B-19D from various viewpoints within the physical gaming suite 1900). In some embodiments, respective anamorphic image is viewable in 3D by only those game participants whose viewpoints were utilized in generating the respective anamorphic image (i.e., only some game participants are able to appreciate a 3D effect for a particular anamorphic image that is projected within the physical gaming suite).

[0437] FIG. 19A shows a first anamorphic image 1916 that is presented on a combination of the back surface 1902 and the bottom surface 1904 of the physical gaming suite 1900. In some embodiments, presenting the first anamorphic image 1916 includes presenting a first component 1916-1 of the anamorphic image 1916 on the back surface 1902 and presenting a second component 1916-2 of the anamorphic image 1916 on the bottom surface 1904. In some embodiments, the first and second components are determined so that the anamorphic image 1916 is properly viewable in 3D by at least two game participants within the physical gaming suite (e.g., game participants that are associated with viewpoints 1910 and 1912).

[0438] FIG. 19A also shows a second anamorphic image 1918 that is presented on the back surface 1902 of the physical gaming suite 1900. In some embodiments, the anamorphic image 1918 is properly viewable in 3D by one game participant within the physical gaming suite (e.g., a game participant that is associated with viewpoints 1914).

[0439] FIG. 19A additionally shows a third anamorphic image 1920 that is presented on the bottom surface 1904 of the physical gaming suite 1900. In some embodiments, the anamorphic image 1920 is properly viewable in 3D by at least two game participants within the physical gaming suite (e.g., a first game participant that is associated with viewpoint 1910 and a second game participant that is associated with viewpoint 1912).

[0440] Attention is now directed to FIGS. 19B-19D, which are schematic diagrams showing how the anamorphic images of FIG. 19A appear from different viewpoints within the physical gaming suite, in accordance with some embodiments.

[0441] As shown in FIG. 19B, the anamorphic image 1918 that is presented within the physical gaming suite is not viewable in 3D by a first game participant that is associated with viewpoint 1910. In some embodiments, the viewpoint 1910 is excluded from a viewpoint determination (e.g., such as operation 2108, FIG. 21A, and/or operation 2208, FIG. 22) that is used to generate the anamorphic image 1918 (e.g., because the viewpoint 1910 is too far away or too different (e.g., an angle of viewing would be too great) from another viewpoint, such as viewpoint 1914, and would thus ruin a 3D effect for all game participants). FIG. 19B also shows that the anamorphic images 1916 and 1920 are viewable in 3D by the first game participant that is associated with viewpoint 1910. The 3D effect that is viewable from viewpoint 1910, however, may be different from the 3D effect that is viewable by a second game participant that is associated with a different viewpoint (e.g., such as viewpoint 1912, in that the 3D effect may appear at a different position or may be slightly distorted).

[0442] As shown in FIG. 19C, the anamorphic image 1918 that is presented within the physical gaming suite is not viewable in 3D by the second game participant that is associated with viewpoint 1912. In some embodiments, the viewpoint 1912 is excluded from a viewpoint determination (e.g., such as operation 2108, FIG. 21A, and/or operation 2208, FIG. 22) that is used to generate the anamorphic image 1918 (e.g., because the viewpoint 1912 is too far away or too different (e.g., an angle of viewing would be too great) from another viewpoint, such as viewpoint 1914, and would thus ruin a 3D effect for all game participants). FIG. 19C also shows that the anamorphic images 1916 and 1920 are viewable in 3D by the second game participant that is associated with viewpoint 1912. The 3D effect that is viewable from viewpoint 1912, however, may be different from the 3D effect that is viewable by the first game participant that is associated with a different viewpoint (e.g., such as viewpoint 1910, in that the 3D effect may appear at a different position or may be slightly distorted).

[0443] As shown in FIG. 19D, the anamorphic image 1918 that is presented within the physical gaming suite is viewable in 3D by a third game participant that is associated with viewpoint 1914. In some embodiments, the viewpoint 1914 is included in a viewpoint determination (e.g., such as operation 2108, FIG. 21A, and/or operation 2208, FIG. 22) from which viewpoints 1910 and 1912 were excluded. FIG. 3D also shows that the anamorphic images 1916 and 1920 are not viewable in 3D by the third game participant that is associated with viewpoint 1914. In some embodiments, in accordance with a determination that a particular game participant (e.g., the third game participant) would not be able to view any 3D effects, the system 100 generates and presents a new anamorphic image for viewing by the particular game participant (i.e., to ensure that all game participants are able to experience and view a 3D object during game play within the physical gaming suite).

[0444] Attention is now directed to FIGS. 20A-20G, which are schematic diagrams showing example anamorphic images that are projected on one or more surfaces of a physical gaming suite, in accordance with some embodiments. In some embodiments, anamorphic images are presented in 2D on only a bottom surface (e.g., FIGS. 20A-20B), a bottom surface and a back surface (e.g., FIGS. 20C-20E), and only a back surface (e.g., FIGS. 20F-20G).

[0445] As shown in FIGS. 20A-20B, an anamorphic image 2002 of a gopher popping out of a hole can be presented on only a bottom surface of a physical gaming suite and can be presented so that at least one game participant is able to view and appreciate a 3D effect (such as that shown in FIG. 20B). In some embodiments, multiple game participants are able to appreciate and view the 3D effect (e.g., so that the viewpoint/user shown in FIG. 20B is a representation of an average/common/optimal viewpoint for the multiple game participants that are all able to view the 3D effect of the gopher popping out of the hole).

[0446] In some embodiments, an intended beginning position for the 3D effect is used (e.g., notated as a in FIGS. 20A-20G) in order to determine optimal viewpoints (as explained below in reference to FIG. 22).

[0447] As shown in FIGS. 20C-20E, an anamorphic image 2004 of a bottle that is standing up within the physical gaming suite 1900 can be presented on both the bottom surface 1904 and back surface 1902. In some embodiments, a first component 2004-1 is presented on the bottom surface 1904 (as shown in FIGS. 20C-20D) and a second component 2004-2 is presented on the back surface 1902 (as shown in FIG. 20D). In some embodiments, multiple game participants are able to appreciate and view the 3D effect (e.g., so that the viewpoint/user shown in FIG. 20E is a representation of an average/common/optimal viewpoint for the multiple game participants that are all able to view the 3D effect of the bottle standing up within the physical gaming suite 1900).

[0448] As shown in FIGS. 20F-20G, an anamorphic image 2006 of a bottle that is standing up within the physical gaming suite 1900 can also be presented on only the back surface 1902. In some embodiments, multiple game participants are able to appreciate and view the 3D effect (e.g., so that the viewpoint/user shown in FIG. 20G is a representation of an average/common/optimal viewpoint for the multiple game participants that are all able to view the 3D effect of the bottle standing up within the physical gaming suite 1900).

[0449] Attention is now directed to FIGS. 20A-20C and FIG. 22, which are flowcharts of methods of programmatically generating anamorphic images for presentation (and 3D viewing) in a physical gaming suite.

[0450] In particular, FIGS. 20A-20C are flowcharts of a method of programmatically generating anamorphic images for presentation (and 3D viewing) in a physical gaming suite, in accordance with some embodiments.

[0451] In some embodiments, the method 2100 is performed by an electronic device (e.g., system controller 114, FIG. 1A) and/or one or more components of the electronic device (e.g., an I/O subsystem, an operating system, etc.). In some embodiments, the method 2100 is governed by instructions that are stored in a non-transitory computer-readable storage medium and that are executed by one or more processors of a device, such as the one or more processors 202-1 of a management module 114-2 (FIG. 1D). For ease of explanation, the following describes method 2100 as performed by the system controller 114 (also referred to simply as the device). In some embodiments, with reference to FIG. 1D, the operations of method 2100 are performed by or use, at least in part, a surface modification module (e.g., surface modification module 222), a ball color detection module (e.g., ball color detection module 224), a ball path determining module (e.g., ball path determining module 226), a graphics rendering module (e.g., graphics rendering module 228), a game selection module (e.g., game selection module 230), user-specific putting data (e.g., user-specific putting data 218), a leaderboard updating module (e.g., leaderboard updating module 220), a visual characteristics monitoring module (e.g., visual characteristics monitoring module 250), and/or a viewpoint determining module (e.g., viewpoint determining module 252). Some operations in method 2100 are, optionally, combined and/or the order of some operations is, optionally, changed.

[0452] In some embodiments, the method 2100 is performed at a computing device (e.g., system controller 114, FIG. 1A) with one or more processors (e.g., CPUs 202-1, FIG. 1D) and memory (e.g., memory 206, FIG. 1D). The computing device is in communication with one or more visual sensors (e.g., via tracking system interface 204, FIG. 1D, the computing device is able to communicate with one or more cameras 103, FIG. 1A) and one or more display devices (e.g., via tracking system interface 204, FIG. 1D, the computing device is able to communicate with one or more projectors 105, FIG. 1A). In some embodiments, the one or more cameras 103 and the one or more projectors 105 are positioned within a physical gaming suite 1900 (FIG. 19A).

[0453] The computing device monitors (2102), using data received from the one or more visual sensors, viewing characteristics associated with one or more game participants in the physical gaming suite. For example, the viewing characteristics include eye gaze, head position, current standing/sitting positing within the physical gaming suite for each of the one or more game participants. In some embodiments, the viewing characteristics also include a viewing angle for each respective game participant to view an intended position for presenting a digital 3D object (e.g., a 2D image that is projected within the physical gaming suite in such a way so that some of the one or more game participants are able to view a 3D effect for the 2D image that begins at the intended position) within the physical gaming suite. Additional details regarding the intended position are provided below in reference to, e.g., operation 2206 of method 2200 (FIG. 22).

[0454] The method 2100 continues with the computing device determining (2104) a viewpoint that is based on at least some of the monitored viewing characteristics. In some embodiments, the viewpoint that is determined is a predicted common/optimal viewpoint that represents a common viewpoint for one or more of the game participants at some predetermined interval in the future (e.g., 1, 2, 3, 4, or 5 seconds). In this way, the computing device is able to use the monitored viewing characteristics to predict where game participants will be looking at the predetermined interval in the future and use those predictions to determine a common/optimal viewpoint that is forward-looking (additional details are provided below in reference to method 2200, FIG. 22).

[0455] In some embodiments, the computing device determines (2106) respective viewpoints for each of the one or more game participants based at least in part on the monitored viewing characteristics, and the computing device determines the viewpoint using a weighted average of respective viewpoints for two or more of the one or more game participants. In some embodiments, the weighted average is biased towards a respective game participant that is closest to a position (e.g., the intended position for viewing a digital 3D object, as discussed above) in the physical gaming suite at which the anamorphic image is to be provided.

[0456] In some embodiments, all of the monitored viewing characteristics are not used to determine the viewpoint (e.g., only a viewing angle for each game participant is used and other monitored viewing characteristics are not used) or, in other embodiments, some viewing characteristics from only a subset of the game participants are used (e.g., only a viewing angle for a subset of the game participants).

[0457] For example, as shown in FIG. 19A, anamorphic image 1920 is presented within the physical gaming suite 1900 for 3D viewing by both a first game participant associated with viewpoint 1910 and a second game participant associated with viewpoint 1912. In order to ensure, that both the first and second game participants will be able to view and appreciate a 3D effect for anamorphic image 1920, the computing device determines respective viewpoints for the first and second game participants (i.e., using monitored viewing characteristics for each game participant, the computing device determines that viewpoint 1910 is associated with the first game participant and that viewpoint 1912 is associated with the second game participant) a common/optimal viewpoint that takes into account viewpoint 1910 and viewpoint 1912.

[0458] In some embodiments, the computing device excludes (2108) at least one viewpoint for a third game participant from the weighted average, in accordance with a determination that a respective viewpoint for the third game participant does not meet predefined viewpoint criteria (e.g., including a criterion for distance away from a respective anamorphic image, a criterion for viewing angle of a respective anamorphic image, and other criterion that affect a game participant's ability to appreciate a 3D effect for a respective anamorphic image). For example, continuing the above example in reference to FIG. 19A, viewpoint 1914 for the third game participant is excluded in accordance with a determination that a viewing angle from viewpoint 1914 to the position at which anamorphic image 1920 is presented is too great.

[0459] The computing device generates (2110), based on the viewpoint determined in operation 2104, an anamorphic image (e.g., anamorphic image 1920, FIG. 19A) for presentation within the physical gaming suite. In some embodiments, the anamorphic image is not a stereoscopic image (2112) and, thus, users need not wear any external wearable device (e.g., glasses) to view anamorphic image 1920 in 19D (e.g., as shown in FIGS. 19B-19C). In some embodiments, the viewpoint is determined (2114) based at least in part on viewing characteristics associated with an active game participant (e.g., the first game participant associated with viewpoint 1910 is currently putting), and the anamorphic image is generated in respective to an input (e.g., the first game participant striking a golf ball with a putting motion) from the active game participant. For example, anamorphic image 1920 is generated so that it appears near the golf ball as the golf ball travels towards a target (e.g., flames that appear to follow the golf ball as it rolls).

[0460] In some embodiments, generating the anamorphic image includes selecting the anamorphic image based on a current number of game participants that will be viewing the anamorphic image, so that an appropriate image is selected so that most of the users will appreciate and enjoy the 3D effect (e.g., a larger image is selected and is placed closer to a back surface of the physical gaming suite, if more game participants are to view the image in 3D).

[0461] The computing device also provides (2116), to the one or more display devices, data to present the anamorphic within the physical gaming suite (e.g., the anamorphic image is presented near (e.g., over, on, or on top of) at least one physical object that is in the physical gaming suite). In some embodiments, the at least one physical object is a surface within the physical gaming suite (e.g., bottom surface 1904 and/or back surface 1902, FIG. 19A) or some other object within the suite, such as a chair, a hula hoop, or a golf club. In some embodiments, the data includes information identifying the intended position for viewing of the anamorphic image in 3D (as discussed above) and includes information identifying an anamorphic effect/technique to apply to the anamorphic image (e.g., to further stretch a stored image, change a brightness of the stored image, and other manipulations to improve how the game participants are able to perceive the anamorphic image in 3D).

[0462] In some embodiments, the bottom surface 1904 is substantially perpendicular to the back surface 1902. For example, the bottom surface 1904 is a playing surface (e.g., playing surface 104, FIGS. 1A-1B) on which a game is played by game participants (e.g., a putting game).

[0463] In some embodiments, monitoring the viewing characteristics (e.g., operation 2102) includes monitoring viewing characteristics for at least two game participants, determining the viewpoint (e.g., operation 2104) includes determining the viewpoint based on at least some of the monitored viewing characteristics for the at least two game participants, and providing the data to present the anamorphic image (e.g., operation 2116) includes providing data to present the anamorphic image for 3D viewing by the at least two game participants. In other words, the anamorphic image is generated and specifically tailored for 3D viewing by a subset of the game participants, so that each of them is able to appreciate and view a 3D effect for a respective anamorphic image (that is projected within the physical gaming suite) simultaneously.

[0464] In some embodiments, providing the data to present the anamorphic image includes providing a first portion of the data to a first display device (e.g., a first projector 105 positioned at a first location within the physical gaming suite 1900) and providing a second portion of the data to a second display device (e.g., a second projector 105 positioned at a second location within the physical gaming suite 1900) that is distinct from the first display device. In some embodiments, the first portion corresponds to data used to render the anamorphic image and the second portion corresponds to data used to render a shadow effect proximate to the anamorphic image. Stated another way, the shadow effect is used to enhance, improve, and sharpen the 3D effect produced by the display of the anamorphic image.

[0465] In some embodiments, the one or more game participants are not wearing any external wearable device for 3D viewing (2118) and the anamorphic image (when presented in the physical gaming suite) appears, to at least two of the one or more game participants, to have visual depth (i.e., the at least two game participants are able to perceive the anamorphic image in 3D, e.g., as shown in FIGS. 19B-19C for anamorphic image 1920 as viewed from viewpoints 1910 and 1912).

[0466] In some embodiments, the anamorphic image appears (when presented within the physical gaming suite) with different visual characteristics to at least two of the game participants (2120). For example, each game participant's perception of the 3D effect is slightly different, so that a first game participant sees slight distortions in order to ensure that other game participants also appreciate the 3D even if each participant views the 3D effect with some slight distortions. In some embodiments, the at least two game participants view a respective anamorphic image in 3D at slightly different positions within the physical gaming suite (as shown for anamorphic image 1920, FIGS. 19B-19C, with respect to viewpoints 1910 and 1912).

[0467] In some embodiments, the at least one physical object is a bottom surface (e.g., a deformable or tilt-able surface, such as those shown in FIGS. 19A-19C) of the physical gaming suite (2122), and providing the data includes providing data to present two or more component parts of the anamorphic image, such that a first component part is displayed on the bottom surface (e.g., first component part 1916-1 of anamorphic image 1916, FIG. 19A, is displayed on the back surface 1902) and a second component part is displayed on a back surface that is distinct from the bottom surface (e.g., second component part 1916-2 of anamorphic image 1916, FIG. 19A, is displayed on the bottom surface 1904). Stated another way, the anamorphic image is generated in such a way so that it will be stitched together by human eyesight (e.g., those viewing the anamorphic image from the respective viewpoints determined above) to produce a viable 3D effect for most of the game participants.

[0468] In some embodiments, generating the anamorphic image includes generating the anamorphic image based at least in part on both the viewpoint and based on a current topography of the bottom surface. In some embodiments, at least three distinct inputs are utilized in order to generate the anamorphic image, including (i) a common viewpoint that represents a viewpoint that allows two or more game participants to view a 3D effect for the anamorphic image; (ii) a current topography of the bottom surface; and (iii) an intended viewing position for the 3D effect for the anamorphic image (e.g., a starting position within the physical gaming suite at which the 3D effect is intended to begin). Other inputs may also be utilized, including measured levels of ambient light (e.g., operations 2130-2134 below), gaming events (game participants moving around, striking golf balls, and the like, as discussed herein), and desired shadowing effects (e.g., additional shadow to add to the anamorphic image in order to improve perception of the 3D effect). Additional details regarding these inputs are provided throughout this description.

[0469] In some embodiments, the computing device is also configured to generate and provide data to present multiple anamorphic images within the physical gaming suite 1900. As a few non-limiting examples: (i) a new anamorphic image can be generated for a game participant that was excluded from viewing the anamorphic image 1920 (as discussed as an example above); (ii) a new anamorphic image can be generated to distract a player during game play; (iii) a new anamorphic image can be generated in response to interactions with the anamorphic image while it is display in the physical gaming suite; and (iv) a new anamorphic image can be presented in response to movement of a game participant.

[0470] More specifically, as to (i), in accordance with the determination that the respective viewpoint for the first game participant does not meet predefined viewpoint criteria (e.g., operation 2108), the computing device determines a second viewpoint for at least the first game participant (e.g., the game participant whose viewpoint was excluded in conjunction with operation 2108) and generate a second anamorphic image based on the second viewpoint (e.g., anamorphic image 1918, FIG. 19A, for viewing by a game participant associated with viewpoint 1914); and the computing device provides, to the one or more display devices, data to present the second anamorphic image within the physical gaming suite (e.g., so that a 3D effect is viewable from at least the second viewpoint, as shown in FIG. 3D for anamorphic image 1918 as seen from viewpoint 1914).

[0471] As to (ii), in some embodiments, the computing device generates (2124) a second anamorphic image in accordance with a determination that an active game participant (e.g., game participant associated with viewpoint 1910, FIG. 19A) of the one or more gaming participants is about to strike a golf ball; and the computing device provides, to the one or more display devices, data to present the second anamorphic image. For example, the second anamorphic image is a distraction such as a gopher and the second anamorphic image that is presented before the active game participant strikes the golf ball (such as anamorphic image 2002 as shown in FIGS. 20A-20B).

[0472] As to (iii), in some embodiments, the computing device detects (2126) that a first game participant of the one or more game participants has interacted with a predefined portion of the anamorphic image (e.g., the anamorphic image is a beach ball that the user can push around the physical gaming suite). In response to detecting that the first game participant has interacted with the predefined portion of the anamorphic image, the computing device provides, to the one or more display devices, data to present the anamorphic image at a new position within the physical gaming suite that is distinct from a first position at which the anamorphic image was presented during the first game participant's detected interactions.

[0473] As to (iv), in some embodiments, the computing device detects (2128), using the one or more visual sensors, movement (e.g., the detected movement corresponds to a change in one or more of the viewing characteristics) of a first game participant of the one or more game participants within the physical gaming suite. In response to detecting the movement, the computing device determines an updated viewpoint (e.g., by selecting respective viewpoints for one or more game participants who will view a 3D effect for a second anamorphic image). Based on the updated viewpoint, the computing device generates a second anamorphic image for presentation within the physical gaming suite; and provides, to the one or more display devices, data to present the second anamorphic image near (or over, on, or on top of) at least one physical object (e.g., one or more surfaces within the physical gaming suite, such as a bottom surface and a back surface perpendicular to the bottom surface, a chair, a hula hoop, or any other object or surface within the suite) that is included within the gaming suite.

[0474] In some embodiments, the computing device performs one or more of example operations (i), (ii), (iii), and (iv) in sequence or in parallel, in order to generate and present multiple anamorphic images within the physical gaming suite 1900 simultaneously (e.g., as shown in FIGS. 19A-19D, three different anamorphic images are presented within the physical gaming suite 1900 at once).

[0475] In some embodiments, the computing device stores (2130), in the memory of the computing device, feedback from users regarding presentation of the anamorphic image within the physical gaming suite. For example, the feedback includes both quantitative (such as mood sensing feedback, whether users are looking at the displayed anamorphic image, how long a user remains focused on a displayed anamorphic image, and the like) and qualitative feedback (e.g., verbal reactions detected and stored by one or more microphones positioned within the physical gaming suite 1900, input from a caddy, input from an engineer, and the like).

[0476] In some embodiments, the stored feedback is used to improve (2132) presentation of the anamorphic image within the physical gaming suite (i.e., the anamorphic image is re-generated and re-presented within the physical gaming suite 1900 in accordance with a determination that presentation of the anamorphic image can be improved based on the stored feedback, and/or the stored feedback is used to improve future generations and presentations of the anamorphic image).

[0477] In some embodiments, the computing device measures (2134), using a light-sensing device that is in communication with the computing device, ambient light levels within the physical gaming suite; and re-generates the anamorphic image in response to changes in the measured ambient light levels within the physical gaming suite (or in accordance with a determination that the changes in the measured ambient light levels will affect presentation and ability to perceive a 3D effect for the anamorphic image).

[0478] Additional operations of method 2100 may be interchanged or added to include operations discussed below with respect to FIG. 22. For example, the method 2100 may include one or more operations of method 2200, such as operation 2206 in which an intended position for viewing a 3D object is also used to help determine how to generate and present an anamorphic object for 3D viewing.

[0479] It should be understood that the particular order in which the operations of method 2100 have been described is merely exemplary and is not intended to indicate that the described order is the only order in which the operations could be performed. One of ordinary skill in the art would recognize various ways to reorder the operations described herein.

[0480] FIG. 22 is a flowchart of a method of programmatically generating anamorphic images for presentation (and 3D viewing) in a physical gaming suite, in accordance with some embodiments.

[0481] In some embodiments, the method 2200 is performed by an electronic/computing device (e.g., system controller 114, FIG. 1A) and/or one or more components of the electronic device (e.g., an I/O subsystem, an operating system, etc.). In some embodiments, the method 2200 is governed by instructions that are stored in a non-transitory computer-readable storage medium and that are executed by one or more processors of a device, such as the one or more processors 202-1 of a management module 114-2 (FIG. 1D). For ease of explanation, the following describes method 2200 as performed by the system controller 114 (also referred to simply as the device). In some embodiments, with reference to FIG. 1D, the operations of method 2200 are performed by or use, at least in part, a surface modification module (e.g., surface modification module 222), a ball color detection module (e.g., ball color detection module 224), a ball path determining module (e.g., ball path determining module 226), a graphics rendering module (e.g., graphics rendering module 228), a game selection module (e.g., game selection module 230), user-specific putting data (e.g., user-specific putting data 218), a leaderboard updating module (e.g., leaderboard updating module 220), a visual characteristics monitoring module (e.g., visual characteristics monitoring module 250), and/or a viewpoint determining module (e.g., viewpoint determining module 252). Some operations in method 2200 are, optionally, combined and/or the order of some operations is, optionally, changed.

[0482] In some embodiments, a method 2200 optionally includes: receiving (2202), from one or more visual sensors (e.g., one or more cameras 103, FIGS. 1A-1B, that are positioned within a physical gaming suite 1900 of a sport simulation system 100), data about positions of one or more game participants in the physical gaming suite. The method optionally also includes tracking (2204), using the received data positions of the one or more game participants within the physical gaming suite. In some embodiments, the game participants are tracked in real-time as they stand, sit, or move within the physical gaming suite. In some embodiments, a combination image data, 3D triangulation techniques and active sensors (e.g., Kinect) are all used to provide raw data as inputs. In some embodiments, the raw data drives a state estimation engine capable of accurately localizing each player and predicting player motion several seconds into the future with a high degree of confidence.

[0483] In some embodiments, the method includes determining (2206) an intended position for viewing a 3D object within the physical gaming suite. For example, a beginning point is determined (such as beginning point a shown in FIGS. 20A-20G for a variety of example anamorphic images that are projected in 2D and viewable in 3D) at which the 3D object begins.

[0484] The method further includes: determining (2208), based at least in part of respective viewpoints for two or more of the game participants, an optimal viewpoint for viewing the 3D object within the physical gaming suite (e.g., a common viewpoint at which two or more of the game participants will be able to appreciate and view the 3D effect). In some embodiments, the computing device will a number of inputs while determining the optimal viewpoint, including: (1) the position of each player as determined above (e.g., in operations 2202-2204) and (2) the intended location on the playing surface of the 3D object to be displayed (e.g., operation 2206). The computing device then determines a subset of the most self-consistent viewpoints.

[0485] Game participants whose viewpoints are not within this inlier set will not be considered when computing/generating an anamorphic image for view-dependent 3D viewing within the physical gaming suite. If no two views are sufficiently consistent to produce a common 3D viewpoint, the system will select a current active player and generate the 3D object from the viewpoint of that player.

[0486] In some embodiments, selection of the subset of players to be included in determining a common 3D viewpoint will be based on minimizing the relative angles between the target 3D object (e.g., the 3D object that is viewable after rendering a 2D anamorphic image) and any two players within the inlier subset. From this, the computing device determines the optimal viewpoint by selecting a point in space that minimizes the angle with respect to the 3D object of any player in the inlier set with the optimal viewpoint. In some embodiments, a quadratic least squares model is used.

[0487] The method additionally includes: providing (2210), to one or more display devices (e.g., one or more projectors 105 positioned within the physical gaming suite, data to render 2D images (e.g., anamorphic images, such as the examples shown in FIGS. 19A-19D and 20A-20G) that are viewable in 3D by two or more of the game participants. In some embodiments, once a common viewpoint is determined (e.g., operation 2208), the computing device generates a virtual camera at the location of the common viewpoint and a virtual illumination source at some overhead location coincident with the room lighting. In some embodiments, a virtual light source and model of the 3D object will be used to generate a mathematical model of shadows on the playing surface. In some embodiments, the 3D object and associated shadows will be projected mathematically into the virtual camera view.

[0488] The resulting 2D image representation of the 3D virtual object and shadow will then be mapped mathematically onto the flat (or actuated) surface of the green by ray casting from the virtual camera. The resulting mapping will be communicated to the projection system and displayed directly on the green. The effect will be a perfectly rendered 2D projection of a 3D object with shadowing as seen from the optimal viewpoint. This virtual 3D object will become more distorted as the viewpoint moves away from the optimal viewing location.

[0489] In some embodiments, positions of each game participant are tracked in near real-time as described above. As play progresses, the computing device in some embodiments predicts optimal viewpoints for 3D rendering as described above based on the predicted locations of players. This reduces computational lag in both viewpoint determination and 3D rendering as the most likely optimal viewpoint several seconds into the future can be pre-computed and the associated 3D object pre-rendered.

[0490] Additional operations of method 2200 may be interchanged or added to include operations discussed above with respect to FIGS. 21A-21C (e.g., to generate second anamorphic images, re-generated anamorphic images based on changes in ambient light, and/or to modify presentation of an anamorphic object based on changes in a topography of a surface on which an anamorphic image is presented).

[0491] While golf is used as an example above to explain to help explain various aspects of some embodiments described herein, it should be appreciated that the systems and techniques disclosed herein can be used and/or adapted for any other sport or game VR experience. For example, the VR system can be used for playing other sports or games (e.g., snowboarding, skiing, surfing, laser tag, first-person shooters, or other similar games). In some embodiments, a snowboarder rides on a high speed rotating and deformable surface that replicates a downhill run and the related surface contour. In some embodiments, a mechanical system provides the snowboarder an immersive experience supported and enhanced by the 3D effects and other environmental simulation techniques described herein in order to provide a fully immersive VR experience. In some embodiments, projectors present images to place the snowboarder on any slope over the world and the images are projected onto the walls and the ceiling of the physical gaming or entertainment suite 1900. In some embodiments, the audio systems replicate sounds of rushing down the hill, wind in the trees, and/or crowd cheering as the snowboarder enter a finish line. In some embodiments, a simulated environment with cold, fog, moisture, and wind as the snowboarder speeds down the hill is provided using the techniques discussed above (e.g., via the environmental simulation devices 192, FIG. 1E).

[0492] In some embodiments, an immersive VR experience includes 2D images (e.g., one or more anamorphic images) presented for 3D viewing is provided for surfing. In some embodiments, the surfing simulator simulates surfing on a wave generator. In some embodiments, the projectors present views to immerse the surfer in an environment of any beach around the globe. In some embodiments, the audio system replicates an aural environment including the seagulls and the crowd cheering when the wave is taken all the way in, or when the surfer is found in a tube. In some embodiments, the environmental simulation devices 192 (FIG. 1E) replicate the heat and humidity for any beach location from Hawaii to California. In some embodiments, the projectors updates the views rapidly based upon the surfer's head and eye location using the high speed cameras and constantly adjusts the views for the associated viewpoint (to produce constantly changing and adapting 3D effects that move based on the surfer's current position).

[0493] In some embodiments, the 3D effects described herein are used to study history or take a tour in any city in the world from any time period. For example, the system is used for simulating a tourist walking through the streets of Paris in the 1800s. In some embodiments, the walk is simulated with a mechanical system that rotates and simulates motion forward. In some embodiments, the deformable surfaces of the physical gaming suite 1900 change to cobblestone. In some embodiments, the audio systems simulate sounds around the user of the environment they are immersed into. In some embodiments, the environmental simulation devices 192 (FIG. 1E) simulate the external environment and temperature to be of any location transported to in the world. In some embodiments, the projectors update the views rapidly based upon the tourist's head and eye location using the high speed cameras and constantly adjust the views for the associated viewpoint of either one tourist or up to four tourists (e.g., by presenting 2D images that are viewable in 3D by presenting the 2D images at an optimal viewpoint, as described above).

[0494] Although some of various drawings illustrate a number of logical stages in a particular order, stages that are not order dependent may be reordered and other stages may be combined or broken out. While some reordering or other groupings are specifically mentioned, others will be obvious to those of ordinary skill in the art, so the ordering and groupings presented herein are not an exhaustive list of alternatives. Moreover, it should be recognized that the stages could be implemented in hardware, firmware, software or any combination thereof.

[0495] For situations in which the systems discussed throughout this description collect information about users, the users may be provided with an opportunity to opt in/out of programs or features that may collect personal information (e.g., information about a user's preferences or usage of a smart device). In addition, in some embodiments, certain data may be anonymized in one or more ways before it is stored or used, so that personally identifiable information is removed. For example, a user's identity may be anonymized so that the personally identifiable information cannot be determined for or associated with the user, and so that user preferences or user interactions are generalized (for example, generalized based on user demographics) rather than associated with a particular user.

[0496] The foregoing description, for purpose of explanation, has been described with reference to specific embodiments. However, the illustrative discussions above are not intended to be exhaustive or to limit the scope of the claims to the precise forms disclosed. Many modifications and variations are possible in view of the above teachings. The embodiments were chosen in order to best explain the principles underlying the claims and their practical applications, to thereby enable others skilled in the art to best use the embodiments with various modifications as are suited to the particular uses contemplated.