Camera-Based System for Game Recognition

Abstract

A system for recognition of objects in a game, the system has a table with playing board for pucks on which the game is played, and reference markings distributed across the playing board that are used together with corner of the playing board in a surface mapping process. At least one area scan camera acquires images of the playing table during a game and reads fiducial marker of pucks that have the fiducial markers. The at least one area scan camera is arranged to cover the playing board and the reference markings. A processing unit processes the images acquired by the at least one camera.

Claims

1. A system for recognition of objects in a game, the system comprising: a table (100), comprising a playing board (110) for pucks (200) on which the game is played, the playing board (110) having sides and corners and defining a surface; reference markings (140) distributed across the playing board (110) that are used together with the corners of the playing board (110) in a process for surface mapping; at least one area scan camera (300) arranged to cover the playing board (300) and the reference markings (140) for acquiring images of the playing board (110) during a game; a processing unit (600) for processing images acquired by the at least one camera (300), wherein each puck (200) comprises a fiducial marker (210), and the at least one camera (300) reads the fiducial marker (210) of the pucks (200).

2. The system according to claim 1, wherein the processing unit (600) communicates with a user interface (500) for game participants to initiate and choose games.

3. The system according to claim 2, wherein the user interface (500) presents processed data from the processing unit (600) about the game.

4. The system according to claim 2, wherein the user interface (500) comprises a display (510).

5. The system according to claim 1, wherein the system comprises at least two area scan cameras (300).

6. The system according to claim 1, wherein the fiducial marker (210) on at least one puck (200) is a RUNE-tag comprising a plurality of circles distributed around an outer edge of the puck (200) or a puck cap configured to be arranged on the puck (200).

7. The system according to claim 1, wherein the playing board (110) comprises scoring regions (130) distributed across the playing board (110).

8. A method for executing steps for playing a game using a system according to claim 1, comprising the steps of: capturing at least one image of at least one puck (200) on a playing board (110) via at least one area scan camera (300); transmitting the captured images to a computer vision processing unit (600); recognizing the fiducial marker (210) of the at least one puck (200); receiving puck positions and orientations of the at least one puck from the computer vision processing unit (600); calculating a state and scoring of all pucks (200) on the table (100) based on the at least one puck positions or puck orientation.

9. The method according to claim 8, wherein the at least one area scan camera (300) captures a plurality of images of the puck (200) on the playing board (110) in order to determine when the puck (200) is in a steady state.

10. The method according to claim 9, comprising a step of identifying a trajectory or path of the puck (200) while the puck (200) moves across the playing board (110) from throwing the at least one puck (200) until it is in the steady state.

11. The method according to claim 9, comprising a step of identifying a spin of the puck (200) while moving by comparing the orientation of the fiducial mark (210) from throwing the at least one puck (200) until it is in the steady state.

12. The method according to claim 9, wherein an unwanted relative movement between the playing board (110) and the at least one camera (300) is detected by comparing reference markings (140) of the playing board (110) with an initial position or relation between the playing board (110) and the at least one camera (300).

13. The method according to claim 9, wherein an order of the game is communicated through a user interface (500).

14. A method for surface mapping of a system according to claim 1, comprising the steps of: capturing images of the table comprising corners and reference markings (140) using at least one camera (300) that is calibrated to account for lens distortion and image sensor parameters; matching the captured image with image templates of coordinates of corners and reference markings (140); computing a transformation of coordinates of the corners and reference markings (140) in the image; mapping the coordinates from the image with the coordinates of the image templates.

15. The method of claim 14, wherein the at least one camera is calibrated via capturing a plurality of images of a checkerboard while the camera and lens parameters stay constant.

16. The method according to claim 14, wherein a plurality of cameras (300) capture images of the table, the plurality of cameras having (300) overlapping views for generation of a full view of the playing board (110).

17. The method according to claim 14, comprising a step of selecting a game on a user interface (500).

18. The method according to claim 17, wherein a projector projects a board image on the table appropriate for the selected game.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0099] The above and further features of the invention are set forth in the following detailed description of a non-limiting exemplary embodiment given with reference to the accompanying drawings.

[0100] The invention will be further described below in connection with exemplary embodiments which are schematically shown in the drawings, wherein:

[0101] FIG. 1 shows a system for recognition of objects in a game.

[0102] FIG. 2 shows a top view of a playing surface with score markings, pucks and reference points.

[0103] FIGS. 3A-3C show different exemplary fiducial marker alternatives for various puck caps.

[0104] The following reference numbers and signs refer to the drawings:

TABLE-US-00001 TABLE 1 100 Table 110 Playing board 120 Crate 130 Score markings/scoring regions 140 Reference markings 200 Puck 210 Fiducial marker 300 Camera 400 Lighting fixture 500 Touch screen/User interface 510 Displays for game information and visualizations 600 Processing unit/computer

DETAILED DESCRIPTION

[0105] Various aspects of the disclosure are described more fully hereinafter with reference to the accompanying drawings. This disclosure may, however, be embodied in many different forms and should not be construed as limited to any specific structure or function presented throughout this disclosure. Rather, these aspects are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art. Based on the teachings herein one skilled in the art should appreciate that the scope of the disclosure is intended to cover any aspect of the disclosure disclosed herein, whether implemented independently of or combined with any other aspect of the disclosure. For example, an apparatus may be implemented, or a method may be practiced using any number of the aspects set forth herein. In addition, the scope of the disclosure is intended to cover such an apparatus or method which is practiced using other structure, functionality, or structure and functionality in addition to or other than the various aspects of the disclosure set forth herein. It should be understood that any aspect of the disclosure disclosed herein may be embodied by one or more elements of a claim.

[0106] The inventive embodiments will be further described in connection with exemplary embodiments which are schematically shown in the drawings.

[0107] FIG. 1 shows a system for recognition of objects in a game comprising a table or court 100 comprising a playing surface 110 on which the game is played. A puck 200 is shown on the table or court 100. A camera 300 is positioned above the table or court 100 in a position where it can capture images of the whole top surface of the table or court 100. FIG. 1 shows one camera, but it is not limited to camera, it might have more cameras, preferably two or three cameras. User interfaces 500 and monitors or projectors 510 are used for interaction between the game participant and a processing unit 600 controlling the course of the game. The game participants can choose what game set up to play and the course of the game will be communicated through monitors or projectors 510. Images will be captured throughout the game and processed in the control unit. The control unit calculates and sends information about scoring, course of the games etc. and communicates to the game participants through the monitors or projectors 510.

[0108] The key components of the system comprise the table 100 comprising a playing board 110 and a crate 120. Where the crate 120 is the structure surrounding the playing board 110 and receives the pucks falling off the playing board 110 together with the shuffleboard sand. The pucks 200 are tracked by one or more area scan cameras 300 such that their fiducial markers 210 can be read on the whole playing surface 110. The computer vision algorithms that process the image are executed on a processing unit 600. The processing unit 600 is operatively connected with the camera, processing images captured by the camera. Players can interact with the system through a user interface 500 preferably using a touchscreen. Based on the current game rules the scores and other visualizations are presented to the user on one or more screens, displays or projectors 510.

[0109] The user interface 500 is configured as an input device to detect game options selected by a user. The user selects or chooses between a set of pre-programmed game options from a menu allowing the user to e.g. select between different game types, levels, graphics, etc. and to initiate the start of the game. The monitor or projector 510 displays game information prior to and during the course of the game and at the end of the game. The game information displayed is dependent on where in the course of the game being at any given time, and the given information can span from information on when to throw a puck 200, who to throw a puck 200, the path of the puck 200, the end position of the puck 200, information on a previously thrown puck 200 being hit and follow the new path of puck 200 being hit, scorings during the game and final scoring results, the information to be communicated on said monitor or projector 510 may comprise an inexhaustible pre-programmed information to be communicated.

[0110] The top surface of the playing board 110 comprises a set of scoring segments or scoring lines. FIG. 2 shows a top view of a playing surface with score markings, 130 pucks and reference points or markings 140, the playing board 110 is not limited to such a setup of score markings 130 and reference points or markings 140 as shown in FIG. 2 but may have any given setup. The system is usually pre-programmed to a given setup of score markings 130 and reference points or markings 140. Within a given setup it may be possible to play a range of different pre-programmed games and/or the system may be reloaded or updated with a set of new gaming features or implement new rules or games. In FIG. 2 is shown a plurality of pucks 200 either shown during the course of the game, or it shows the final position when the last puck 200 is thrown and the game is finalized. The at least one camera 300 captures images during the game, starting with at least one image where the user has initiated the game through the user interface 500. During the game, at least one camera 300 captures images initiated when a puck is thrown and/or captures a plurality of images during the game.

[0111] FIGS. 3A-3C show different fiducial marker 210 alternatives for various puck caps. The different unique fiducial markers 210 may be placed directly on a top surface of a puck 200 or a puck 200 may be provided with a cap having such unique fiducial marker 210. FIG. 3A shows a puck cap with a typical RUNE-tag, where a plurality of white or any other given colour, i.e., black, red, green, etc. circles are alternated around the edge or circumference on a top surface of the puck 200. For individual recognition of each individual puck in a game, the plurality of circles may have its unique distribution alternated on each individual puck. The RUNE-tag is not limited to be marked by a plurality of circles, it may have shapes of squares, stars, etc. It is not limited to alternate around the edge or circumference of the top surface of the puck 200, but may find its position between the edge or circumference to the centre of the puck 200. When alternated around the edge or circumference, the centre of the puck 200 may comprise other information such as logos, colour identification in order for the user to visually identify his or her puck, etc. Or such information may be displayed on the edge or circumference in the case where the markers (circles) are arranged closer to the centre of the puck 200.

[0112] With markers alternated around the edge or circumference, a logo can be integrated into the fiducial marker.

[0113] FIG. 3B shows a puck with a ARTag marker and FIG. 3C shows a puck with an ARToolkit marker. The black square border in both ARTag and ARToolkit enables detection, but while ARToolkit uses image correlation to differentiate markers, ARTag relies in error-correcting binary codes and uses image correlation for marker identification. However, the small area in the centre of the puck does not allow for robust identification given the resolution and optics of cameras used for cost-effective setups.

[0114] To obtain sharp images of moving objects with a camera 300 a short exposure time is required. To make sure enough light hits the image sensor, artificial lighting 400 may be required. This way the fiducial marker 210 identity can be read while the puck 200 is moving. This allows for a smooth game experience for the players, by keeping the visualization on the monitors 510 synchronized with the real puck positions in real time.

[0115] When using a plurality of cameras 300, their digital images may be stitched together using the reference markings 140 for the surface mapping process.

[0116] In embodiments, to automate the software setup and increase robustness, the camera position may be determined by detecting certain features, such as corners or centre logos on the left or right of the table 100.

[0117] In embodiments, the cameras 300 can be used to detect gestures as a user interface during game play. When the user holds a puck, the system knows that it is held by a legitimate user. With a gesture such as a circle the system knows that the puck is not thrown and a menu can be projected on to the gaming surface. The user can move the puck to interact with the projected menu.

[0118] In embodiments, a projector can be used to project the game onto the playing surface. This takes away the need of monitors to show the game progression and will make the games more immersive.

[0119] In embodiments, smart pucks can have embedded LEDs and sensors that receive a command from the game machine, processing unit, computer or the like to determine what colour the LED light should have, and change colour based on game mode, which team is playing, etc. The LED-light is controlled by a microcontroller positioned within the smart puck. Using a built-in accelerometer or gyro, the smart puck can detect collisions and change brightness of the LED-light based on this.