APPARATUS, METHOD AND COMPUTER PROGRAM FOR TRACKING, CAPTURING AND OBSERVING SPORTS-RELATED EVENTS

Abstract

The present invention relates to an apparatus, method and computer program for collecting and processing digital information by computer vision and/or tracking as well as observing specific events during sports, in particular during the practice of racket sports, comprising: at least one camera pair for obtaining real image-related information in sports-related stereo, processor for processing the actual image-related information from the at least one camera, memory space containing stored comparison data for typical actions of a fictitious athlete and for typical movements and positions of a fictitious ball, the camera, processor and memory space all to each other for both storage and retrieval of digital information, the processor comparing the actual image-related information from the at least one camera with the corresponding stored comparison data from the memory space and categorizing the work image-related information.

Claims

1. An apparatus for collecting and processing digital information captured by means of computer vision and/or tracking, for the purpose of observing events during the practice of sport, in particular during the practice of racket sports, comprising: at least one pair of camera sensors for capturing actual image-related information in stereo related to a sport, a processor for processing the actual image-related information from the at least one pair of camera sensors, memory space containing stored comparison data for normally occurring actions of a fictitious athlete and for normally occurring movements and positions of a fictitious ball, wherein the camera sensors, the processor, and the memory space are all connected to each other for both storage and retrieval of digital information, and wherein the processor is configured to perform operations comprising comparing the actual image-related information from the at least one pair of camera sensors with the corresponding stored comparison data from the memory space and categorizing the actual image-related information, and circuitry and/or code configured to observe and interpret categorized information itself or to transmit image-based information for external interpretation.

2. The apparatus according to claim 1, wherein the at least one pair of camera sensors is synchronized at the pixel level.

3. The apparatus according to claim 1, wherein the processor utilizes image processing and data processing for tracking, computer vision, and machine learning in the acquisition, processing, comparison, and categorization of digital information.

4. The apparatus according to claim 1, wherein the processor is configured to select a certain section of the entire imaged area of the camera sensor as the priority area, based on similarity of the corresponding stored comparison data, and then process this area at a higher speed than surrounding areas.

5. The apparatus according to claim 1, wherein the processor is configured to select a particular section of the entire imaged area of the camera sensor as the priority area, based on similarity of the corresponding stored comparison data, and then process this area with higher resolution than surrounding areas.

6. The apparatus according to claim 1, wherein the at least one pair of camera sensors are adapted to retrieve information related to determining and timing at least one of the following: track or table position, player, racket and ball position.

7. The apparatus according to claim 1, wherein the processor is further connected to a communication means, which by means of a connected sound and/or light source, such as buzzer or diode, is adapted to signal whether the point of impact of a ball is outside the path limit and indicate the ball is out.

8. The apparatus according to claim 1, wherein the processor is further connected to a communication means, which is configured to transmits a digital signal containing information received by an application, in which observation and interpretation in, for example, a mobile telephone is done, in such a way that the user of the mobile telephone receives assistance to determine whether a certain ball bounced inside or outside a line, or get help with statistics about their own, their teammate's, their opponent's play and/or one about an arbitrary player.

9. The apparatus according to claim 1, wherein the at least one pair of camera sensors, the processor, and the memory space are all mounted in a housing configured to be placed adjacent to the at least a first power pole or to a referee's chair, the at least one pair of camera sensors being aimed in such placement at obtaining real image-related information related to the sport on at least part of a game plan.

10. The apparatus according to claim 1, wherein the at least one pair of camera sensors, the processor, and the memory space are all mounted in a housing configured to be placed in or in proximity to a net post or pole, the at least one pair of camera sensors when so placed being directed to obtain real image-related information related to the practice of sports on at least part of a playing field.

11. The apparatus according to claim 10, wherein the housing is at the same time a camera sensor cover which is designed as a hard shell in impact-resistant material, for protection against impacts, such as a direct ball hit, without adversely affecting the image quality, by obscuring parts of the image.

12. The apparatus according to claim 10, wherein the housing is at least functionally integrated in the net post/pole or the referee's chair.

13. The apparatus according to claim 10, wherein the housing is formed in one piece with the net post/pole or the referee's chair.

14. The apparatus according to claim 10, wherein the housing is configured to occupy less than 50% of the total area of the net post/pole and housing.

15. The apparatus according to claim 9, wherein the housing is at the same time a camera sensor cover which is designed as a hard shell in impact-resistant material, for protection against impacts, such as a direct ball hit, without adversely affecting the image quality, by obscuring parts of the image, for example by preventing sufficient light entry or otherwise breaking the light path.

16. A system, comprising: a first housing configured to be mounted to a first net pole of a racket-sport court; a first printed circuit board (PCB) inside the first housing, the first PCB being smaller than or equal to 65 millimeters (mm) tall by 20 mm wide; a first processor mounted to the first PCB; a first camera communicatively coupled to the first processor via the first PCB; and a second camera communicatively coupled to the first processor via the first PCB, wherein the first camera and the second camera are between 30 mm and 70 mm apart from one another and at least partially inside the first housing, and wherein the first PCB includes a first set of signal paths for a first synchronization signal by which the first camera is synchronized with the second camera; wherein the processor is configured to effectuate operations by one or more computer-vision systems to: detect a ball in a first set of synchronized images including images from both the first camera and the second camera; crop or reduce resolution of portions of the first set of synchronized images not including the ball; track movement of the ball; and determine whether the ball bounced in-bounds or out-of-bounds on the racket-sport court.

17. The system of claim 16, comprising: a second housing configured to be mounted to a second net pole of a racket-sport court; a second printed circuit board (PCB) inside the second housing, the second PCB being smaller than or equal to 65 millimeters (mm) tall by 20 mm wide; a second processor mounted to the second PCB; a third camera communicatively coupled to the second processor via the second PCB; and a fourth camera communicatively coupled to the second processor via the second PCB, wherein the third camera and the fourth camera are between 30 mm and 70 mm apart from one another and at least partially inside the second housing, and wherein the second PCB includes a second set of signal paths for a second synchronization signal by which the third camera is synchronized with the fourth camera; wherein the processor is configured to effectuate operations by one or more computer-vision systems to: detect the ball in a second set of synchronized images including images from both the third camera and the fourth camera; crop or reduce resolution of portions of the second set of synchronized images not including the ball; and track movement of the ball; and determine whether the ball bounced in-bounds or out-of-bounds on the racket-sport court.

18. The system of claim 16, wherein the one or more computer vision systems are executed at the racket-sport court.

19. The system of claim 16, comprising: the racket-sport court having a pair of net poles to which the first housing and the second housing are mounted.

20. A method comprising: obtaining, with a processor, a first set of synchronized images including images from both a first camera and a second camera, the images depicting overlapping portions of a racket-sport court having a boundary line; detecting, with the processor, a ball in the first set of synchronized images including images from both the first camera and the second camera; cropping or reducing resolution of portions of the first set of synchronized images not including the ball with the processor; tracking, with the processor, movement of the ball; and determining, with the processor, whether the ball bounced in-bounds or out-of-bounds on the racket-sport court relative to the boundary line.

Description

BRIEF DESCRIPTION OF DRAWINGS

[0015] FIG. 1 schematically shows different components that are included in the device and how they are connected to each other.

[0016] FIG. 2 shows the device in the open condition with special focus on how the housing is shaped and how the bottom and lid parts of the housing are attached, preferably screwed together.

[0017] FIG. 3 shows the bottom and lid of the device, illustrating the main printed circuit board and processor together with the data and network connection.

[0018] FIGS. 4 and 5 show the design, technical construction and dimensions of the main printed circuit board in top and bottom views, respectively.

[0019] FIGS. 6 and 7 show the design, technical construction and dimensions of the housing in top and bottom views, respectively.

DESCRIPTION OF EMBODIMENTS

[0020] For a description of various alternatives, examples and embodiments of the present invention, reference is made to the accompanying drawings.

[0021] FIG. 1 is a schematic sketch of the various components included in the device and how they are connected to each other. A printed circuit board (PCB) is located centrally in the device, and is typically rectangular in shape with the sides 65 mm?20 mm as dimensions. According to this particular embodiment, the mentioned dimensions are maximum dimensions, but for alternative embodiments, in particular with larger inner space, the dimensions may of course be altered in either directions. Mounted on this printed circuit board is a chip with a processor 20, from which a plurality of functions are controlled and controlled, such as camera control, data processing and synchronization of input and cooperating components. A USB connector 30 is provided at the main printed circuit board for connection to a power source, which power source can be both a mains connection and battery operation. Furthermore, one at least one camera module 40 is connected to the main printed circuit board, either directly or via a so-called flex-PCB connection 50, preferably including a first connector 55 for contacting the camera module or another sensor module.

[0022] According to an alternative embodiment of the device according to the present invention, a further camera module 60 is connected to the main printed circuit board. This too is then connected to the main printed circuit board via an additional flex-PCB connection 70 by means of a second connector 75 for different types of sensor modules. However, this second camera module is located at a distance from the main circuit board, so direct contact is not possible. The distance between the camera modules is preferably between 30 mm to a maximum of approximately 70 mm. The upper limit is due to practical limitations of separating the camera modules within the same installation, without the installation becoming awkward or too bulky, and thus incompatible with other dimensional limitations. Such limitations may for instance be applicable regulations in the context of competition for a particular racquet sport or other limiting or influencing factors to take into account. Needless to say, the mentioned distance may be varied both upwards and downwards, depending on the construction and space available, but also in dependence of other relevant restrictions.

[0023] As mentioned, the main circuit board can be used with both one and two camera modules. In addition, two camera modules from different main circuit boards can work together, which is made possible by synchronizing the various components. According to this embodiment, there is also a pin 80 (pin for camera sync) provided for the purpose on the main circuit board for clocking different units, i.e. synchronize different input devices in time, such as in the current case the two input camera modules. This allows the camera modules to be placed at longer distances from each other, whereby synchronization of the camera modules and the data processing from them is still possible, either via network, power or other cables or wirelessly via Wi-Fi connections or any other applicable communication protocol. A typical installation with cooperating camera modules which belong to different main printed circuit boards is in installations of equipment according to the invention in net posts/poles on both sides of the net, for example on a tennis court. Another advantage of having synchronized and cooperating camera modules at a relatively large distance from each other is that the precision can be improved by large triangulation angles, and that the combined field of view of the two camera modules becomes significantly larger than the corresponding field of view from two closer cameras, for example controlled from the same circuit board. It should also be mentioned in this context that synchronization of different camera modules with respect to time is not necessary for the equipment according to the invention to function. However, the synchronization can have an important functional significance for the invention, as the precision and thus the reliability of the equipment can be further improved by controlling the camera modules to cooperate in a time-synchronized manner.

[0024] According to yet another alternative embodiment of the invention, it is possible, as an alternative to one or two camera pairs operating in stereo, instead to connect four or more single cameras operating individually. In other words, utilization of mono units instead of stereo units would be both feasible and advantageous, whereby system flexibility can be even further enhanced.

[0025] FIG. 2 shows the device in the open condition with special focus on how the housing is shaped and how the bottom and lid of the housing are attached, preferably screwed together. According to a preferred embodiment, the housing is a camera sensor cover, which is designed as a hard shell in impact-resistant material, for example hard plastic, composite material or metal. This is to protect against impacts, such as a direct ball hit, without adversely affecting the image quality, such as by at least partially obscuring the image, preventing sufficient light entry or otherwise interrupting the light path or otherwise obstructing optimal image capture.

[0026] In addition to what has been mentioned previously, it is also possible to activate and control the device by voice control to, for example, only receive support from the invention to determine whether a ball bounced on or off a line in doubtful cases.

[0027] The dimensions in the drawings are stated to suit tennis and the dimensions that a regulated net post/pole in tennis must comply with. The dimensions are intended to be adapted to the dimensions and designs that other applicable racket sports allow. The drawings are thus to be regarded as schematic and intended to be adapted to each specific racket sport, including its specific equipment and rule collections, in which the device may be used.

[0028] Stored comparison data with which acquired information is intended to be compared with may consist of information and associated logic that interprets and classifies an object's appearance and movement pattern.

[0029] According to the present invention, the imaging properties of the camera sensor can be varied in different ways. An example previously mentioned is cropping, where an algorithm is used to select a smaller area of the camera sensor based on image information, which is analysed in detail by image processing while the rest of the camera sensor field is left completely without analysis or with only a more rudimentary analysis.

[0030] The resolution of the camera sensor can also be varied, whereby the same algorithm as above or a similar algorithm is used to treat a certain part of the area of the camera sensor in high resolution while another area is treated in lower resolution.

[0031] In both of the above cases, i.e. cropping and resolution, the purpose is to treat the most interesting and informative parts of the camera sensor area at high speed, while other areas are prioritized down. In this way, a good quality and precision can be achieved and maintained over time, even though the components used according to the invention can show significantly lower performance and price than other system components, which could be alternatives for creating comparable results in terms of precision and speed, but as previously discussed, must be excluded for purely practical reasons.

[0032] To further clarify what is meant by compliance with the rule, the term compliant means that with a typical net post/pole including equipment/cover according to the invention is still within the rules for what is the maximum permitted size for a tennis post (according to national and international tennis rules).

[0033] Functional compliance means that by being compliant, the net post/pole, even with equipment/casing, still achieves the same functional purpose (with regard to the net post's/pole's function in the game, the net post's/pole's location on the court, and the possibility to adjust the tension of the net and finally to mount/dismount the net post/pole and the net).