APPARATUS, METHOD AND COMPUTER PROGRAM FOR TRACKING, CAPTURING AND OBSERVING SPORTS-RELATED EVENTS

Abstract

The present invention relates to an apparatus, method and computer program for collecting and processing digital information by computer vision and/or tracking as well as observing specific events during sports, in particular during the practice of racket sports, comprising: at least one camera pair for obtaining real image-related information in sports-related stereo, processor for processing the actual image-related information from the at least one camera, memory space containing stored comparison data for typical actions of a fictitious athlete and for typical movements and positions of a fictitious ball, the camera, processor and memory space all to each other for both storage and retrieval of digital information, the processor comparing the actual image-related information from the at least one camera with the corresponding stored comparison data from the memory space and categorizing the work image-related information.

Claims

1. An apparatus for collecting and processing digital information by computer vision, for the purpose of observing events during a sporting event, the apparatus comprising: at least one camera sensor for capturing actual image-related information related to a sport, a processor configured to process actual image-related information from the at least one camera sensor, memory space containing stored information, wherein at least one camera sensor, the processor, and the memory space are connected to each other for exchange of digital information, and wherein the processor is configured to perform operations comprising analysing the actual image-related information from the at least one camera sensor with the stored information from the memory space and categorizing the actual image-related information, and circuitry and/or code configured to observe and interpret categorized information itself or to transmit image-based information for external interpretation.

2. The apparatus according to claim 1, comprising only one camera sensor.

3. The apparatus according to claim 1, wherein the processor is configured to use image processing and data processing for tracking, computer vision, and machine learning in acquisition, processing, comparison, and categorization of digital information.

4. The apparatus according to claim 1, wherein the processor is configured to select a subsection of an imaged area of the camera sensor based on similarity of corresponding stored information, and then process the subsection area at a higher speed than at least some other areas of the imaged area.

5. The apparatus according to claim 1, wherein the processor is configured to select a subsection of an imaged area of the camera sensor based on similarity of at least part of the subsection to corresponding stored information, and then process the subsection with higher resolution than other areas of the imaged area.

6. The apparatus according to claim 1, wherein the at least one camera sensor is adapted to obtain information related to determining and timing at least one of the following: track or table position, player, racket and ball position.

7. The apparatus according to claim 1, wherein the processor is further connected to a sound and/or light source configured to signal that a place of impact of a ball sensed with the camera sensor is outside a spatial limit.

8. The apparatus according to claim 1, wherein the processor is further connected to a network interface that is configured to transmit information to be received by an application in which observation and interpretation in a computing device is done in such a way that a user of the computing device receives assistance to determine whether a ball bounced inside or outside a line, or get help with statistics about their own, their teammate's, their opponent's play and/or one about an arbitrary player.

9. The apparatus according to claim 1, wherein the at least one camera sensor, the processor, and the memory space are all mounted in a housing configured to be placed adjacent to, or integrated with, a pole or to a referee's chair.

10. The apparatus according to claim 1, wherein the stored information comprises representations of actions of a fictitious athlete and movements and positions of a fictitious ball; and wherein the at least one camera sensor, the processor, and the memory space are all mounted in a housing configured to be integrated with or mounted to a post or pole, the at least one camera sensor when so placed being configured to obtain real image-related information related to a sporting event on at least part of a playing field or court, and wherein the actions of the fictious athlete are actions in the sport and the movements and positions of the fictious ball are movements and positions in the sport.

11. The apparatus of claim 1, comprising: four housings, a first housing among the four having the at least one camera sensor, the first housing having only one camera sensor, each of the other three housings also having only one respective camera sensor.

12. The apparatus according to claim 11, comprising a court or playing field with a plurality of structures to which the four housings are mounted to or integrated with.

13. The apparatus according to claim 1, comprising a housing in which the at least one camera sensor is disposed, wherein the housing is integrated with a net post or pole or a referee's chair.

14. The apparatus according to claim 1, comprising a housing in which the at least one camera sensor is disposed, wherein the housing is configured to be integrated with or mounted to a net post or pole associated with the sport and to be compliant with rules of the sport.

15. The apparatus according to claim 1, comprising only one camera sensor, the only one camera sensor being the at least one camera sensor, and wherein the only one camera sensor is configured to image a stereoscopic view of a part of a court or playing field of the sport, wherein the processor is configured to determine a position of a ball relative to a boundary of the court or playing field by triangulating the position based on the stereoscopic view.

16. A system, comprising: a first housing configured to be mounted to, or integrated into, a first structure that is part of a playing field or court of a sport, the first housing having a size and shape configured to be compliant with rules of the sport; a first printed circuit board (PCB) inside the first housing; a first processor mounted to the first PCB; and a first camera sensor communicatively coupled to the first processor via the first PCB; wherein the first processor is configured to effectuate operations by one or more computer-vision systems to: detect a ball in a first set of images from the first camera sensor; crop or reduce resolution of portions of the first set of images not including the ball; track movement of the ball; and determine whether the ball is in-bounds or out-of-bounds of the playing field or court.

17. The system of claim 16, comprising: a second housing configured to be mounted to, or integrated into, a second structure that is part of the playing field or court of the sport, the second housing having a size and shape configured to be compliant with rules of the sport; a second printed circuit board (PCB) inside the second housing; a second processor mounted to the second PCB; and a second camera sensor communicatively coupled to the second processor via the second PCB; wherein the second processor is configured to effectuate operations by one or more computer-vision systems to: detect the ball in a second set of images including images from the second camera sensor; crop or reduce resolution of portions of the second set of images not including the ball; track movement of the ball; and determine whether the ball is in-bounds or out-of-bounds of the playing field or court.

18. The system of claim 16, wherein: the one or more computer-vision systems are executed at the playing field or court of the sport, and wherein the first camera sensor is the only camera sensor in the first housing; the sport is tennis; the first camera sensor is configured to image a stereoscopic view of a part of a tennis court, wherein the first processor is configured to determine whether the ball is in-bounds or out-of-bounds of the court by triangulating a position of the ball based on the stereoscopic view; and the first processor is configured to analyze different parts of an area of the first camera sensor independently.

19. A method comprising: obtaining, with two or more processors, a first set of images including images from both a first camera sensor and a second camera sensor, the images depicting overlapping portions of a court or playing field of a sport having a boundary line, wherein the first camera sensor and the second camera sensor are in different housings positioned at different sides of the court or playing field; detecting, with at least some of the two or more processors, a ball in at least some of the first set of images; tracking, with at least some of the two or more processors, movement of the ball; and determining, with at least some of the two or more processors, whether the ball is in-bounds or out-of-bounds on the court or playing field relative to the boundary line.

20. The method of claim 19, comprising: transmitting data to a computing device executing an application in which a result of the determination is displayed.

Description

BRIEF DESCRIPTION OF DRAWINGS

[0016] FIG. 1 schematically shows different components that are included in the device and how they are connected to each other.

[0017] FIG. 2 shows the device in the open condition with special focus on how the housing is shaped and how the bottom and lid parts of the housing are attached, preferably screwed together.

[0018] FIG. 3 shows the bottom and lid of the device, illustrating the main printed circuit board and processor together with the data and network connection.

[0019] FIGS. 4 and 5 show the design, technical construction and dimensions of the main printed circuit board in top and bottom views, respectively.

[0020] FIGS. 6 and 7 show the design, technical construction and dimensions of the housing in top and bottom views, respectively.

DESCRIPTION OF EMBODIMENTS

[0021] For a description of various alternatives, examples and embodiments of the present invention, reference is made to the accompanying drawings.

[0022] FIG. 1 is a schematic sketch of the various components included in the device and how they are connected to each other. A printed circuit board (PCB) is located centrally in the device, and is typically rectangular in shape with the sides 65 mm20 mm as dimensions. According to this particular embodiment, the mentioned dimensions are maximum dimensions, but for alternative embodiments, in particular with larger inner space, the dimensions may of course be altered in either directions. Mounted on this printed circuit board is a chip with a processor 20, from which a plurality of functions are controlled and controlled, such as camera control, data processing and synchronization of input and cooperating components. A USB connector 30 is provided at the main printed circuit board for connection to a power source, which power source can be both a mains connection and battery operation. Furthermore, one at least one camera module 40 is connected to the main printed circuit board, either directly or via a so-called flex-PCB connection 50, preferably including a first connector 55 for contacting the camera module or another sensor module.

[0023] According to an alternative embodiment of the device according to the present invention, a further camera module 60 is connected to the main printed circuit board. This too is then connected to the main printed circuit board via an additional flex-PCB connection 70 by means of a second connector 75 for different types of sensor modules. However, this second camera module is located at a distance from the main circuit board, so direct contact is not possible. The distance between the camera modules is preferably between 30 mm to a maximum of approximately 70 mm. The upper limit is due to practical limitations of separating the camera modules within the same installation, without the installation becoming awkward or too bulky, and thus incompatible with other dimensional limitations. Such limitations may for instance be applicable regulations in the context of competition for a particular racquet sport or other limiting or influencing factors to take into account. Needless to say, the mentioned distance may be varied both upwards and downwards, depending on the construction and space available, but also in dependence of other relevant restrictions.

[0024] As mentioned, the main circuit board can be used with both one and two camera modules. In addition, two camera modules from different main circuit boards can work together, which is made possible by synchronizing the various components. According to this embodiment, there is also a pin 80 (pin for camera sync) provided for the purpose on the main circuit board for clocking different units, i.e. synchronize different input devices in time, such as in the current case the two input camera modules. This allows the camera modules to be placed at longer distances from each other, whereby synchronization of the camera modules and the data processing from them is still possible, either via network, power or other cables or wirelessly via Wi-Fi connections or any other applicable communication protocol. A typical installation with cooperating camera modules which belong to different main printed circuit boards is in installations of equipment according to the invention in net posts/poles on both sides of the net, for example on a tennis court. Another advantage of having synchronized and cooperating camera modules at a relatively large distance from each other is that the precision can be improved by large triangulation angles, and that the combined field of view of the two camera modules becomes significantly larger than the corresponding field of view from two closer cameras, for example controlled from the same circuit board. It should also be mentioned in this context that synchronization of different camera modules with respect to time is not necessary for the equipment according to the invention to function. However, the synchronization can have an important functional significance for the invention, as the precision and thus the reliability of the equipment can be further improved by controlling the camera modules to cooperate in a time-synchronized manner.

[0025] According to yet another alternative embodiment of the invention, it is possible, as an alternative to one or two camera pairs operating in stereo, instead to connect four or more single cameras operating individually. In other words, utilization of mono units instead of stereo units would be both feasible and advantageous, whereby system flexibility can be even further enhanced.

[0026] FIG. 2 shows the device in the open condition with special focus on how the housing is shaped and how the bottom and lid of the housing are attached, preferably screwed together. According to a preferred embodiment, the housing is a camera sensor cover, which is designed as a hard shell in impact-resistant material, for example hard plastic, composite material or metal. This is to protect against impacts, such as a direct ball hit, without adversely affecting the image quality, such as by at least partially obscuring the image, preventing sufficient light entry or otherwise interrupting the light path or otherwise obstructing optimal image capture.

[0027] In addition to what has been mentioned previously, it is also possible to activate and control the device by voice control to, for example, only receive support from the invention to determine whether a ball bounced on or off a line in doubtful cases.

[0028] The dimensions in the drawings are stated to suit tennis and the dimensions that a regulated net post/pole in tennis must comply with. The dimensions are intended to be adapted to the dimensions and designs that other applicable racket sports allow. The drawings are thus to be regarded as schematic and intended to be adapted to each specific racket sport, including its specific equipment and rule collections, in which the device may be used.

[0029] Stored comparison data with which acquired information is intended to be compared with may consist of information and associated logic that interprets and classifies an object's appearance and movement pattern.

[0030] According to the present invention, the imaging properties of the camera sensor can be varied in different ways. An example previously mentioned is cropping, where an algorithm is used to select a smaller area of the camera sensor based on image information, which is analysed in detail by image processing while the rest of the camera sensor field is left completely without analysis or with only a more rudimentary analysis.

[0031] The resolution of the camera sensor can also be varied, whereby the same algorithm as above or a similar algorithm is used to treat a certain part of the area of the camera sensor in high resolution while another area is treated in lower resolution.

[0032] In both of the above cases, i.e. cropping and resolution, the purpose is to treat the most interesting and informative parts of the camera sensor area at high speed, while other areas are prioritized down. In this way, a good quality and precision can be achieved and maintained over time, even though the components used according to the invention can show significantly lower performance and price than other system components, which could be alternatives for creating comparable results in terms of precision and speed, but as previously discussed, must be excluded for purely practical reasons.

[0033] To further clarify what is meant by compliance with the rule, the term compliant means that with a typical net post/pole including equipment/cover according to the invention is still within the rules for what is the maximum permitted size for a tennis post (according to national and international tennis rules).

[0034] Functional compliance means that by being compliant, the net post/pole, even with equipment/casing, still achieves the same functional purpose (with regard to the net post's/pole's function in the game, the net post's/pole's location on the court, and the possibility to adjust the tension of the net and finally to mount/dismount the net post/pole and the net).