Jump Shot and Athletic Activity Analysis System
20220215689 · 2022-07-07
Inventors
Cpc classification
G16H20/30
PHYSICS
H04M1/72403
ELECTRICITY
A63B2225/50
HUMAN NECESSITIES
A63B2220/05
HUMAN NECESSITIES
A63B2225/20
HUMAN NECESSITIES
G06V40/23
PHYSICS
International classification
G06T7/246
PHYSICS
G06V10/75
PHYSICS
G09B19/00
PHYSICS
G16H20/30
PHYSICS
Abstract
Techniques are described where a computer or mobile device performs video analysis of a person performing a basketball jump shot. The computer or mobile device performs video analysis to (1) determine whether the basketball jump shot resulted in the basketball going through a hoop as a made shot, (2) determine metric data for the basketball jump shot based on a plurality of pixels within a plurality of the frames of the video data, and (3) repeat the made shot and metric data determinations for a plurality of basketball jump shots by the person to thereby track the made shots and the metric data for those shots. The metric data may include knee bend data, release point data, and/or arc angle data for the shots.
Claims
1. A computer program product for video analysis of a basketball jump shot, the computer program product comprising: a plurality of processor-executable instructions configured for processing video data, the video data comprising a plurality of frames of a person performing the basketball jump shot with a basketball, each frame comprising a plurality of pixels, the instructions being resident on a non-transitory computer-readable storage medium and being configured, upon execution by a processor, to: perform pixel processing on a plurality of the frames of the video data to (1) determine whether the basketball jump shot resulted in the basketball going through a hoop as a made shot based on pixels within one or more frames of the video data that depicts the basketball relative to the hoop, (2) determine metric data for the basketball jump shot, wherein the metric data comprises at least two members of the group consisting of (i) data relating to a knee bend for the basketball jump shot based on pixels within one or more frames of the video data that depict the person in a base phase of the basketball jump shot, (ii) data relating to a release point for the basketball jump shot based on pixels within one or more frames of the video data that depict the person in a release phase of the basketball jump shot, and (iii) data relating to an arc angle for the basketball jump shot based on pixels within a plurality of frames of the video data that depict the basketball in flight; and repeat the pixel processing for a plurality of basketball jump shots by the person to thereby track the made shots and the metric data for the plurality of basketball jump shots by the person.
2. The computer program product of claim 1, wherein metrics data comprises the knee bend data and the release point data.
3. The computer program product of claim 2, wherein the release point data comprises data indicative of how quickly the person released the basketball for the basketball jump shots.
4. The computer program product of claim 2, wherein the release point data comprises data indicative of a release pinnacle for the basketball jump shots.
5. The computer program product of claim 2, wherein the metrics data further comprises the arc angle data.
6. The computer program product of claim 1, wherein the metrics data comprises the knee bend data and the arc angle data.
7. The computer program product of claim 1, wherein the metrics data comprises the release point data and the arc angle data.
8. The computer program product of claim 1, wherein the plurality of processor-executable instructions are further configured to: compare body positions of the person in a plurality of the frames to reference skeletons that represent a fundamental basketball jump shot in the release phase and the base phase; determine deviations for the basketball jump shots from the fundamental basketball jump shot based on the comparisons; and track the determined deviations in a database.
9. The computer program product of claim 8, wherein the plurality of processor-executable instructions are further configured to: correlate the determined deviations with the made shot data to identify a plurality of the determined deviations that are outcome-determinative.
10. The computer program product of claim 1, wherein the plurality of processor-executable instructions are embodied by a mobile application for download onto a smart phone or tablet computer.
11. The computer program product of claim 10, wherein the mobile application is configured to obtain the video data from a camera on the smart phone or tablet computer.
12. A method for video analysis of a basketball jump shot, the method comprising: processing video data by a processor, the video data comprising a plurality of frames of a person performing the basketball jump shot with a basketball, each frame comprising a plurality of pixels, wherein the video data processing step comprises: the processor determining whether the basketball jump shot resulted in the basketball going through a hoop as a made shot based on a plurality of pixels within one or more frames of the video data that depicts the basketball relative to the hoop; the processor determining metric data for the basketball jump shot based on a plurality of pixels within a plurality of the frames of the video data, wherein the metric data comprises at least two members of the group consisting of (i) data relating to a knee bend for the basketball jump shot based on pixels within one or more frames of the video data that depict the person in a base phase of the basketball jump shot, (ii) data relating to a release point for the basketball jump shot based on pixels within one or more frames of the video data that depict the person in a release phase of the basketball jump shot, and (iii) data relating to an arc angle for the basketball jump shot based on pixels within a plurality of frames of the video data that depict the basketball in flight; and repeating the made shot determining step and the metric data determining step for a plurality of basketball jump shots by the person to thereby track the made shots and the metric data for the plurality of basketball jump shots by the person.
13. The method of claim 12 wherein the processor comprises a plurality of processors.
14. A system for video analysis of a basketball jump shot, the system comprising: a processor configured for processing video data, the video data comprising a plurality of frames of a person performing the basketball jump shot with a basketball, each frame comprising a plurality of pixels, wherein the processor is configured to: determine whether the basketball jump shot resulted in the basketball going through a hoop as a made shot based on a plurality of pixels within one or more frames of the video data that depicts the basketball relative to the hoop; determine metric data for the basketball jump shot based on a plurality of pixels within a plurality of the frames of the video data, wherein the metric data comprises at least two members of the group consisting of (i) data relating to a knee bend for the basketball jump shot based on pixels within one or more frames of the video data that depict the person in a base phase of the basketball jump shot, (ii) data relating to a release point for the basketball jump shot based on pixels within one or more frames of the video data that depict the person in a release phase of the basketball jump shot, and (iii) data relating to an arc angle for the basketball jump shot based on pixels within a plurality of frames of the video data that depict the basketball in flight; and repeat the made shot determination and the metric data determination for a plurality of basketball jump shots by the person to thereby track the made shots and the metric data for the plurality of basketball jump shots by the person.
15. The system of claim 14, further comprising: a camera for cooperation with the processor, the camera configured to generate the video data.
16. The system of claim 15, wherein the processor and the camera are part of a mobile device.
17. The system of claim 16, wherein the mobile device comprises a smart phone.
18. The system of claim 14, wherein the processor comprises a plurality of processors.
19. The system of claim 14, wherein the processor is part of a server.
20. The system of claim 14, wherein the metric data comprises the knee bend data and the release point data.
21. The system of claim 14, wherein the metric data comprises the release point data and the arc angle data.
22. The system of claim 14, wherein the metric data comprises the knee bend data and the arc angle data.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
[0019] The accompanying drawings, which are incorporated in and form a part of the specification, illustrate the embodiments of the present invention and together with the description, serve to explain the principles of the invention. In the drawings:
[0020]
[0021]
[0022]
[0023]
[0024]
[0025]
[0026]
[0027]
[0028]
[0029]
[0030]
[0031]
[0032]
[0033]
[0034]
DETAILED DESCRIPTION OF EXAMPLE EMBODIMENTS
[0035] Referring to the accompanying drawings in which like reference numbers indicate like elements,
[0036] The mobile device 102 may be a smart phone (e.g., an iPhone, a Google Android device, a BlackBerry device, etc.), tablet computer (e.g., an iPad), a laptop computer (e.g. MacBook) or the like. Furthermore, the mobile device 102 may be a position tracking-enabled mobile device. That is, the mobile device 102 may be configured to track its geographic position and communicate data regarding same to other computing devices (e.g., the remote server 106). The mobile device preferably employs a touchscreen or the like for interacting with a user. However, it should be understood that any of a variety of data display techniques and data input techniques could be employed by the mobile device. For example, to receive inputs from a user, the mobile device need not necessarily employ a touchscreen—it could also or alternatively employ a keyboard or other mechanisms. The mobile device 102 also comprises a video camera for capturing photos and videos of the player's jump shot 108.
[0037]
[0038]
The instructions may further include instructions defining a control program 254. The control program may be configured to provide the primary intelligence for the mobile application 250, including orchestrating the data outgoing to and incoming from the I/O programs 256 (e.g., determining which GUI screens 252 are to be presented to the user).
[0044] The network 104 may be any data communications network capable of supporting communications between the remote server 106 and the mobile device 102, wherein at least a portion of the data communication is wireless data communication as shown in
[0045] The remote server 106 may be a server or collection of servers that are configured to support video processing as described herein. The remote server 106 may perform other activities other than or in addition to video analysis, such as tracking improvements in a player's jump shot 108, storing tutorial and training videos, storing training games, generating a training schedule including training activities to be performed on each day, sharing training results via social media, storing all the videos taken of the player's jump shot 108, connecting with coaches, providing a community where a player may upload questions about training, or any other web-based service to help a player improve their training and jump shot 108. The remote server 106 may be operated by sports training company. However, it should be understood that other entities may operate the system.
[0046]
[0047] While basketball has been and will be the focus of the disclosure, the video analysis features taught by the present disclosure can be applied to any sport or physical activity.
[0048] Either the mobile device 102 processor 200 or the remote server 106 processor 300 performs video analysis of a player's jump shot 108 based on an reference jump shot skeleton. If the remote server 106 performs video analysis, the mobile device 102 transmits the video files of the jump shot to the remote server 106. If the mobile device 102 performs video analysis, the mobile device may still communicate with the remote server 106 by requesting the remote server to store training results, training videos, or other data, in the database 308.
[0049]
[0050]
[0051] The skeleton 400, 500 illustrated in
[0052] The side view angle may capture ball flight and also whether the ball went through the hoop. The front view skeleton 400 may have the same number of phases as the side view skeleton 500, but the front view skeleton 400 is different than the side view skeleton 500 because of the camera's 206 perspective.
[0053] The reference jump shot skeleton 400, 500 may comprise joints and connecting lines between the joints. The reference jump shot skeletons 400, 500 should roughly resemble a human body taking a jump shot, and the joints of the reference jump shot skeletons 400, 500 should generally overlap the joints of the player being filmed. Referring to
[0054] In another embodiment, the computer performing video analysis 102 or 106 may analyze a frame of video and find joints, limbs, or body positions of the player 430. The computer performing video analysis 102 or 106 may determine joint locations and body positions by determining the edges of a player's body and the size of the player's body. The computer performing video analysis 102 or 106 may quantify the size of the player 430 and also define coordinates for certain edges of the player 430 in the video. Based on the size and edge coordinates, the computer performing video analysis 102 or 106 may determine joint coordinates in the image. Coordinates may be pixel locations, generated X-Y plane values, or any other generated image coordinate system. The computer performing video analysis 102 or 106 has a basic understanding of the shape of a human and may determine shoulder joints from elbow joints using the edge coordinates and the general shape understanding. For example, if a first edge coordinate is located relatively closely to a second edge coordinate along the X-axis, the computer may determine an arm or a leg. The computer may scan a number of lines along the X-axis searching for known areas of the body. After finding the joints and limbs, the computer performing video analysis 102 or 106 may generate a player skeleton that actually represents the player's body positions in the video. The player skeleton may be compared to the reference skeleton on an X-Y plane to find deviations from the skeleton 400, 500. In places where joints or connecting lines of the reference skeleton 400, 500 substantially deviate from the player skeleton, the computer performing video analysis 102 or 106 may determine deviations from the skeleton 400, 500 in the jump shot. The computer performing video analysis 102 or 106 may choose either method of video analysis.
[0055] The skeleton 400, 500 has a finite number of stages.
[0056] Basketball players differ in height, weight, and gender, so the skeleton 400, 500 may stretch or constrict based on a player's size. In order to account for a basketball player's size, a user of the mobile device 102 may input size measurements before capturing video so that the computer performing video analysis 102 or 106 may generate a skeleton roughly matching the basketball player's 430 size. These body measurements may be general or specific. For example, a general body measurement may include the basketball player's 430 height, weight, and gender. Based on the height, weight, and gender, the computer performing video analysis 102 or 106 may generate a skeleton that roughly matches what an average person having those height and weight attributes should look like. For example, if a player is very tall, the computer performing video analysis 102 or 106 may stretch the connecting lines between joints representing the legs, arms, and torso. As another example, a male skeleton may have a slightly longer torso than a female skeleton because females generally have a lower center of gravity. Alternatively, a user could input more specific body measurements, such as height, weight, arm length, shoulder width, leg length, etc. The application 250 may request even more specific attributes, like forearm length, a measurement from knee to ankle, foot size, or the like. Basketball players tend to be lanky, so arm length and leg length may greatly vary from player to player. If the computer performing video analysis 102 or 106 receives very accurate and specific measurements, the computer performing video analysis 102 or 106 may generate a very accurate skeleton 400, 500 and determine jump shot deviations from the skeleton 400, 500 very accurately.
[0057] The computer performing video analysis 102 or 106 may further be able to stretch or shrink the skeleton 400, 500 based on the distance between the player 430 and the camera 206. The player 430 may appear smaller or larger depending on the distance between the camera 206 and the player 430. The computer performing video analysis 102 or 106 may account for camera distance by stretching or constricting the skeleton 400, 500. The application 250 may also notify a user of a suggested distance between the camera 206 and the player 430 for the video footage from each angle.
[0058] The computer performing video analysis 102 or 106 may receive multiple videos from a front perspective and a side perspective. The video taken from the side perspective allows the video performing video analysis 102 or 106 to determine the arc of the jump shot, determine whether the ball went through the hoop, and perform analysis of the player's 430 mechanics from the side based on the side perspective skeleton 500. For example, the computer performing video analysis 102 or 106 may determine if a player 430 is jumping forward a significant amount. Ideally, the location where the player 430 left the ground and the location where the player 430 lands should be substantially similar. If the player 430 jumped forward a significant amount or leaned forward while shooting, the computer performing video analysis 102 or 106 may detect such mechanical flaws using video taken from the side perspective.
[0059] Receiving high quality video greatly reduces errors in video analysis. Typically good video analysis footage includes a high resolution, good lighting, camera stability, and a consistent background. For example, if the background is substantially the same color, the computer performing video analysis 102 or 106 may determine the subject of the analysis quickly and with less processing power. The application 250 may offer video capturing tips before the application 250 begins capturing video of the jump shot 108, such as tips for holding the camera still, tips for a preferred background, and an idea distance from the shooter. Further, the application 250 may check the video footage quality before performing video analysis or sending the video to the remote server 106 for video analysis. If the application 250 determines that the video quality is poor, the application 250 may alert the user that another video should be taken of the player's 430 jump shot 108.
[0060] When a video is taken of the player's 430 jump shot 108, the mobile device's 102 camera 206 captures a plurality still frames, which when played in a sequence appear to the human eye as a moving picture. The mobile device 102 camera 206 captures approximately 24-30 frames per second. So, depending on how long the camera 206 captures video, the computer performing video analysis 102 or 106 determines which frames match the various skeleton 400, 500 stages. The computer performing video analysis 102 or 106 may need to only analyze one video frame per skeleton 400, 500 stage. So, if the skeleton comprises five stages, the computer performing video analysis 102 or 106 may search for five video frames that match the five skeleton 400, 500 stages. For example, the base stage 402, 502 has low ball position and slightly bent knees. So, the computer performing video analysis may determine which video frame depicts the player 430 with bent knees and a low ball position. Given the characteristics of the base phase 402, 502, the computer performing video analysis 102 or 106 may determine which frame captured the player 430 with the most knee bent by finding, for example, the frame where the player's head is lowest in the frame. A frame determined to be the frame embodying the base phase 402, 502 may be analyzed, for example, by superimposing the base stage 402, 502 image of the skeleton 400, 500. Because each phase of the jump shot has identifying characteristics, the computer performing video analysis 102 or 106 may use the identifying characteristics to find a frame that matches each phase of the jump shot. For example, the computer performing video analysis 102 or 106 may find a frame where the player 430 is on his toes but not quite in the air for the frame phase 404, 504, a frame where the player 430 is at the crest of his jump for the release phase 406, 506, a frame where the ball has left the player's hand for the follow through phase 408, 508, and a frame where the player has landed again for the finish phase 410, 510.
[0061] The computer performing video analysis 102 or 106 finds a video frame for each jump shot phase from both the front perspective and the side perspective. In this way, the computer performing video analysis 102 or 106 finds a frame from the front perspective for each jump shot phase and a frame from the side perspective for each jump shot phase. Subsequently, each skeleton stage from the front skeleton 400 is superimposed on a corresponding frame captured from the front perspective, and each skeleton stage from the side skeleton is superimposed on a corresponding frame captured from the side perspective.
[0062] It is important that the computer performing video analysis 102 or 106 determine the base phase 402, 502 first because then the computer 102 or 106 may ignore any frames occurring before the determined base phase 402, 502 when searching for the subsequent phases of the jump shot.
[0063] Based on the number of frames between the frame determined to be the base phase 402, 502 and the frame determined to be the follow through phase 408, 508, the computer performing video analysis may determine how quickly the player is releasing the ball. A quick release is very important in basketball because it gives a defender less time to react to a jump shot, thereby decreasing the likelihood of a blocked shot or the defender blocking the player's 430 view of the basket. So, the computer performing video analysis 102 or 106 may suggest drills and fundamental tweaks for a quicker release if the computer performing video analysis 102 or 106 determines that the release is happening too slowly.
[0064] Further still, the computer performing video analysis 102 or 106 may determine whether the ball was released at the pinnacle of the player's 430 jump by determining which video frame displays the player at the highest jump point. The computer performing video analysis 102 or 106 may determine where the ball is located within the frame determined to be the highest jump point to determine whether the player released the ball at the pinnacle of the jump shot. The computer performing video analysis 102 or 106 may analyze body control during all phases of the jump shot. For example, the computer performing video analysis 102 or 106 may analyze a player's balance and body position while stepping into the jump shot, or the computer performing video analysis 102 or 106 may analyze whether the player jumped and landed at substantially the same location on the floor. If body control issues are found, the computer performing video analysis 102 or 106 may suggest that the player moves closer to the basket knowing it is easier to master mechanics from a closer, achievable range without jumping forward. The computer performing video analysis 102 or 106 may also suggest drills and critiques to improve the player's body control during the jump shot. For example, having a proper knee bend and squat may help the player control his body during the jump and increase the player's range over time.
[0065] The computer performing video analysis 102 or 106 may also track the arc angle of the shot taken. Using the side angle video, the computer performing video analysis 102 or 106 may find the ball and track the ball's position at various points in the ball flight to calculate an arc angle. For example, the optimal arc angle may be between 45 and 60 degrees. The arc angle may be calculated on the ascent when the ball leaves the player's 430 hand and during the decent as the ball travels toward the basket. Based on the calculated arc angle, the computer performing video analysis 102 or 106 may suggest critiques, games, or training videos to help a player improve their jump shot arc angle.
[0066] As described above, the computer performing video analysis 102 or 106 may superimpose one of the skeleton 400 or 500 stages onto a frame determined to correspond with the stage. For example, the computer performing video analysis 102 or 106 determines a frame that best matches the base phase 402 and superimposes a base skeleton stage 402s onto the frame. The computer performing video analysis 102 or 106 compares the skeleton's 402s joint and connector locations and the player's 430 joint, limb, and body positions. Alternatively, the computer performing video analysis 102 or 106 may generate the player skeleton and then compare the player skeleton to the reference skeleton 400, 500. Based on the comparisons, the computer performing video analysis 102 or 106 determines deviations from the skeleton 400, 500 in the jump shot, if any.
[0067] For example, if the player 430 performs very little knee bend and squat in the base phase 402, 502, the knees and hip joints for the skeleton 400, 500 are going to be lower than the player’ 430 knees and hip locations. After determining that the skeleton's 400, 500 knees and hip joint are lower than the player's 430 knees and hips, the computer performing video analysis 102 or 106 may find the deviations from the skeleton 400, 500 in the player's base phase 402, 502, more specifically, in their knee bend and squat. A deeper knee bend and squat may produce more upward thrust when leaping, which helps send the ball toward the basket.
[0068] For example,
[0069] The computer performing video analysis 102 or 106 performs same frame determination and skeleton 400, 500 comparison processes for all phases of the jump shot from both the front perspective and the side perspective.
[0070] For example,
[0071] As another example,
[0072]
[0073]
[0074] After deviations from the skeleton 400, 500 are determined, the computer performing video analysis 102 or 106 may detect a player's skill level. The computer performing video analysis 102 or 106 may determine the player's skill level based on the number of determined deviations between the reference skeleton 400, 500 and the player's body movements. If the player is a beginner, the computer performing video analysis 102 or 106 may report only the most basic, major deviations from the skeleton 400, 500 to the player so that the player may build a solid foundation. For example, if a beginner player does not have his feet and shoulders square in the base phase 402, 502, the computer performing video analysis 102 or 106 may recommend drills and critiques to fix this basic deviation from the skeleton 400, 500. After basic deviations from the skeleton 400, 500 are fixed, the computer 102 or 106 may suggest further refinements for more advanced players. Meanwhile, the computer performing video analysis 102 or 106 may report more minor deviations from the skeleton 400, 500 to a more advanced player. For example, if an advanced player has a square body to the basket and a strong follow through, the computer performing video analysis 102 or 106 may recommend drills and methods to take quicker shots or increase ball rotation. The computer may track the player's 430 progress
[0075]
[0076] Subsequently, in step 604, the mobile device 102 determines which frames of the video footage captured by the mobile device 102 camera 206 actually represent the player 430 taking a jump shot. Any video frames not representing the jump shot, such as frames where the player is dribbling, are disregarded. Only frames where the player 430 is taking a jump shot are analyzed. As a result, only about one-three seconds of footage needs to be analyzed for jump shot fundamentals. This determining step is performed on both the front perspective footage and the side perspective footage.
[0077] After disregarding extra footage, the processor 200 of the mobile device 102 searches the frames depicting the jump shot for a frame best representing the base phase 402, 502 in step 606. For example, the processor 200 finds the frame where the player is lowest in his squat to find the frame representing the base phase 402, 502 of the jump shot. This searching step is performed on both the front perspective footage and the side perspective footage.
[0078] After finding the frame depicting the base phase, the processor 200 generates a skeleton 400, that matches the player's 430 measurements in step 608. The method 600 assumes that a user has already inputted player 430 measurements (height, weight, arm length, gender, etc.) into to the mobile device 102. If not, the application 250 may prompt the user to enter such information before generating the skeleton 400, 500. The processor 200 references the front perspective skeleton 400 matching the player's measurements and the side perspective skeleton 500 matching the player's measurements.
[0079] In step 610, the processor 200 superimposes the base stage 402, 502 skeleton 400, 500 onto the frame determined to depict the base phase 402, 502 of the jump shot. This superimposing step is performed on both the front perspective frame and the side perspective frame.
[0080] Next, the processor 200 analyzes the player's 430 base phase 402, 502 fundamentals based on the superimposed skeleton 400, 500 in step 612. Depending on the player's 430 skill level, either minor or major deviations from the skeleton 400, 500 are flagged. The processor 200 may find deviations from the skeleton 400, 500 in the base phase 402, 502 of the player's jump shot by comparing the joints of the skeleton 400, 500 to the player's 430 joints. Also, the he computer performing video analysis 102 or 106 may find deviations from the skeleton 400, 500 in the base phase 402, 502 by generating the player skeleton and comparing it to the reference skeleton 400, 500. For example, if the player's 430 shoulders do not match the skeleton's 400, 500 shoulder joints, the player's 430 shoulders may not be square to the basket. The orientation of the player's 430 shoulders may be a result of feet that are not square to the basket. Feet location may also be analyzed. The processor 200 may further analyze how closely the player's limbs and torso match vectors connecting the skeleton's 400, 500 joints. This analysis step is performed on both the front perspective footage and the side perspective footage.
[0081] After analyzing the base phase 402, 502 against the base stage skeleton, the computer performing video analysis 102 or 106 repeats steps 606-612 for the other phases of the jump shot. For example, the processor 200 may repeat steps 606-612 for the frame phase 404, the release phase 406, the follow through phase 408, and the finish phase 410.
[0082] After analyzing all phases of the jump shot, the mobile device 102 uploads the results to the remote server 106 in step 614, and the mobile device 102 also presents the video analysis results to the user on the I/O device 204 in step 616.
[0083]
[0084] Subsequently, in step 704, the remote server 106 processor 300 determines which frames of the video footage captured by the mobile device 102 camera 206 actually represent the player 430 taking a jump shot. Any video frames not representing the jump shot, such as frames where the player 430 is dribbling, are disregarded. Only frames where the player 430 is taking a jump shot are analyzed. This determining step is performed on both the front perspective footage and the side perspective footage.
[0085] After disregarding extra footage, the processor 300 searches the frames depicting the jump shot for a frame best representing the base phase 402, 502 of the jump shot in step 706. For example, the processor 300 finds the frame where the player is lowest in his squat to find the frame representing the base phase 402, 502 of the jump shot. This searching step is performed on both the front perspective footage and the side perspective footage.
[0086] After finding the frame depicting the base phase 402, 502, the processor 300 generates a skeleton 400, 500 that matches the player's 430 body measurements in step 708. The method 700 assumes that the remote server 106 has already received the body measurements from the mobile device 102. In the exemplary embodiments, the application 250 on the mobile device 102 may not allow the user to upload video to the remote server 106 until the body measurements have been entered into the mobile device 102. In one embodiment, the mobile device 102 sends player body measurements with the captured video. Alternatively, the remote server 106 may save player body measurements for each player 430.
[0087] In step 710, the processor 300 superimposes a base stage 402, 502 of the referenced skeleton 400, 500 onto the frame determined to depict the base phase 402, 502 of the jump shot. This superimposing step is performed on both the front perspective frame and the side perspective frame.
[0088] Next, the processor 300 analyzes the player's 430 base phase 402, 502 fundamentals based on the superimposed skeleton 400, 500 in step 712. Depending on the player's 430 skill level, either minor or major deviations from the skeleton 400, 500 are flagged. The processor 200 may find deviations from the skeleton 400, 500 in the base phase 402, 502 of the player's jump shot by comparing the joints of the skeleton 400, 500 to the player's 430 joints. Also, the he computer performing video analysis 102 or 106 may find deviations from the skeleton 400, 500 in the base phase 402, 502 by generating the player skeleton and comparing it to the reference skeleton 400, 500. For example, if the player's 430 shoulders do not match the skeleton's 400, 500 shoulder joints, the player's 430 shoulders may not be square to the basket. The orientation of the player's 430 shoulders may be a result of feet that are not square to the basket. Feet location may also be analyzed. The processor 300 may further analyze how closely the player's limbs and torso match vectors connecting the skeleton's 400, 500 joints. This analysis step is performed on both the front perspective footage and the side perspective footage.
[0089] After analyzing the base frame 402, 502 against the base stage skeleton 400, 500, steps 706-712 for the other phases of the jump shot. For example, the processor 300 may complete steps 706-712 for the frame phase 404, the release phase 406, the follow through phase 408, and the finish phase 410.
[0090] After analyzing all phases of the jump shot, the remote server 106 saves the results to a database in step 714 and sends the results to the mobile device 102 in step 716. When the remote server 106 sends the results, the remote server 106 may include hyperlinks to training videos to help the player fix the fundamental deviations from the skeleton 400, 500 found by the remote server 106.
[0091] As described above, the remote server 106 may perform tasks other than just video analysis. For example, the remote server 106 may save a video of the player's 430 best shot in the database 308. The player 430 can download that shot at any time to view what they did right and what still needs improvement. The database 308 may save many best shot videos for all players 430 registered with the remote server 106. This best shot practice may be used more frequently by professionals who do not need to improve mechanics, but should simply maintain their method of shooting. For example, the professional player may save their best shot in the database 308, and the computer performing video analysis 102 or 106 may generate a skeleton 400, 500 based on that best shot saved in the database 308. In this way, the professional player may quickly determine what changes in his jump shot he may have developed from his best shot.
[0092] In addition, the remote server 106 or the mobile device 102 may prepare a video of the player's 430 jump shot with the skeleton 400, 500 superimposed over the video. In this way, the player 430 can see how the computer performing video analysis 102 or 106 determined the deviations from the skeleton 400, 500 reported to the player 430.
[0093] The remote server 106 may save many videos in the database 308. For example, the database 308 may save one video from each training session, and the player 430 may view past jump shot videos to see the progress he is making. The database 308 may further save progress data. For example, in training session one the remote server 106 may note in the database 308 that the player 430 was not square to the basket. However, in training session ten, the remote server 106 may find that the player 430 was square to the basket. Using these two pieces of information, the remote server 106 may note that the player 430 has made significant progress on a previous issue with his or her jump shot.
[0094] The video analysis system 100 may further be configured to detect the ball and a basketball hoop to determine whether the player shot the ball through the hoop. The remote server 106 or the mobile device 102 may further be configured to track the number of made baskets during a training session. The remote server 106 or the mobile device 102 may compile statistics for the player 430 detailing the number of made shots, the number of shots taken, the percentage of shots made, etc.
[0095] If the video analysis system 100 is configured to detect whether the ball went through the basketball hoop, the video analysis system 100 may use this outcome data (e.g. whether the ball went through the hoop) and correlate the outcome data with detected deviations from the reference skeleton 400, 500. For example, if a player 430 has a tendency to jump forward on his jump shots, this would show up as a deviation from the reference skeleton 400, 500. But, if the player 430 generally makes most of the shots where he jumps forward, the jumping forward deviation may be a low priority fundamental deviation for the player 430 to work on. As another example, the player 430 may miss 80% of his shots when he leaves his elbow out versus making 70% of his shots when his elbows are properly positioned. So if the video analysis system 100 detects both the jumping forward deviation and the elbow deviation, the video analysis system 100 may only report the elbow deviation to the player 430. In this way, the video analysis system 100 communicates which deviations are outcome determinative, which may be helpful in identifying which aspects of the jump shot the player 430 should focus on. In subsequent training sessions, when the elbow problem has been addressed, the video analysis system 100 may suggest a more balanced jump and less jumping forward.
[0096] The video analysis process described herein can easily be performed on other physical activities, such as a golf swing, throwing a football, a baseball swing, a pitching throw, running, or any other physical activity. To apply the exemplary embodiments to other sports or activities, the remote server 106 needs only to receive skeletons for other physical activities. More specifically, the video analysis process may be applied to a volleyball serve, a tennis serve, throwing a football, kicking or punting a football, a golf swing, discus throwing, shot put throwing, track events (high jump, pole vaulting, hurdling, long jumping), or swimming strokes (e.g. back stroke, free style, butterfly, breast stroke). To apply the video analysis process described herein to other physical activities, the system may include different skeletons uploaded to the database 308 based on the various physical activities.
[0097] For physical activities other than a jump shot, the remote server 106 or the mobile device 102 may store different skeletons than the jump shot skeleton 400, 500 illustrated in
[0098] The skeleton-based video analysis process may also be configured to perform injury prevention tests. For example, the system may receive video of a person performing an anterior cruciate ligament (ACL) jump test. During the ACL jump test, a person may jump from a relatively low height (e.g. a chair or bench) and land on the floor. The way the person jumps and lands may indicate weakness in the person's knee ligaments, which may lead to significant injury. The skeleton for the ACL test may illustrate an ideal way to jump and land from a bench. The computer performing video analysis 102 or 106 may look for deviations from the ACL test skeleton. If deviations found have been shown to increase injury, the person may try to correct their jumping and landing style to prevent injury.
[0099] Similarly to the ACL test, under-pronation or over-pronation during running, which can lead to serious foot or ankle injuries, may be detected while a person runs. By analyzing video of a person running based on a running skeleton, the computer performing video analysis 102 or 106 may determine whether the person exhibits over-pronation or under-pronation problems. The computer 102 or 106 may then suggest simple injury prevention videos to help the person prevent injury as a result of their over-pronation or under-pronation problems.
[0100] In view of the foregoing, it will be seen that the several advantages of the invention are achieved and attained.
[0101] The exemplary embodiments can include one or more computer programs that embody the functions described herein and illustrated in the appended flow charts. However, it should be apparent that there could be many different ways of implementing aspects of the exemplary embodiments in computer programming, and these aspects should not be construed as limited to one set of computer instructions. Further, those skilled in the art will appreciate that one or more acts described herein may be performed by hardware, software, or a combination thereof, as may be embodied in one or more computing systems.
[0102] The functionality described herein can be implemented by numerous modules or components that can perform one or multiple functions. Each module or component can be executed by a computer, such as a server, having a non-transitory computer-readable medium and processor. In one alternative, multiple computers may be necessary to implement the functionality of one module or component.
[0103] In another example embodiment, the jump shot analysis system can be deployed in a multi-camera environment on a basketball court as shown by
[0104] For example, a first camera 1402 can be referred to as a baseline camera because it is positioned near baseline 1404 at a location sufficient to capture a baseline perspective view of a player shooting a jump shot on the court. It should be understood that camera 1402 can be positioned at any point along the baseline 1404 sufficient to generate a desired perspective of the shooter (as indicated by the arrows near camera 1402). It should also be understood that the baseline camera 1402 need not be positioned precisely on the baseline so long as it is positioned to generate baseline perspective video of the shooter.
[0105] A second camera 1406 can be referred to as a sideline camera because it is positioned near sideline 1408 at a location sufficient to capture a sideline perspective view of a player shooting a jump shot on the court. A third camera 1410 can also be referred to as a sideline camera, and it is positioned near the sideline 1412 opposite from sideline 1408. As with baseline camera 1402, it should be understood that cameras 1406 and 1410 can be positioned at any point along their respective sidelines 1408 and 1412 sufficient to generate a desired perspective of the shooter (as indicated by the arrows near cameras 1406 and 1410). It should also be understood that the sideline cameras 1406 and 1410 need not be positioned precisely on the sidelines so long as they are positioned to generate sideline perspective videos of the shooter.
[0106] A fourth camera 1414 can be referred to as a backcourt camera because it is positioned behind where a shooter will likely be shooting (e.g., near half court or the baseline opposite baseline 1404) so as to generate a back view of the shooter.
[0107] Through any combination of these cameras, multi-perspective views (e.g., front views, side views, back views, angle views), including 3D models, can be generated of players who are shooting jump shots. The video data generated by these cameras can be transmitted to a computer 1450 via wired or wireless links 1452. For ease of illustration, link 1452 is shown with respect to baseline camera 1402. However, it should be understood that each camera can include such a link 1452 with computer 1450 so as to provide computer 1450 with the shooter videos. Computer 1450 can be configured to execute program code as discussed herein that analyzes the video data to perform jump shot analysis. For example, computer 1450 can process the video data from multiple perspectives to detect and identify deviations from a reference skeleton and suggest corrections.
[0108] Furthermore, if desired by a practitioner, any of the cameras 1402, 1406, 1410, and 1414 can be positioned at a sufficiently high elevation that reduces the impact of obstructions caused by other players who are sharing the court with the shooter. For example, if used during a live game or scrimmage environment, other players on the court can be expected to be positioned between one or more of the cameras and the shooter when the shooter shoots the ball. Elevating the cameras above player height can help reduce possible occlusions/blocking. For example, the cameras could be positioned at a height of around 10 feet to help reduce obstruction. However, when used in a practice environment when there is less of a risk of obstruction, the cameras can be held or positioned a shoulder height or the like.
[0109] Unless specifically stated otherwise as apparent from the following discussion, it is appreciated that throughout the description, discussions utilizing terms such as “determining” or “comparing” or “processing” or the like, can refer to the action and processes of a data processing system, or similar electronic device, that manipulates and transforms data represented as physical (electronic) quantities within the system's registers and memories into other data similarly represented as physical quantities within the system's memories or registers or other such information storage, transmission or display devices.
[0110] The exemplary embodiments can relate to an apparatus for performing one or more of the functions described herein. This apparatus may be specially constructed for the required purposes, or it may comprise a general purpose computer selectively activated or reconfigured by a computer program stored in the computer. Such a computer program may be stored in a machine (e.g. computer) readable storage medium, such as, but is not limited to, any type of disk including floppy disks, optical disks, CD-ROMs and magnetic-optical disks, read only memories (ROMs), random access memories (RAMs) erasable programmable ROMs (EPROMs), electrically erasable programmable ROMs (EEPROMs), magnetic or optical cards, or any type of media suitable for storing electronic instructions, and each coupled to a bus.
[0111] The exemplary embodiments described herein are described as software executed on at least one server, though it is understood that embodiments can be configured in other ways and retain functionality. The embodiments can be implemented on known devices such as a personal computer, a special purpose computer, cellular telephone, personal digital assistant (“PDA”), a digital camera, a digital tablet, an electronic gaming system, a programmed microprocessor or microcontroller and peripheral integrated circuit element(s), and ASIC or other integrated circuit, a digital signal processor, a hard-wired electronic or logic circuit such as a discrete element circuit, a programmable logic device such as a PLD, PLA, FPGA, PAL, or the like. In general, any device capable of implementing the processes described herein can be used to implement the systems and techniques according to this invention.
[0112] It is to be appreciated that the various components of the technology can be located at distant portions of a distributed network and/or the Internet, or within a dedicated secure, unsecured and/or encrypted system. Thus, it should be appreciated that the components of the system can be combined into one or more devices or co-located on a particular node of a distributed network, such as a telecommunications network. As will be appreciated from the description, and for reasons of computational efficiency, the components of the system can be arranged at any location within a distributed network without affecting the operation of the system. Moreover, the components could be embedded in a dedicated machine.
[0113] Furthermore, it should be appreciated that the various links connecting the elements can be wired or wireless links, or any combination thereof, or any other known or later developed element(s) that is capable of supplying and/or communicating data to and from the connected elements. The term module as used herein can refer to any known or later developed hardware, software, firmware, or combination thereof that is capable of performing the functionality associated with that element. The terms determine, calculate and compute, and variations thereof, as used herein are used interchangeably and include any type of methodology, process, mathematical operation or technique.
[0114] The embodiments were chosen and described in order to best explain the principles of the invention and its practical application to thereby enable others skilled in the art to best utilize the invention in various embodiments and with various modifications as are suited to the particular use contemplated.
[0115] As various modifications could be made in the constructions and methods herein described and illustrated without departing from the scope of the invention, it is intended that all matter contained in the foregoing description or shown in the accompanying drawings shall be interpreted as illustrative rather than limiting. Thus, the breadth and scope of the present invention should not be limited by any of the above-described exemplary embodiments, but should be defined only in accordance with the following claims appended hereto and their equivalents.