ADAPTABLE EXERCISE SYSTEM AND METHOD

Abstract

A system and method for correlating a video content presentation rate with an exercise machine operation rate is disclosed. A preferred embodiment of the apparatus is an interactive video system that adapts easily to use with a wide range of gym equipment utilizing a video presentation device that includes an accelerometer and/or camera. The camera embodiment utilizes vision algorithms to determine periodicity and motion orientation prior to modifying the input as a function of the type of exercise equipment is being utilized, in order to determine a cadence. In one embodiment the camera does not rely on seeing the limbs being exercised but utilizes the observation that even highly trained athletes have a degree of extraneous motion of their theoretically immobile body parts during exercise. A prerecorded video presentation is modulated by sensing proximate vibrational energy to determine cadence in the case of the accelerometer.

Claims

1. A system to control a video content while the content is viewed during use of any of a plurality of different exercise machines comprising: a display device contained within a first housing including a first display in electronic communication with a processor, a sensor in electronic communication with the processor, and a set of machine code adapted for operation on said processor and implemented to modulate the presentation of video content at least in part as a function of cadence or intensity of user exercise, said cadence or user intensity, to be determined by data from the sensor wherein the sensor is adapted to measure displacement of an internal mass.

2. The system in claim 1 wherein the sensor is an accelerometer.

3. The system in claim 1 wherein the sensor is a gyroscope.

4. The system in claim 2 wherein the accelerometer is disposed at a distance from the first housing and the electronic communication is wireless.

5. The system of claim 4 wherein the accelerometer is disposed on a user.

6. The system in claim 2 wherein the accelerometer is disposed within a second housing adapted for attachment to a treadmill such that the accelerometer can detect footfalls.

7. The system in claim 1 where the first housing includes a harness adapted to attach the housing to a human head and further including at least one optical lens disposed between the display and the head.

8. The system in claim 7 where the sensor is a gyroscope.

9. The system in claim 7 where the sensor is an accelerometer.

10. A method for analyzing a running step cycle of a user on a treadmill comprising: collecting camera frames from a camera pointing at the user's torso; determining a periodicity from motile pixels; measuring the treadmill belt speed; comparing the periodicity of motile pixels to the treadmill belt speed; and deriving the belt distance traveled of each running step cycle.

11. The method of claim 10 further comprising: storing the distance traveled of each running step cycle in a memory module.

12. The method of claim 11 further comprising: analyzing the database to derive statistical measures of recent running step cycles observed.

13. An exercise monitoring system including: a first display, a memory module, and a camera, each in electronic communication with a processor, further including a set of machine code adapted for operation on said processor to store camera frames in a database, retrieve camera frames from the first database and to analyze a set of repetition patterns to enable a counting and a recording of repetitions of a user's exercise motions for a plurality of different exercise types.

14. The system of claim 13 where the set of repetition patterns are identified as a particular form of exercise by means of a trained classifier algorithm running in the machine code.

15. The system of claim 13 further including a speaker adapted to audibly enumerate the number of repetitions of the user's exercise motions in real time.

16. The system of claim 13 wherein the database stores the number of discrete repetitive exercise motions identified by the machine code.

17. The system of claim 13 wherein the counting is operable to control display rate of video content.

18. The system of claim 13 further including a virtual user interface enabling captured sets of camera frames to be manually categorized.

19. The system of claim 13 further including a component in which an identifying exercise motion pattern is derived from measured performance criteria of unspecified periodic motions in a larger sequence of still frames.

20. The system of claim 13 further including a virtual user interface which displays a summary of the exercise motions comprised of camera frames retrieved from the database.

Description

DESCRIPTION OF DRAWINGS

[0032] FIG. 1 shows a flowchart for modifying a video presentation by exercise machine.

[0033] FIG. 2 shows a flowchart of an embodiment for determining cadence.

[0034] FIG. 3 shows a display device disposed on a cycling exercise machine.

[0035] FIG. 4 shows a display device disposed on a treadmill.

[0036] FIG. 5 shows a flowchart for manipulating visual content to provide a desired exercise profile.

[0037] Like reference symbols in the drawings indicate like elements.

DETAILED DESCRIPTION

[0038] FIG. 1 shows a generic flowchart of system operation. Video tracking module 14, host application 12, display 20, processor 21, camera 15, user exercise tracker 16 and user position tracker 13 all reside within display device 11. In an alternate embodiment display device 11 includes an accelerometer 18 and vibration tracking module 17 in lieu of or in addition to camera 15 and its associated elements. Display device 11 can be a smartphone, tablet computer, laptop or other kind of computing device as long as it has a camera 15 and/or accelerometer 18 integrally manufactured therein. Host application 12 is software running on display device 11 that accepts inputs from the user position tracker 13 and/or user exercise tracker 16. This input is used to modify video (and associated audio) presented on display device 11.

[0039] The video presentation may be forward motion video, mimicking the experience of walking the streets of Paris, hiking through a forest path, swimming underwater, and the like. In such presentations it is desirable to correlate the speed of exercise, such as walking in the case of treadmill, pedaling in the case of bicycle, stroking in the case of rowing, or performing repetition in the case of weightlifting. When the user stops exercise, the video likewise becomes still. The video presentation may likewise be a game such as: a) a race game wherein the user's activity is represented on screen so that exercise rate correlates with speed within the game. Opponents within the game can be synthetic or can be on screen representations of other people exercising; b) a point accumulation game in which the user accumulates points by accomplishing tasks such as steering their character, or the like or c) navigating through a series of stored images, allowing the user to advance (such as by leaning torso to the right), rewind (such as by leaning torso to the left) rotate right (such as by tilting head to the right) and rotate left (such as by rotating head to the left) and deleting images (such as by crossing arms twice). In this manner the user can perform useful and relatively mindless work while exercising.

[0040] Camera 15 sends motion and position vectors to video tracker 14, which outputs periodicity and “center of mass” data to user position tracker 13 and user exercise tracker 16. User position tracker 13 has an established centerline for the torso and head and can therefore determine if the user is leaning right or left any given time. In an embodiment with a camera 15, user exercise tracker 16 receives periodicity information from video tracker 14 and utilizes knowledge of the exercise machine type (either extrapolated from machine vision or input directly by the user in a setup screen) the system determines an exercise cadence. In a preferred embodiment, the output from the camera is interpreted by the system in its entirety, meaning it is viewing the entire scene simultaneously without differentiation. The system therefore tracks consistent nd/or repetitive motion of a large area of adjacent pixels in order to determine cyclical activity. This approach is useful because in most cases the visual surface of the user will constitute the largest region of motile pixels in the frame. Furthermore if the region being tracked falls off the visual surface of the user or if the user steps out of the scene and another replaces them, the tracker will gracefully and seamlessly re-center itself as these conditions remain true. In one embodiment, the system identifies edges and/or regions of motion and tracks those features as user features, such as a head, torso or hand. A further variant of this previous embodiment includes the system tracking motile pixels (e.g. variants in pixel darkness and/or color) that enter the camera view distal to said user features. The system subsequently rejects these motions as extraneous activity such as a trainer or fellow exercise participant walking behind the user.

[0041] In one embodiment, the user will tell the system the type of exercise; however, depending on the exercise being performed the small motion periodicity will be vertical or horizontal. In cycling the small motion periodicity is horizontal in orientation whereas on the treadmill and rowing machines, the small motion periodicity is vertical. This data may be used for the system to determine which exercise is being performed. In a preferred embodiment the system relies on the observation that even highly trained athletes have a degree of extraneous motion of their theoretically immobile body parts during exercise. Therefore, exercise tracker 16 is responsible for: a) determining when to trust the periodicity output of the video tracker, and when to apply a correction factor relative to the cadence as a function of what type of exercise is being performed; and b) how to interpret the motion (e.g. horizontal for bikes, vertical for treadmills), and c) if both camera 15 and accelerometer 18 are used in combination, when to defer to the vibrational energy measurement to provide smoother and more responsive exercise output to the host application 12.

[0042] In an embodiment utilizing accelerometer 18, vibration tracker 17 correlates data from accelerometer 18 with a lookup table or calculates a value that scales relatively with exercise intensity. When using a treadmill, this signal will have a frequency corresponding to steps per minute.

[0043] FIG. 2 shows the input from camera 15 being stored in database 32 as a series of sequential still images, each with a timestamp. Because images are at fixed intervals and sequential is understood that a timestamp may simply be a number in the sequence. At a capture rate of 24 frames per second, each image will be stored with a timestamp varying by 1/24 of a second. Database 32 will typically store at least 2 seconds of still images. At any given instant in time there will be a live frame 31, the frame that most closely represents the action of the user in real time. In step 34, live frame 31 is compared to the frames stored in database 32. In one embodiment pixels are compared for lightness and darkness enabling the system to determine which still image, closest in time to live frame 31 represents a similar pattern to live frame 31) In step 36, the timestamp of the image that is closest in time to live frame 31 (and the knowledge of the time between sequential images) is used to determine cadence 36. (i.e The system determines which two still images are closest in time and most similar to each other allows the system to understand the periodicity of motion of the user's activity and is elsewhere described within the application this information is used to determine cadence.)

[0044] In one embodiment the system can determine additional resolution by extrapolation. The extrapolation may be performed by artificially creating incremental frames, or by measuring the distance between light and dark pixels of the two approximately similar images and determining an offset (either slightly advanced or slightly decremented), thereby allowing the system to determine the cadence 36 in higher resolution than the capture rate of camera 15.

[0045] FIG. 3 shows display device 11 temporarily disposed on a shelf 30 that is itself integrally manufactured into cycling exercise machine 50. As shown, shelf 30 may be slanted to appear more like a rack. View region 52 is the view seen by camera 15 and inherent to the manufacture of display device 11 and typically not under user control. View region 52 sees a portion of the user not producing the cadence 36, while the legs are traditionally seen as the body part being exercised (and producing the cadence 36) on this particular machine. View region 52 views a portion of the user that is relatively stationary. In this case the camera 15 is directed to the user's head and torso while the legs are producing cadence 36. Each form of exercise has a characteristic relationship between its visual periodicity and orientation and user cadence 36. In the case of bicycling, video tracker 14 measures the periodicity of small horizontal motions of the user's head and/or torso to determine the cadence 36, which is equal to measured periodicity. Therefore the camera extrapolates exercise cadence while disposed to view away from the machine itself.

[0046] Furthermore, the user can indicate leftward motion or a leftward decision to host application 12 by tilting his head and/or torso to the left, as disclosed in FIG. 1. This action may be taken without interrupting exercise and of course may also be done toward the right.

[0047] Embodiments using accelerometer 18 are measuring intensity through the vibrational energy transmitted from the user through cycling exercise machine 50 and shelf 30.

[0048] Display device 11 adapts between different exercise machines simply by picking it up, resting it on shelf 30 (associated with the machine of interest) and indicating in a “setup” feature within host application 11 the type of exercise machine is being used.

[0049] FIG. 4 shows display device 11 temporarily disposed on a shelf 30 that is itself integrally manufactured into treadmill 51. As shown, shelf 30 may be slanted to appear more like a rack. As in FIG. 3, view region 52 is inherent to the manufacture of display device 11 and typically not under user control. As in FIG. 3, view region 52 sees a portion of the user not producing the cadence 36, while the legs are traditionally seen as the body part being exercised (and producing cadence 36) on this machine as well. View region 52 sees a portion of the user that is relatively stationary. In this case the camera 15 is directed to the user's head and torso while the legs are producing cadence 36.

[0050] Each form of exercise has a characteristic relationship between its visual periodicity and orientation and user cadence 36. In the case of running, in one embodiment, video tracker 14 determines the motion of center of mass 55 of the entire image presented by camera 15 within view region 52 and determines the periodicity of the vertical component to determine the cadence 36, which is equal to one half the measured periodicity. Therefore the camera extrapolates exercise cadence of the user (and the machine) while disposed to view away from the machine itself.

[0051] Furthermore, the user can indicate rightward motion or a rightward decision to host application 12 by tilting his head and/or torso to the right, as disclosed in FIG. 1. This action may be taken without interrupting exercise and of course may also be done toward the left.

[0052] Embodiments using accelerometer 18 are measuring intensity through the vibrational energy transmitted from the user through cycling exercise machine 50 and shelf 30.

[0053] FIG. 5 shows an embodiment to allow the user to experience a desired exercise regimen or profile within the context of interactively viewing a video presentation such as forward motion video or a game. It is common for users to prefer certain exercise profiles. An example of such a profile is: a warm-up period, followed by short periods of ever increased intensity interspersed with periods of rest, followed by a cooling down period. Furthermore, each aspect of the profile may have certain time periods associated, such as a 10 min. warm-up, or 30 seconds of intensity followed by 45 seconds of rest, followed by 10 min. of cooling down.

[0054] To achieve this profile 43 within the context of a video presentation the video is edited to provide scenes that correlate with the experience needed to facilitate the exercise outcome. For example, if the context is a walk through the streets of San Francisco, the video will be modified to include stairs, up hills and down hills, all at the appropriate timing sequence in order to facilitate the desired exercise profile. The video is edited to match the profile desired by the user. This embodiment may also be provided in the game context, by providing a synthesized game in which the gameplay is artificially manipulated to provide the desired exercise profile. While the user perceives the experience as a series of random events it is in fact an experience manufactured to produce a specific exercise profile desired by the user. In one embodiment, the experience is manufactured dynamically as a function of the measured physiology of the user.

[0055] For example, in a car race game in which the user is riding a rowing machine (or any other machine) the activities within the race will be modified in accordance with the desired exercise profile. In order to establish a rest period for the user, the system may have one of the other cars (within the video presentation of the game) crash, causing a warning flag to force all racers to slow down, or a dog can come onto the roadway, or a herd of cattle, or the course can become very twisty, or the course may become strewn with damaged car parts, or the terrain may dip steeply downward, etc. As a result of this change in the “story” of the game, the user will slow down until the user's cadence 36 matches the desired exercise profile. Likewise, in order to artificially create the environment in which it would be appropriate for the user to exercise more vigorously, the game may present an uphill stretch, or one of the other “racers” may pass, or whatever obstacle caused the user to slow down is simply removed. Each scenario act as a portion of a visual story that is orchestrated by the system to provide a particular exercise profile. The intent is to provide a system in which the video presentation is sequenced in order to provide a visual and/or storied context that requires the user to increase or decrease the rate of exercise, thereby allowing the user to experience a user-selected exercise profile. In one embodiment this includes forward motion video that includes alternate paths, selectable by the methods explained in FIG. 1.

[0056] In one embodiment, the user first selects an exercise profile 43. Database 32 includes the artificial scenarios (such as described above) with which to make the exercise easier or more difficult within the context of the video presentation. The host application 11 synthesizes a presentation based on the profile 46. The “story” options available to alter the level of exercise intensity within database 32. Segments are created to have edits points, allowing the length of a particular segment to be increased or decreased as well as increasing and decreasing the intensity, so that any user profile may be accommodated. The system receives input from cadence 36 simultaneously enabling the user to experience a seemingly random presentation which is in fact enabling the user to experience a particular and desired exercise profile. The system endeavors to randomize the events to make the synthesized presentation appear as natural as possible. In one embodiment, the system further includes physiologic input 47 from the user such as pulse rate, allowing the user to select a desired exercise level in profile 43. In this embodiment the synthesized video 46 is created dynamically, selecting video features from database 32 as a consequence of physiologic input 47. The host application 12 therefore provides a visual experience that is perceived as random exercise to a user that follows a pre-desired exercise profile (selected in step 43) in which the users exercise cadence 36 and/or physiologic input 47 is utilized to manufacture (i.e. edit) a visual experience from video presentation contained within database 32.

[0057] The details of implementing the functionality above falls within the abilities of one skilled in the art. Although embodiments of the invention are set forth in the accompanying drawings and the description herein, other features, objects, and advantages of the invention will be apparent from the description and drawings, and from the claims.