SYSTEM AND METHOD FOR MULTI-SENSOR COMBINATION FOR INDIRECT SPORT ASSESSMENT AND CLASSIFICATION
20220260442 · 2022-08-18
Inventors
Cpc classification
A43B3/44
HUMAN NECESSITIES
International classification
Abstract
A system for measuring power output of a runner is disclosed. In some embodiments the system comprises a first sensor component including a first sensor, microprocessor, and a signal transceiver; a second sensor component including a second sensor and a signal transmitter; wherein the first sensor is configured to measure a vertical velocity and horizontal velocity, the second sensor is configured to measure the slope angle of a foot of the runner during a stance phase of the foot, the signal transmitter configured to send slope angle data, the signal transceiver configured to receive the slope angle data from the signal transmitter, and the microprocessor has computing instructions configured to calculate a power output based on the vertical velocity, horizontal velocity, and slope angle data.
Claims
1. A method for measuring a power output of a runner, comprising: attaching a single sensor at a single sensor position on the runner; detecting slope changes of sensor data from said single sensor; and retroactively applying said slope changes to spatio-temporal data to measure the power output; wherein said single sensor comprises an inertial measurement unit (IMU).
2. The method of claim 1, wherein said single sensor position is selected from the group consisting of a foot, a wrist or a head of the runner.
3. The method of claim 1, wherein said spatio-temporal data comprises data relating to ground contact time, flight phase duration, swing phase duration, and cadence.
4. The method of claim 3, wherein foot pronation angle and foot strike angle are calculated according to said sensor data.
5. The method of claim 1, wherein the power output is calculated according to both initial contact and terminal contact events in footfall of the runner.
6. The method of claim 5, further comprising performing an initialization method to remove an influence of an orientation of the sensor on accuracy of the power output calculation, wherein said initialization method comprises detecting initial contact and terminal contact of the foot with the ground, and determining said orientation of the sensor before power output is calculated.
7. The method of claim 1, wherein said sensor comprises a sensor assembly, wherein said sensor assembly comprises a single IMU, a microcontroller, and a wireless communications device to transmit sensor data externally.
8. The method of claim 1, further comprising analyzing said sensor data from said single sensor by a machine learning algorithm; wherein said machine learning algorithm is trained according to a combination of data from a force plate on a shoe worn by the runner and said sensor data from said single sensor.
9. The method of claim 8, wherein said single sensor or a separate wearable device worn by the runner comprises a self-learning power meter, wherein said power meter comprises a microprocessor and an embedded machine learning library, wherein said machine learning algorithm is executed by said power meter for analyzing said sensor data from said single sensor.
10. The method of claim 9, wherein said machine learning algorithm is selected from the group consisting of an LSTM (long short-term memory) network; an RNN (recurrent neural network); a CNN (convoluted neural network); and an MNN (modular neural network).
11. A system for measuring the power output of a runner, comprising: a sensor configured to detect slope changes and attached to or worn by the runner; and a computing device having a processor and a memory for storing buffered data from said sensor for a predetermined number of sampling cycles and having stored thereon instructions for execution by a processor to cause the computational device to receive data sampled from said sensor; during an activity of the user, to retroactively apply slope measurement to spatio-temporal data; and to estimate a power output based in part on the slope trajectory.
12. The system of claim 11, wherein said sensor comprises a sensor assembly, wherein said sensor assembly comprises a single IMU, a microcontroller, and a wireless communications device to transmit sensor data externally.
13. The system of claim 12, wherein said single sensor position is selected from the group consisting of a foot, a wrist or a head of the runner.
14. The system of claim 13, wherein said sensor is integrated as part of a shoe.
15. The system of claim 14, further comprising a machine learning algorithm; wherein said machine learning algorithm is trained according to a combination of data from a force plate on a shoe worn by the runner and said sensor data from said single sensor; wherein said machine learning algorithm analyzes said sensor to estimate said power output.
16. The system of claim 11, wherein said single sensor comprises a self-learning power meter, wherein said power meter comprises a microprocessor and an embedded machine learning library, wherein said machine learning algorithm is executed by said power meter for analyzing said sensor data from said single sensor.
17. The system of claim 16, wherein said machine learning algorithm is selected from the group consisting of an LSTM (long short-term memory) network; an RNN (recurrent neural network); a CNN (convoluted neural network); and an MNN (modular neural network).
18. The system of claim 11, further comprising a separate wearable device worn by the runner, said separate wearable device comprising a self-learning power meter, wherein said power meter comprises a microprocessor and an embedded machine learning library, wherein said machine learning algorithm is executed by said power meter for analyzing said sensor data from said single sensor.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
[0034] Embodiments of the disclosure are herein described, by way of example only, with reference to the accompanying drawings. With specific reference now to the drawings in detail, it is stressed that particulars shown are by way of example and for purposes of illustrative discussion of the various embodiments of the present disclosure only and are presented in order to provide what is believed to be a useful and readily understood description of the principles and conceptual aspects of the various embodiments of inventions disclosed therein.
[0035]
[0036]
[0037]
[0038]
[0039]
[0040]
[0041]
[0042]
[0043]
[0044]
DETAILED DESCRIPTION OF SOME OF THE EMBODIMENTS
[0045] Preferred embodiments include a research-grade power meter software library customized for running, to be embedded in wearable devices of various kinds (earphones, smartwatches, smartshoes, smartphones, etc.). Its innovation comes from its scientific approach and validity, and the use of multiple sensors at different body locations to derive the most accurate power estimation of running. Preferred embodiments can also include be used with existing sensor systems or assemblies or more simplified sensor systems or assemblies.
[0046] Preferred embodiments can estimate running power in every condition, such as indoor and outdoor running, on road, hill, trails, etc. using a complete data collection protocol covering the various situations and providing sufficient data as input to power output estimation models. Some preferred embodiments use quality reference systems and meta-data about the user such as age and gender to properly estimate physiological responses.
[0047] Additionally, preferred embodiments include real-time on-board processing to provide useful feedback during running and create value propositions for consumer wearables. Sensor systems in accordance with preferred embodiments can estimate power output in real-time with a minimal latency. In addition, preferred embodiments include on-board, real-time feature processing at the sensor level and minimal buffers and processing to limit battery leakage.
[0048] Preferred embodiments model and validate the instantaneous power measured using inertial sensors. The signal processing in preferred embodiments is performed in real-time and the ground reaction forces acting on the runner's body is modeled using the acceleration and angular velocity obtained from body-worn sensors. Moreover, spatio-temporal parameters are used for real-time estimation of mechanical power, but also to give direct feedback with parameters that could be modified by the athlete (e.g., changing cadence, stride length, etc.).
[0049] Referring now to the figures,
[0050] Computing device 110 includes machine learning engine and feedback engine. Machine learning engine can include an embedded machine learning library. The machine learning library can be implemented according to different AI techniques known in the art. For example, some non-limiting suitable techniques include LSTM networks; various types of RNN (recurrent neural network) such as a Siamese RNN; a CNN (convoluted neural network); and an MNN (modular neural network).
[0051] Feedback engine 120 can include machine instructions for creating feedback messages to the runner based on the computed power output from machine learning engine or changes in the rate of power output as determined in the machine learning engine.
[0052] Power data storage device 130 can store raw velocity, slope, and other measurements from sensor arrays as well as power computations from machine learning engine 115. Computing device 110 can be implemented using standard micro-controller processors such as the family of ARM M4 Cortex architectures.
[0053] Sensor assemblies 140, 160 include sensor arrays 145, 165. Sensor 145, 165 arrays can each be adapted to placement on different body parts of the runner. For example, a sensor array can include only IMUs to measure velocity, can include only a force sensor, accelerometer, or barometric pressure sensor to measure gait phase from a location on or near the runner's foot. Each of sensor assemblies include a wireless transceiver to send measurement data signals to sensor assembly. Sensor assembly includes a wireless transceiver to receive the measurement data signals from sensor assemblies for processing by machine learning engine 115.
[0054] Other preferred embodiments include machine instructions without a machine learning engine 115 for computing power output directly from measurements data signals received from sensor arrays.
[0055]
[0056]
[0057] If the user is on stairs, then at step 308, the slope data is adjusted for stair terrain. Additionally, in some embodiments, heel strike, heel off, and toe strike data are removed from the sensor data. Heal strike, heal off, and toe strike data are not as relevant when the user is on stairs because of how gait is adjusted to accommodate the flat surface of a stair. If those temporal phases are included, then gait analysis and timing could be inaccurate because those phases do not exist on stair terrain in the same way as for non-stair terrain. If the user is not on stairs, then at step 310, the slope data is adjusted for non-stair terrain. At step 312, the stride length and velocity are calculated based on the slope trajectory or slope estimation for later input into a power output estimation. Inertial data typically indicates gait phases inaccurately. Consequently, in some preferred embodiments, gait phase data (e.g., stride length) and velocity data are adjusted by calculating the interrupt of inertial data through the slope trajectory. At step 314, abrupt slope change determination data is buffered. Slope change data can be derived from pressure sensor data changes (e.g., barometric pressure), topographical data, infrared time-of-flight sensor data, and the like, as well as foot orientation. The data can indicate a slope change when there is only a temporary change in the terrain and not necessarily from an actual slope change. For example, a user can be running over a protruding rock, log, or bump which causes the foot to be inclined at a greater angle than the overall current terrain would suggest. Additionally, because the runner can be elevated according to the terrain protrusion, barometric data can also suggest a rising slope. However, over just a few cycles the data will revert back to indicating a lack of change in the overall slope or only slight changes in the slope. Therefore, in preferred embodiments, abrupt slope changes are buffered for a number of cycles for comparison with slope data from later cycles. An abrupt slope change can be a change in slope that results in a change to the power output but a change in heart rate or other biosignals lags. In some embodiments, a parameter indicating whether a particular slope change is abrupt can be user-defined. If slope data from later cycles confirms the new slope, then an abrupt change can be determined and be used as an input to the power output estimation. Buffered abrupt slope change determination data is maintained preferably for 3-5 cycles. It should be understood that the embodiments other than a whole-body kinetics embodiment can use methods similar to the method exemplified in
[0058] Buffering abrupt slope change data is important to achieve superior power estimation results. As discussed herein, biometric data that would indicate slope change and, thus, increased power output, typically lags actual slope change. Additionally, sensor data typically used to determine slope changes (e.g., barometer, topographic information, etc.) may present inaccuracy or similarly lag. To increase accuracy in determining slope changes and particularly, abrupt slope changes, foot orientation is included in the determination. Foot orientation angles change immediately upon a slope change. However, foot orientation can present a false positive when the user's foot is on a temporary protrusion or a false negative when the user is on stairs. To account for the false positives and negatives, preferred embodiments will sample both traditional types of sensor data and foot orientation and buffer them both over a number of cycles as discussed herein so that proper and more accurate power output estimates can be calculated in the event the user remains on a similar slope and encountered a protrusion (e.g., pressure change rate is steady but foot orientation indicates abrupt change—false positive), is on stairs (e.g., pressure change rate increases but foot orientation indicates essentially flat terrain—false negative), or has encountered an abrupt slope change (e.g., both pressure change rate increases and foot orientation indicates abrupt change).
[0059] Thus, preferred embodiments estimate power while accounting for abrupt slope changes without the lag inherent in previous power estimation devices and systems. This is important for assisting users to maintain a proper pace during the slope changes to maximize power efficiency, as illustrated in
[0060]
[0061]
[0062] If the slope angle is not greater than 0°, then at step 370 it is determined whether the change in slope is greater than the slope change threshold. In preferred embodiments, the slope change threshold used in step 370 is the same as the slope change threshold used in step 364. If the slope change is beyond the slope change threshold, then at step 372, the terrain is classified as “down stair” and the slope counter is decremented. In a preferred embodiment, the slope counter is decremented by a predetermined amount. If the slope change is not beyond the slope change threshold, then at step 374, the terrain is classified as “down slope” and decremented. In a preferred embodiment, the slope counter is decremented by a predetermined amount. It should be understood that the embodiments other than a whole-body kinetics embodiment can use methods similar to the method exemplified in
[0063] Different methods have been proposed for calculation of mechanical power during running. Power estimation is based on the estimation of the product of force-velocity or moment-angular velocity. The common reference method to estimate power is based on force plate. The ground reaction force during stance phase F.sub.GRF is used as the force acting on CoM and the power (P.sub.GRF) is estimated from the integration of CoM acceleration obtained through F.sub.GRF as follow:
[0064] With v.sub.0 an integration constant corresponding to the mean velocity of the runner of mass M. Skilled artisans can appreciate that as compared to other methods, e.g., using multi-segment kinematics, a whole-body kinetics method shows good behavior and correspondence with oxygen uptake.
Whole-Body Kinetic Embodiment
[0065]
[0066] At least one sensor assembly (for example, sensor 415) is located on the body so that the ground inclination during a run can be measured. Preferably, such a sensor assembly is located on the shoe or at the foot and includes an IMU, accelerometer, barometric pressure sensor, some other sensor, or a combination thereof configured to measure the inclination of the slope during the stance phase of the runner's gait as described herein. One or more of the sensors can be used to measure foot orientation for estimating slope. It is important, particularly for more advanced users, that the weight and size of a sensor assembly at the foot be as small and light as possible. Thus, in preferred embodiments, the architecture of a sensor assembly mounted at the foot is simplified and efficient by using a smaller microprocessor and memory, only essential sensors, and less PCB material. For example, in one preferred embodiment, a sensor assembly can include a single IMU, a microcontroller, and a wireless communications device to send data to another assembly or device. To further reduce mass of the sensor assembly, the components or packaging of components can be integrated or be made as part of footwear or clothing at the foot or area of the foot. Placing a sensor at the foot as opposed to elsewhere on the body can allow for drift correction and for measuring the foot-ground interface (e.g., ground slope and other interface characteristics). Sensor arrangements similar to the arrangement of
[0067]
P.sub.ToT=F.sub.Hv.sub.H+Mgv.sub.v sin sin α Eq. (2)
[0068] Force and velocity can be estimated from body acceleration measured with a sensor placed on waist (close to CoM):
F.sub.H=MA.sub.H Eq. (3)
v.sub.H=v.sub.0+∫ A.sub.Hdt Eq. (4)
and
v.sub.V=v′.sub.0+∫ A.sub.Vdt Eq. (5)
and the slope can be estimated from the foot inclination during stance phase using shoe accelerometer or barometric pressure sensor. Force can be estimated at the sensor assembly measuring acceleration or at another sensor assembly or computing device.
[0069] Total power estimations are computed at predetermined intervals. In some embodiments, total power estimations can be calculated when some triggering event occurs such as when a velocity, acceleration, or force changes or reaches a threshold. According to preferred embodiments, total power estimates can be calculated and changes in total power can be determined. At step 560, a feedback indicator is determined. At step 570, a feedback indicator is displayed to the runner. Such a feedback indicator can be based on the total power estimate or a change in the rate of the total power estimate to indicate to the runner increasing or decreasing power output during a session. Additionally, according to preferred embodiments a total power estimate that exceeds or falls below a predetermined threshold can trigger a feedback indicator. In some embodiments, a feedback indicator can take the form or a display element on a smartwatch, smartphone, smart glasses, and the like. In other embodiments, a feedback indicator can be audial or pressure indicators.
[0070] According to some preferred embodiments, an initialization step is included to calibrate the sensors and test their alignment according to the anatomical frame of the runner. Calibration of the sensors can be performed at each use and alignment initialization can be performed once to determine the best or a sufficient location of a sensor or at each use.
[0071]
[0072]
P.sub.ToT=Σ l.sub.jw.sub.j Eq. (6)
[0073] At step 760, a feedback indicator is determined. At step 770, the feedback indicator is presented to the runner or some other user.
[0074] In some embodiments, estimating power output can use machine learning in accordance with preferred embodiments. Here we consider also several body segments but instead of using the above equation we will use machine learning techniques to estimate P.sub.ToT. Embodiments can use reference data (from a force plate, for example) for a learning phase. Here, a sensor is located on the foot, the sensor including a force plate that measures the forces on the foot as it strikes and releases from the fly through the stance gait phases.
[0075] Preferred embodiments, including embodiments described herein, can be used to detect, measure, and report a user's training or activity progress, fatigue, or injury risks. Embodiments can be used to provide real-time feedback on performance and economy (i.e., feedback during an activity). Embodiments can also be used to measure and follow speed and pace changes of racers. Such embodiments can be useful during broadcasts of races. Embodiments of the present invention can obtain data at a greater granularity than current devices, including: [0076] Data at each stride (˜11′000) [0077] Various parameters (VO2, kinematics) [0078] Real-time data [0079] 3 ms Accuracy
[0080] Embodiments can be used to understand runner profiles for better experience of purchasing equipment, shoes in particular, and generate more brand loyalty.
[0081] Machine learning could overcome some drawbacks of a biomechanical model where many segments are necessary to estimate accurately the P.sub.ToT. Here we exploit some correlation/association between segments to minimize the number of segments and consequently the number of sensors. In embodiments implementing a machine learning technique, one of the sensors or another wearable device is a self-learning power meter equipped with a microprocessor and an embedded machine learning library. The machine learning library can be implemented according to different AI techniques known in the art. For example, some non-limiting suitable techniques include LSTM networks; various types of RNN (recurrent neural network) such as a Siamese RNN; a CNN (convoluted neural network); and an MNN (modular neural network).
[0082] In preferred embodiments, the machine learning library would be incorporated in a wearable device.
[0083] Any and all references to publications or other documents, including but not limited to, patents, patent applications, articles, webpages, books, etc., presented in the present application, are herein incorporated by reference in their entirety.
[0084] Example embodiments of the devices, systems and methods have been described herein. As noted elsewhere, these embodiments have been described for illustrative purposes only and are not limiting. Other embodiments are possible and are covered by the disclosure, which will be apparent from the teachings contained herein. Thus, the breadth and scope of the disclosure should not be limited by any of the above-described embodiments but should be defined only in accordance with claims supported by the present disclosure and their equivalents. Moreover, embodiments of the subject disclosure may include methods, systems and apparatuses which may further include any and all elements from any other disclosed methods, systems, and apparatuses, including any and all elements corresponding to target particle separation, focusing/concentration. In other words, elements from one or another disclosed embodiment may be interchangeable with elements from other disclosed embodiments. In addition, one or more features/elements of disclosed embodiments may be removed and still result in patentable subject matter (and thus, resulting in yet more embodiments of the subject disclosure). Correspondingly, some embodiments of the present disclosure may be patentably distinct from one and/or another reference by specifically lacking one or more elements/features. In other words, claims to certain embodiments may contain negative limitation to specifically exclude one or more elements/features resulting in embodiments which are patentably distinct from the prior art which include such features/elements.