COMPUTER IMPLEMENTED METHOD AND WEARABLE ELECTRONIC SYSTEM FOR PREDICTING THE OXYGEN UPTAKE DURING AN EXERCISE, AND, NON-TRANSITORY COMPUTER READABLE STORAGE MEDIUM

Abstract

A computer implemented method for predicting the oxygen uptake (% VO.sub.2Max) during an exercise of a user in proportion to the user's maximum oxygen uptake through a wearable device. The method comprising obtaining information from a user profile, capturing at least three temporal input data from a user during an exercise, using the information from a user profile and the at least three temporal input data to extract features to compose features vectors, identifying the exercise being performed by the user based on the feature vectors, selecting a predictor model to compute predictions for the % VO.sub.2Max from the user based on the exercise being performed, and predicting the % VO.sub.2Max from the user based on the selected predictor model.

Claims

1. A computer implemented method for predicting oxygen uptake (% VO.sub.2Max) during an exercise of a user in proportion to the user's maximum oxygen uptake through a wearable device, the method comprising: obtaining information from a user profile; capturing at least three temporal input data from the user during an exercise; using the information from the user profile and the at least three temporal input data to extract features to compose features vectors, identifying the exercise being performed by the user based on the feature vectors; selecting a predictor model to compute predictions for the % VO.sub.2Max from the user based on the exercise being performed, and predicting the % VO.sub.2Max from the user based on the selected predictor model.

2. The computer implemented method according to claim 1, wherein the obtaining information from user profile comprises collecting at least user data like age, gender, height and weight.

3. The computer implemented method according to claim 1, wherein the capturing at least three temporal input data from the user profile comprises: collecting readings of at least heart rate, speed, or step frequency.

4. The computer implemented method according claim 1, wherein the using the information from the user profile and the at least three temporal input data to determine the features vectors comprises: splitting readings of the at least three temporal input data in time windows of N seconds without overlap to obtain a moving average, discarding temporal data that falls outside a predetermined range to obtain valid temporal data, wherein the predetermined range comprises threshold values based on the moving average of the temporal data, and associating the user profile data with valid temporal data to calculate maximum heart rate (HRMax), estimated heart rate (% HRMax) and Body-Mass-Index (BMI) from the user.

5. The computer implemented method according to claim 1, wherein readings of the at least three temporal input data are sampled at 1 Hz and split into time windows of 30 seconds.

6. The computer implemented method according to claim 1, wherein the at least three temporal input data comprises a temporal signal and a temporal series, wherein a moving average of the temporal signal with a sliding window of M seconds with a stride of N seconds starting in the second A is calculated by using the following equation:
x.sub.A.fwdarw.A+M=x.sub.A.fwdarw.A+N−1+x.sub.A+N.fwdarw.A+2N−1+x.sub.A+2N.fwdarw.A+3N−1+ . . . +x.sub.A+M−N+1.fwdarw.A+M wherein the moving average of the temporal series from the point X to the point Y is calculated by using the following equation: ${\stackrel{\_}{x}}_{X.fwdarw.Y}=\frac{1}{Y-X+1}\underset{i=X}{\overset{Y}{.Math.}}{x}_i$

7. The computer implemented method according to claim 1, wherein the determining the features vectors is carried by: concatenating the user profile data, valid temporal data and the HRMax, % HRMax and BMI data.

8. A method according to claim 1, wherein the identifying the exercise performed by the user comprises: detecting whether a type of exercise is at least one of the scenarios: running scenario, walking scenario or treadmill scenario.

9. The computer implemented method according to claim 1, wherein the selecting the predictor model to compute predictions for the % VO.sub.2Max from the user comprises: choosing at least one machine learning model among a Multi-Layer Perceptron (MLP) regressor for the running scenario, an Extra Trees Regressor for the walking scenario and a Ridge Regressor for the treadmill scenario.

10. The computer implemented method according to claim 1, wherein predicting the % VO.sub.2Max from the user, comprises: associating the feature vectors with the predictor model to determine the % VO.sub.2Max from the user, wherein a new prediction is output provided a heart rate deviation does not exceed a predetermined instability threshold, or a last stable prediction is output provided the heart rate deviation exceeds a predetermined instability threshold.

11. A wearable electronic system for predicting the oxygen uptake (% VO.sub.2Max) during an exercise of a user in proportion to the user's maximum oxygen uptake through a wearable device, comprising: a processor; a memory including computer readable instructions that, when executed by the processor, causes the processor to perform the method as defined in claim 1.

12. The wearable electronic system according to claim 11, wherein the wearable electronic system is a smartwatch, smart band, fitness tracker, smart clothing or body sensors.

13. A non-transitory computer readable storage medium, which stores computer readable instructions which, when executed by a processor, causes the processor to perform the method as defined claim 1.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The invention is explained in greater detail below on the basis of figures. Shown therein are:

[0034] FIG. 1 shows an overview how the steps of predicting the oxygen uptake (% VO.sub.2Max) during an exercise of a user in proportion to their maximum through a wearable device are performed, according to an embodiment of the present invention.

[0035] FIG. 2 shows with more detail the prediction pipeline of the proposed method, according to an embodiment of the present invention.

DETAILED DESCRIPTION OF THE INVENTION

[0036] The present invention aims at providing a new method for predicting the oxygen uptake (% VO.sub.2Max) during an exercise of a user in proportion to their maximum through a wearable device.

[0037] For this, we propose a new method to assist users in improving their cardio-respiratory and endurance capacity as well as monitoring their energy expenditure during physical exercises, capable of estimating in real time a fitness measurement known as % VO.sub.2Max without requiring an exhaustive test nor expensive equipment. VO.sub.2 measures the amount of oxygen a person inhales during a single breath, in ml/min/kg.

[0038] The proposed method is also useful to monitor exhaustion levels of workers at the workplace (the International Labor Organization suggests a break should be enforced when the worker exceeds a predefined threshold of % VO.sub.2max). This method is not a direct measurement of the value of interest, but an estimation based on a combination of computations over indirect biological signals to extrapolate the value of interest and, as such, requires a refined process to match the operational parameters.

[0039] The method uses machine learning to detect patterns to compute predictions, running under restricted memory footprint and using signals that are already available in the wearable (speed, step frequency and heart rate) resulting in a small requirement for execution. Overall, this method is to achieve good estimations using machine learning and signal processing, requiring less than 10 KB of memory.

[0040] The machine learning models were trained using a proprietary dataset that allowed understanding correlations between VO.sub.2 information collected by professional equipment and information from wearable sensors.

[0041] The machine learning ensemble to estimate % VO.sub.2Max, relying on pattern detection to improve the overall performance in different demographics.

[0042] In order to achieve a more precise estimation of % VO.sub.2Max, FIG. 1 shows the computer implemented method proposed by the invention. First, the information from user profile 101, 202 is obtained, which includes age (in years old), gender (male/female), height (in centimeters) and weight (in kilograms). Second, it is captured at least three temporal inputs 102, 201 composed of readings of heart rate, speed (estimated using GPS or wearable device pedometer) and step frequency (estimated from the wearable device accelerometer) during an exercise.

[0043] The information obtained from the user profile 101, 202 and the temporal inputs 102, 201 will be used to extract features 103 to composed features vectors 104. The features will be extracted by splitting the temporal inputs 102, 201 in time-windows of N seconds without overlap (e.g. 30 seconds), which are processed by a feature extraction module 103 to obtain a moving average. The feature extraction module is assigned with discarding windows for which the heart rate, speed and step frequency averages fall outside a predefined range, as well as computing new hand-engineered features such as HRMax (the theoretical maximum heart rate, estimated from the user's age), % HRmax (the HR in proportion to HRMax) and BMI (Body-Mass-Index, computed from the user's height and weight).

[0044] In order to reduce memory consumption and calculate the mean from the time series, instead of storing M minutes of the signal sampled at 1 Hz we just use the sum of the signal and the sum of the squared signal.

[0045] Considering that each temporal data window has N seconds. It is possible to calculate the moving average, defined by:

x.sub.A.fwdarw.A+M  (1)

that is, the mean of the temporal signal with a sliding window of M seconds with a stride of N seconds starting in the second A by using the equation defined by:

x.sub.A.fwdarw.A+M=x.sub.A.fwdarw.A+N−1+x.sub.A+N.fwdarw.A+2N−1+x.sub.A+2N.fwdarw.A+3N−1+ . . . +x.sub.A+M−N+1.fwdarw.A+M  (2)

[0046] Where the mean of the temporal series x.sub.X.fwdarw.Y from the point X to the point Y is defined by:

[00001]$\begin{array}{cc}{\stackrel{\_}{x}}_{X.fwdarw.Y}=\frac{1}{Y-X+1}{.Math.}_{i=X}^Y{x}_i& \left(3\right)\end{array}$

by using this representation, it is possible to calculate the moving average just storing M/N variables and obtain a valid temporal data.

[0047] Therefore, the feature vectors 104 will be determined by concatenating the profile data 101, 202 with the valid temporal data and the HRMax, % HRMax and BMI data, which will be fed to trained machine learning methods 106.

[0048] These feature vectors are used as input for an ensemble composed of three machine learning models, each tailored for a specific scenario and depending on whether the user is walking or running and whether the user is exercising on a treadmill or outdoor 105, 205 a specific method will be used to compute predictions for the user's % VO.sub.2Max 107, 209. For example, but not limited to:

[0049] 1. A Multi-Layer Perceptron (MLP) regressor for the running scenario.

[0050] 2. An Extra Trees regressor for the walking scenario.

[0051] 3. A Ridge regressor for the treadmill scenario.

[0052] Finally, the % VO.sub.2Max from the user will be predicted 107, 209 based on the selected predictor model 106 and depending on whether the heart rate deviation 208 exceeds a predetermined ‘instability’ threshold, new prediction 207 or the last stable prediction 206 will be outputted 107, 209.

[0053] With the % VO.sub.2Max information in the smartwatch 108, the user can monitor the energy expenditure of a physical exercise in real-time in order to implement improvements and possible adaptations in her or his training. It is an important feedback for the efforts and habits carried out regularly.

[0054] FIG. 2 explains with more detail the prediction pipeline of the proposed method. The diagram illustrates the sequency of a signal processing loop repeated every n seconds. The inputs consist in Temporal data 201 containing sequences of n values for HR, Speed and Step Frequency, Profile data 202 containing Age, Gender, Height and Weight information from the user and the last prediction 203. The first step is to check 204 whether HR, Speed and Step Frequency values are within their respective predetermined bounds and wait for the next window of n values in case not. The model invoked for the estimation (e.g., MLP, Extra Trees or Linear Regression) is dependent on whether the user is running, walking or on a treadmill 205. Finally, there is a post-processing step 208 to decide whether to output the new prediction 207 or to repeat the last stable prediction 206, depending on whether the HR deviation exceeds a predetermined ‘instability’ threshold.

[0055] The proposed approach is composed by the fusion of three different models, each fine-tuned to a specific target scenario (running, walking and treadmill exercise). Beyond the model fusion, the approach introduces novel pre and post-processing steps tailored to each scenario to avoid the influence of bad-quality input data and leverage past predictions to mitigate prediction errors in real-time. The individual models were chosen through a comparative analysis of a set of machine learning techniques including linear regression methods, tree, ensemble-based methods and multilayer perceptron-based methods.

[0056] The Running model is adapted to the scenario of movement at a running pace in an outdoor environment. In the target scenario, the Speed feature will be computed from the GPS signal of wearable devices, like smartwatches and smartbands. Models are compared based on the simple mean between the SP1 and SP2 Mean Average Percentage Error (MAPE) metric, and the model with the lowest error was chosen.

[0057] The Walking model is adapted to the scenario of movement at a walking pace in an outdoor environment. In the target scenario, the Speed feature will be computed from the GPS signal of wearable devices, like smartwatches and smartbands. Models are compared based on walking Mean Average Percentage Error (MAPE) metric, and the model with the lowest error was chosen.

[0058] The Treadmill model is adapted to the scenario of movement at a walking to running pace on a treadmill, context in which the GPS speed is unavailable. For that reason, in the target as in the training scenario, the Speed feature is provided by the Step Frequency estimate, which is computed from accelerometer data from of wearable devices, like smartwatches and smartbands, and was registered during the data-collection process, in which volunteers are subject to running or walking protocols on a treadmill. Models are compared based on the simple mean between SP1, SP2 and walking Mean Average Percentage Error (MAPE) metric, and the model with the lowest error was chosen.

[0059] Among the potential models, the following can be highlighted:

[0060] The Multi-Layer Perceptron (MLP) model consists of a series of fully connected layers of neurons arranged in a feedforward fashion, where each neuron is coupled with a possibly nonlinear activation function. Concretely, the intermediate result of each layer can be expressed by an affine transformation:

A{right arrow over (x)}+{right arrow over (b)}  (4)

over the layer's inputs z, composed with an activation function (in our case the activation function corresponds to:

ReLU(x)=max(0,x)  (5)

[0061] The series of A and {right arrow over (b)} for all layers corresponds to the trainable parameters of this machine learning method.

[0062] The hyperparameters of the Multi-Layer Perceptron (MLP) were fine-tuned through an hyperparameter search process, yielding a fully connected neural network with 2 hidden layers of 15 hidden neurons each and ReLU activations on all neurons but the output.

[0063] The Extra Trees model computes the % VO.sub.2max prediction as the simple mean of 20 outputs, each computed by a different regression tree. Each tree receives the entire vector of input features z and outputs a prediction. Each leaf node of each tree is associated with a single floating-point number. The tree is traversed from the root node to a leaf through a series of threshold tests on one of the input attributes.

[0064] The Ridge model computes the % VO.sub.2max prediction through a simple linear equation:

VO.sub.2Max=−1.005+0.002×Age+0.685×Gender+0.004×Height−0.002×Weight+0.004×HR+0.21×Step Freq.+0.007×BMI−0.001×HRMax+0.008×% HRMax  (6)

Effect

[0065] A dataset was collected with high heterogeneity, considering a large variation in terms of user profiles and physical conditions. The data gathering protocol was defined in collaboration with health domain specialists in order to guarantee that the oxygen consumption (VO.sub.2) and heart rate variations could be achieved during the execution of the exercises by the subjects.

[0066] All selected subjects had a minimum capacity to perform physical activities and the data gathering sessions happened in an indoor environment.

[0067] For stimulating the subjects to achieve different levels of their maximum oxygen consumption and heart rate and allow us to obtain useful data for the method of the present invention, it is defined a data gathering protocol with two steps. The first step is a maximal test using an incremental intensity running to determine the subject's VO.sub.2Max. The second step is a submaximal test to emulate possible regular running conditions.

[0068] During the exercises in each protocol, we get the demographic information of the subjects and collected data from wearable devices, including different models of smartwatches and reference equipment.

[0069] The collected dataset comprises hundreds of subjects (females and males) with high variability in terms of weight, height, age, skin color and physical conditions.

[0070] The innovation of the present invention is a new approach for % VO.sub.2Max estimation based on wearable sensors available in smartwatches, which uses an ensemble of machine learning approaches and introduces new ideas that lead in precise estimations and compact models. The solution of the present invention requires less than 10 KB of memory and was embedded in the microcontroller of smartwatches and smartbands.

Hardware Implementation

[0071] The example embodiments described herein may be implemented using hardware, software or any combination thereof and may be implemented in one or more computer systems or other processing systems. Additionally, one or more of the steps described in the example embodiments herein may be implemented, at least in part, by machines.

[0072] Examples of machines that may be useful for performing the operations of the example embodiments herein includes smart bands, smartwatches, fitness trackers, smart clothing, body sensors and other wearable devices.

[0073] For instance, one illustrative example system for performing the operations of the embodiments herein may include one or more components, such as one or more microprocessors, for performing the arithmetic and/or logical operations required for program execution, and storage media, such as one or more disk drives or memory cards (e.g., flash memory) for program and data storage, and a random-access memory, for temporary data and program instruction storage.

[0074] As is well known in the art, microprocessors can run different operating systems, and can contain different types of software, each type being devoted to a different function, such as handling and managing data/information from a particular source or transforming data/information from one format into another format. The embodiments described herein are not to be construed as being limited for use with any particular type of wearable device, and that any other suitable type of device for facilitating the exchange and storage of information may be employed instead.

[0075] Software embodiments of the illustrative example embodiments presented herein may be provided as a computer program product, or software, that may include an article of manufacture on a machine-accessible or non-transitory computer-readable medium (also referred to as “machine-readable medium”) having instructions. The instructions on the machine accessible or machine-readable medium may be used to program a computer system or other electronic device. The machine-readable medium may include, but is not limited to, floppy diskettes, optical disks, CD-ROMs, and magneto-optical disks or other type of media/machine-readable medium suitable for storing or transmitting electronic instructions.

[0076] The storage medium comprising computer readable instructions that, when performed by the processor, causes the processor to perform the method steps previously described in this disclosure.

[0077] The techniques described herein are not limited to any particular software configuration. They may be applicable in any computing or processing environment. The terms “machine-accessible medium”, “machine-readable medium” and “computer-readable medium” used herein shall include any non-transitory medium that is capable of storing, encoding, or transmitting a sequence of instructions for execution by the machine (e.g., a CPU or other type of processing device) and that cause the machine to perform any one of the methods described herein. Furthermore, it is common in the art to speak of software, in one form or another (e.g., program, procedure, process, application, module, unit, logic, and so on) as taking an action or causing a result. Such expressions are merely a shorthand way of stating that the execution of the software by a processing system causes the processor to perform an action to produce a result.

Technical Effects Achieved by the Invention

[0078] The proposed technique has several advantages for embedded application in wearable devices, such as smartwatches. The method of the present invention can obtain a reasonably accurate prediction (i.e. with a mean absolute error lower than 3 ml/min/Kg) of % VO.sub.2Max using only data and sensors available in most smartwatches, such as profile data, heart rate (HR) based on photoplethysmogram (PPG) sensor, speed computed using accelerometer sensor or Global Positioning System (GPS) libraries and step frequency estimated using accelerometer. Another advantage is that the method does not require maximal exertion on a treadmill or on a cycle ergometer, allowing the user to walk or run at the location of her or his preference and at the pace at which she or he usually trains.

[0079] The proposed method is made possible by key innovations to 1) perform on a restricted memory environment; 2) estimate the real-time % VO.sub.2max for each time window with a pre-defined duration (e.g. 30 seconds); 3) be resilient to noise and unreliable data in the input signals and 4) adapt the estimation procedure to the user's target scenario. This is accomplished by developing an ensemble of diverse and lightweight models, each tailored to a particular scenario (running, walking or treadmill) performed in the user's own pace and custom pre and post-processing treatments to filter out noisy or unreliable data before computing predictions as well as a strategy to mitigate the effects of device malfunctions (such as bad contact between the user's skin and the smartwatch).