METHOD AND SYSTEM FOR DETERMINING A CORRECT REPRODUCTION OF A MOVEMENT
20230004236 · 2023-01-05
Inventors
- André BRANQUINHO GOMES (Aveiro, PT)
- Ana Clara FERREIRA MATOS (Porto, PT)
- Filipe TAVARES DOS SANTOS (Porto, PT)
- Luís Ungaro PINTO COELHO (Porto, PT)
- Virgílio António FERRO BENTO (Porto, PT)
Cpc classification
G06F3/0346
PHYSICS
International classification
G09G5/00
PHYSICS
G06F3/0346
PHYSICS
G06F3/038
PHYSICS
Abstract
Method for determining a correct reproduction of a movement of a target based on a plurality of orientations thereof at different time instants at least including first and second time instants, the second time instant being posterior to the first time instant, the movement being defined by at least a first predetermined constraint, the first predetermined constraint being defined for first and second orientations of the plurality of orientations and defined by a start angle, an end angle and a first plane definition, comprising: providing a first plane and a second plane, each defined by the first plane definition, corresponding to the first and second time instants, respectively; providing a first pair of vectors by projecting the first orientation and the second orientation, corresponding to the first time instant, onto the first plane; providing a second pair of vectors by projecting the first orientation and the second orientation, corresponding to the second time instant, onto the second plane; computing first and second angles between the pair of vectors of the first and second pairs of vectors, respectively; and determining the correct reproduction of the movement if: the first angle is equal to or less than the start angle, and the second angle is equal to or greater than the end angle.
Claims
1.-14. (canceled)
15. A computer-implemented method for determining a reproduction of a movement of a user according to a predetermined movement constraint for an exercise, comprising: (a) receiving, from a motion tracking system, first and second orientations at a first time and third and fourth orientations at a second time; (b) projecting the first and second orientations at the first time onto a first plane; (c) projecting the third and fourth orientations at the second time onto a second plane; (d) computing a first calculation based on the first and second orientations projected onto the first plane and a second calculation based on the third and fourth orientations projected onto the second plane; and (e) determining the reproduction of the movement based at least in part on a comparison of the first calculation and the second calculation with the predetermined movement constraint.
16. The computer-implemented method of claim 15, wherein the first and second orientations are projected onto the first plane as a first pair of vectors.
17. The computer-implemented method of claim 16, wherein the third and fourth orientations are projected onto the second plane as a second pair of vectors.
18. The computer-implemented method of claim 17, wherein computing the first calculation comprises computing a first angle between the first pair of vectors and computing the second calculation comprises computing a second angle between the second pair of vectors.
19. The computer-implemented method of claim 15, wherein the predetermined movement constraint comprises a constraint on range of movement for the exercise.
20. The computer-implemented method of claim 19, wherein the constraint on range of movement comprises an angular range or a start angle and an end angle that limit the range of movement by the user for performing the exercise.
21. The computer-implemented method of claim 19, wherein the predetermined movement constraint further comprises a constraint on one or more accelerations corresponding to the movement of the user.
22. The computer-implemented method of claim 15, wherein the first plane and the second plane each corresponds to an orientation of the user and are defined according to the predetermined movement constraint.
23. The computer-implemented method of claim 22, wherein each the orientation of the user is an orientation of a limb or body segment.
24. The computer-implemented method of claim 23, wherein the limb or body segment comprises a leg, upper arm, lower arm or forearm, head, or torso.
25. The computer-implemented method of claim 15, wherein the exercise comprises a physical therapy exercise.
26. The computer-implemented method of claim 15, wherein the computer-implemented method is performed in real-time.
27. The computer-implemented method of claim 26, further wherein the reproduction of movement is determined to be correct or incorrect.
28. The computer-implemented method of claim 27, wherein the reproduction of movement is determined to be incorrect when the movement does not fulfill the predetermined movement constraint.
29. The computer-implemented method of claim 27, further comprising providing feedback to the user indicating the reproduction of movement is correct or incorrect.
30. The computer-implemented method of claim 29, wherein the feedback indicates what condition of the predetermined movement constraint has not been fulfilled.
31. The computer-implemented method of claim 15, wherein the user is a person or an object.
32. The computer-implemented method of claim 31, wherein the object is a robot, an exoskeleton, or an exosuit.
33. The computer-implemented method of claim 31, wherein the motion tracking system comprises a first sensor adapted to measure the first orientation at the first time and the third orientation at the second time and a second sensor adapted to measure the second orientation at the first time and the fourth orientation at the second time.
34. The computer-implemented method of claim 33, wherein the first sensor and the second sensor each comprise an accelerometer and a gyroscope.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
[0109] To complete the description and in order to provide for a better understanding of the invention, a set of drawings is provided. Said drawings form an integral part of the description and illustrates embodiments of the invention, which should not be interpreted as restricting the scope of the invention, but just as examples of how the invention can be carried out. The drawings comprise the following figures:
[0110]
[0111]
[0112]
[0113]
[0114]
[0115]
[0116]
DESCRIPTION OF WAYS OF CARRYING OUT THE INVENTION
[0117]
[0118] Each limb (21-24) defines one or more orientations (v.sub.1-v.sub.8), for instance but not limited to, as three-dimensional vectors. For example, in
[0119] When carrying out the method of the present disclosure, which in several embodiments is a computer-implemented method, in some embodiments each orientation (v.sub.1-v.sub.8) may be projected onto a plane (10) in accordance with a plane definition of a predetermined constraint. By way of example, the plane (10) of
[0120]
[0121]
[0122]
[0123]
[0124] In one exemplary method explained with reference to
[0125] If only the orientations corresponding to time instants t.sub.1 and t.sub.2 were provided by motion tracking means, the method would determine that the user (20) correctly reproduced the movement. If the motion tracking means also provided the orientations of the user (20) corresponding to time instant t.sub.3, since the angle α.sub.3 is greater than SA.sub.1 and less than EA.sub.1, the method would also determine that the user (20) correctly reproduced the movement. Further, if the motion tracking means also provided the orientations of the user (20) corresponding to time instant t.sub.4, since the angle α.sub.4 is greater than SA.sub.1 and less than EA.sub.1, the method would also determine that the user (20) correctly reproduced the movement.
[0126]
[0127]
[0128] In one exemplary method explained with reference to
[0129] In this example, if only the orientations corresponding to time instants t.sub.1 and t.sub.2 were provided by motion tracking means, the method would determine that the user (20) correctly reproduced the movement since the first and second predetermined constraints are fulfilled (each of α.sub.1, α.sub.2, β.sub.1 and β.sub.2 complies with the conditions of the corresponding predetermined constraint). If the motion tracking means also provided the orientations of the user (20) corresponding to time instant t.sub.3, since the angle β.sub.3 is within AR.sub.1 (and SA.sub.1<α.sub.3<EA.sub.1), the method would also determine that the user (20) correctly reproduced the movement. In contrast, if the motion tracking means also provided the orientations of the user (20) corresponding to time instant t.sub.4, since the angle β.sub.4 is outside AR.sub.1 (and even though SA.sub.1<α.sub.4<EA.sub.1), the method would determine that the user (20) did not correctly reproduced the movement.
[0130]
[0131] The movement is defined by three predetermined constraints (i.e. PC.sub.1, PC.sub.2, PC.sub.3), each defined for a pair of orientations, and defined by a set of angles (SA.sub.1, EA.sub.1, AR.sub.1, AR.sub.2) and a plane definition. Motion tracking means provide orientations of a target in different time instants at least including time instants t.sub.1 to t.sub.5, each in a same global reference frame. Illustrated in the graph are the conditions that the computed angles (α, β, γ) must fulfill for each of PC.sub.1 to PC.sub.3, for the different time instants, in order for the exemplary method to determine that the movement is correctly reproduced by the target.
[0132] In this embodiment, if for example at one time instant taking place before t.sub.2 and after t.sub.1 one of the conditions of any predetermined constraint is not fulfilled, the method determines that the movement is not correctly reproduced and the method is restarted so as to evaluate a subsequent movement of the target. For example, if one of the conditions corresponding to t.sub.4 of PC.sub.1-PC.sub.3 is not fulfilled, the method does not further check whether the conditions corresponding to t.sub.5 and t.sub.2 of the predetermined constraints are fulfilled or not, but rather determines that the movement is not correctly reproduced and checks at a next time instant whether the condition of t.sub.1 is fulfilled or not for the subsequent movement to be reproduced by the target.
[0133] In some examples, it may be determined that the target that was reproducing the movement did not carry out the entire motion of the movement as defined by the first predetermined constraint. Using for instance the example of
[0134] The first predetermined constraint may thus be also defined by the predefined decrease threshold, which is an angular difference that the computed angle(s) between two or more consecutive time instants must reach decreasingly. By way of example, if SA.sub.1 is 30°, EA.sub.1 is 90°, α.sub.3 is 51°, α.sub.4 is 45°, and α.sub.5 is 36°, if the predefined decrease threshold is 14°, then the method would determine that the target did not finish the movement at time instant t.sub.5 because the reduction from α.sub.3 to α.sub.5 exceeds the predefined decrease threshold and because the computed angles α.sub.4, and α.sub.5 are less than previous consecutive computed angles (i.e. α.sub.3>α.sub.4>α.sub.5).
[0135] The feedback produced may reflect that the movement was not correctly reproduced because during reproduction of the same the target performed the movement in the direction reverse to that of the first predetermined constraint.
[0136]
[0137] The movement is defined by three predetermined constraints (i.e. PC.sub.1, PC.sub.2, PC.sub.3), each defined for a pair of orientations, and defined by a set of angles (SA.sub.1, EA.sub.1, AR.sub.1, AR.sub.2) and a plane definition. Motion tracking means provide orientations of a target in different time instants at least including time instants t.sub.1 and t.sub.2, each orientation in a same global reference frame. Illustrated in the graph are the conditions that the computed angles must fulfill for each of PC.sub.1 to PC.sub.3, for the first and second time instants, in order for the exemplary method to determine that the movement is correctly reproduced by the target.
[0138]
[0139] The movement is defined by two predetermined constraints (i.e. PC.sub.1, PC.sub.2), each defined for a pair of orientations, and defined by a set of angles (SA.sub.1, EA.sub.1, AR.sub.1) and a plane definition. Motion tracking means provide orientations of a target in different time instants at least including time instants t.sub.1, t.sub.2 and t.sub.3, each orientation in a same global reference frame. Illustrated in the graph are the conditions that the computed angles must fulfill for each of PC.sub.1 and PC.sub.2, for the first, second and third time instants, in order for the exemplary method to determine that the movement is correctly reproduced by the target.
[0140] Further, if in one exemplary embodiment (for instance as illustrated in
[0141]
[0142] The user (20) moves a right shoulder thereof upwards from a first position, illustrated in
[0143] When the user (20) starts to perform the movement, the shoulder joint (28) accelerates (a.sub.1) in a vertical direction. The user (20) ends the upwards movement of the shoulder upon stopping the vertical displacement of the joint (28), at which point the joint (28) is subject to an acceleration (a.sub.2) in a direction opposite to the direction of the acceleration (a.sub.1) for starting the movement. The accelerations (a.sub.1, a.sub.2) are measurable by, for instance, a sensor of motion tracking means attached to the shoulder or the upper arm; the sensor comprises an accelerometer providing the acceleration measurements including the directions of the measured accelerations.
[0144]
[0145] The movement is defined by three predetermined constraints (i.e. PC.sub.1, PC.sub.2, PC.sub.3): the first one (PC.sub.1) defined for a first acceleration, and by a first direction and both start and end acceleration thresholds (κ.sub.1, κ.sub.2); the second one (PC.sub.2) defined for a pair of orientations and by an angular range (AR.sub.1) and a plane definition; and the third one (PC.sub.3) defined for a second acceleration and by both a second direction and an acceleration interval (AI.sub.1). Illustrated in the graph are the conditions that the measured accelerations and computed angle must fulfill for each of PC.sub.1 to PC.sub.3, for the first and second time instants, in order for the exemplary method to determine that the movement is correctly reproduced by the target.
[0146] Motion tracking means provide the first and second accelerations and the pair of orientations of a target in different time instants at least including time instants t.sub.1 and t.sub.2, each orientation in a same global reference frame. For example, two or more wearable sensors are arranged on an upper arm and a chest of a user such as the user (20) of
[0147] The movement to be performed by the user, according to the three predetermined constraints, is a movement of the shoulder upwards while maintaining the chest relatively steady (accelerations being reduced in a direction parallel to a vertical axis and in directions forming an angle with the vertical axis up to 20° so that the shoulder is not moved due to a movement of the chest) and the upper arm not rotating (or not rotating substantially) relative to the chest. In this sense: the first acceleration measurements (a.sub.1, a.sub.2) correspond to the acceleration that the upper arm has been subjected to at the first and second time instants t.sub.2, respectively, in the first direction; the computed angles (β.sub.1, β.sub.2) correspond to the relative angular differences between the orientations of the upper arm and the chest (and, thus, indicative of the relative angular movement between the two body members) at the first and second time instants (t.sub.1, t.sub.2), respectively; and the second acceleration measurements (b.sub.1, b.sub.2) correspond to the acceleration that the chest has been subjected to at the first and second time instants (t.sub.1, t.sub.2), respectively, in the second direction(s).
[0148] When the user starts the movement, the first acceleration (a.sub.1) at t.sub.1 must be greater than or equal to a start acceleration threshold (Ki, i.e. kappa subindex 1); for this, a device digitally processes the first acceleration so as to establish whether the measured acceleration is in the first direction or comprises a component in the first direction. Also, when the user starts the movement, the computed angle at t.sub.1 shall be within the angular range (AR.sub.1). When the user ends the movement, the first acceleration (a.sub.2) at t.sub.2 must be less than or equal to an end acceleration threshold (κ.sub.2, i.e. kappa subindex 2); for this, the device digitally processes the second acceleration so as to establish whether the measured acceleration is in the second direction or comprises a component in the second direction. Also, when the user ends the movement, the computed angle at t.sub.2 shall be within the angular range (AR.sub.1). Additionally, as explained in more detail with reference to
[0149]
[0150] In the figure a chart is represented with a plurality of discrete values over time. A set of first acceleration ‘a’ values is represented with squares, a set of second acceleration ‘b’ values is represented with circles, and a set of computed angle ‘β’ values is represented with triangles. In the chart are also represented the start and end acceleration thresholds κ.sub.1 and κ.sub.2 (represented with dashed lines), upper and lower angle thresholds A.sub.1 and A.sub.2 (represented with dash-dotted lines) defining the angular range AR.sub.1, and upper and lower acceleration thresholds B.sub.1 and B.sub.2 (represented with dotted lines) defining the acceleration interval AI.sub.1.
[0151] In some embodiments, the magnitude of the acceleration ‘a’ and ‘b’ values corresponds to the norm of the acceleration measurements, yet the sign thereof is maintained based on the direction of the acceleration measurements. In some other embodiments, the magnitude of the acceleration ‘a’ and ‘b’ values corresponds to the acceleration component in the direction of the expected movement, that is to say, it corresponds to the part of the acceleration measurements that is parallel to the expected movement (in accordance with the direction defined in the respective predetermined constraint); the device carrying out the determination of correct reproduction of the movement processes the acceleration measurements so as to compute said acceleration component.
[0152] When the user starts to perform the shoulder movement, the first acceleration ‘a’ values start to raise in the first direction and are positive. At a first time instant 81, the acceleration ‘a’ value is above the start acceleration threshold (κ.sub.1), thus the device carrying out the determination may consider that the user started the movement in accordance with the first predetermined constraint. Based on said predetermined constraint, the user is to end the movement when the acceleration ‘a’ value is below the end acceleration threshold (κ.sub.2), something which occurs at a second time instant 84; prior to reaching that value, the acceleration ‘a’ further increased (after the first time instant 81) and then decreased.
[0153] During the time interval between the first and second time instants 81, 84, the computed angles ‘β’ were within the angular range (AR.sub.1), thereby fulfilling the second predetermined constraint.
[0154] During that same time interval, the second acceleration ‘b’ values in the second direction were within the acceleration interval (AI.sub.1) except for two values occurring at two intermediate time instants 82, 83. In some embodiments, second and/or further predetermined constraints (e.g. PC.sub.2, PC.sub.3, PC.sub.4, etc.) defined for acceleration measurements are also defined by both a percentage threshold and a window size. In these embodiments, the device digitally evaluating the fulfillment of the predetermined constraints provides a sliding window with a size in number of samples equal to the window size, and every time it receives a new acceleration value corresponding to that/those predetermined constraint(s), the window slides so as to encompass the most recent acceleration value while removing the oldest acceleration value inside the window if the window was full (i.e. had as many samples as the window size). The device considers that the predetermined constraint(s) is/are fulfilled if every time the sliding window is filled with samples at least a number N of samples inside the window is within the acceleration interval AI.sub.1, and the ratio N over the window size is equal to or greater than the percentage threshold. Accordingly, if the window size is 10, and the percentage threshold is 70%, at least 7 samples inside the window (while the window is filled with 10 samples) are within AI.sub.1. In some other embodiments, second and/or further predetermined constraints do not have the percentage threshold and the window size defined, thus for fulfilling the corresponding predetermined constraints all the acceleration samples occurring during the movement (in this example, between the first and second time instants 81, 84) need be within the acceleration interval (AI.sub.1).
[0155] In the present example of
[0156] As all three predetermined constraints are fulfilled, the device determines that the user correctly reproduced the movement.
[0157]
[0158] The movement is defined by two predetermined constraints (i.e. PC.sub.1, PC.sub.2): the first one (PC.sub.1) for a pair of orientations, and defined by a set of angles (SA.sub.1, EA.sub.1, AR.sub.1) and a plane definition, and the second one (PC.sub.2) defined for an acceleration and by both a first direction and an acceleration interval (AI.sub.1). Illustrated in the graph are the conditions that the computed angles and the acceleration measurements must fulfill for each of PC.sub.1 and PC.sub.2, for first, second and third time instants in order for the exemplary method to determine that the movement is correctly reproduced by the target. In some examples, the second predetermined constraint (PC.sub.2) is also defined by a percentage threshold and a window size, in which cases it may occur that the acceleration measurements do not have to fall within the acceleration interval (AI.sub.1) in all three time instants (t.sub.1, t.sub.2, t.sub.3).
[0159] Motion tracking means provide the acceleration and the pair of orientations of a target in different time instants at least including time instants t.sub.1, t.sub.2, t.sub.3, each orientation in a same global reference frame. The values of the acceleration measurements (i.e. a.sub.1, a.sub.2, a.sub.3) may be the norm of the acceleration measurements maintaining the sign thereof based on both the direction of the acceleration measurements and the first direction, or the acceleration component in the direction of the expected movement (i.e. the first direction) to be performed by the target.
[0160]
[0161] The motion tracking means (100) provide the orientations and/or accelerations together with an identification of each orientation and/or acceleration, for example an identification of the sensor providing the orientation and/or acceleration, or an identification of the segment or limb associated with the orientation and/or acceleration. The device (200) uses the identification of the orientations and/or accelerations to select the particular orientations and/or accelerations for which each predetermined constraint is defined; similarly, in a method for determining a correct reproduction of a movement according to the present disclosure, the particular orientations and/or accelerations for which each predetermined constraint is defined are selected so as to carry out the different steps of the method.
[0162] The device (200) comprises first communication means (210) for receiving the orientation and/or acceleration information from the motion tracking means (100), that is, the first orientation (v.sub.1), the second orientation (v.sub.2), and any additional orientation vectors, and/or, the first and/or further accelerations, of the different time instants, required for determining the correct execution of the particular movement under analysis. The first communication means (210) may be implemented according to a technology and protocol known in the state of the art, and may either be a direct connection or include any number of intermediate connection networks.
[0163] The device (200) comprises second communication means (220) through which it may transmit and/or receive data to the remote server (300) (e.g. a user may transmit data regarding the definition of movements and/or predetermined constraints, such as updated ranges of constraints, a user may also transmit data for selecting a movement to be assessed so that the appropriate angular ranges and/or acceleration intervals for validating the movement are selected, etc.). Said second communication means (220) may be implemented according to a technology and protocol known in the state of the art, and may be a direct connection or include any number of intermediate connection networks. In a non-limiting example, said remote server (300) may be a database from which a user may then retrieve information about the movements being tracked, for instance through a personal device such as a computer or a phone.
[0164] The first communication means (210) and the second communication means (220) may be implemented either with the same technology, sharing the same physical resources, or with different technologies known in the state of the art. Some embodiments of the invention may be implemented without the remote server (300). In this case, feedback regarding whether the movement is correctly reproduced, as determined by the device (200), may be stored in an internal memory (240) of the device (200) or displayed for the user's knowledge through any kind of user interface (250) of the device (200). Through the user interface (250), a user may also select the angular ranges and/or acceleration intervals of each constraint, and select which movement or movements are to be analyzed.
[0165] Once the first orientation (v.sub.1) and the second orientation (v.sub.2) are received by the device (200), the predetermined constraints of the movement under analysis are verified at the processor (230). For said verification, the device (200) is configured to access the memory (240) for gathering which orientations and, thus, body segments need to be compared (i.e. the vectors provided by the motion tracking means 100 associated with said body segments), the plane(s) onto which said vectors are to be projected, and the angular conditions that need to be met. Additionally or alternatively, once the first acceleration is received by the device (200), the predetermined constraints of the movement under analysis are verified at the processor (230). For said verification, the device (200) is configured to access the memory (240) for gathering which accelerations and, thus, body segments need to be compared (i.e. the acceleration measurements provided by the motion tracking means 100 associated with said body segments), the direction, and the start and end acceleration thresholds that need to be met.
[0166] Even though in the present disclosure it is explained that the start angle is less than the end angle so that the first computed angle (concerning the first predetermined constraint) must be less than or equal to the start angle and the second computed angle (concerning the first predetermined constraint) must be greater than or equal to the end angle in order to correctly reproduce the movement, it is readily apparent that it is also possible that the angles of the first predetermined constraint may be defined the other way around, that is, the start angle is greater than the end angle. In that case, the first computed angle must be greater than or equal to the start angle, and the second computed angle must be less than or equal to the end angle in order to determine that the movement has been reproduced correctly. Similarly, even it is explained that the first acceleration value must be greater than or equal to the start acceleration threshold and the second acceleration value must be less than or equal to the end acceleration threshold (concerning the first predetermined constraint) in order to determine that the movement is correctly reproduced, it is readily apparent that it is also possible to define these thresholds or the direction defined in the first predetermined constraint the other way around. In that case, the first acceleration value must be less than or equal to the start acceleration threshold, and the second acceleration value must be greater than or equal to the end acceleration threshold in order to determine that the movement has been reproduced correctly.
[0167] In this text, the term “time instant” is meant to refer to a particular moment of time, but it is readily apparent that it may involve a time duration, that is as short as possible, inherent to the speed and/or synchronization of the devices. For example, the motion tracking means may not provide the orientations of a target in a moment of time but in a short time duration. The time duration is preferably less than 100 milliseconds, and more preferably less than 50 ms, 25 ms, 10 ms and/or 5 ms.
[0168] In this text, the term “comprises” and its derivations (such as “comprising”, etc.) should not be understood in an excluding sense, that is, these terms should not be interpreted as excluding the possibility that what is described and defined may include further elements, steps, etc.
[0169] Even though the terms first, second, third, etc. have been used herein to describe several parameters or variables, it will be understood that the parameters or variables should not be limited by these terms since the terms are only used to distinguish one parameter or variable from another. For example, the second time instant or t.sub.2 could as well be named third time instant or t.sub.3, and the third time instant or t.sub.3 could be named second time instant or t.sub.2 without departing from the scope of this disclosure.
[0170] On the other hand, the invention is obviously not limited to the specific embodiment(s) described herein, but also encompasses any variations that may be considered by any person skilled in the art (for example, as regards the choice of materials, dimensions, components, configuration, etc.), within the general scope of the invention as defined in the claims.