CORRECTION OF HEADING ERRORS OF INERTIAL MEASUREMENT UNITS OF A MOTION TRACKING SYSTEM
20230003528 · 2023-01-05
Inventors
- Pedro Henrique Oliveira SANTOS (Porto, PT)
- Luís Ungaro PINTO COELHO (Porto, PT)
- Marta Maria Ramalho FERREIRA (Porto, PT)
- Ana Clara FERREIRA MATOS (Porto, PT)
- Pablo PEÑAS (Porto, PT)
- Virgílio António FERRO BENTO (Porto, PT)
Cpc classification
G01P13/00
PHYSICS
A61B2560/0223
HUMAN NECESSITIES
A61B5/1123
HUMAN NECESSITIES
G01C21/16
PHYSICS
A61B2562/0219
HUMAN NECESSITIES
International classification
G01C21/16
PHYSICS
A61B5/00
HUMAN NECESSITIES
A61B5/11
HUMAN NECESSITIES
G01C25/00
PHYSICS
Abstract
A method comprising: digitally processing orientation measurements provided by each of first and second inertial measurement units, the first and second units being arranged on first and second body members of a person, respectively, according to a predetermined unit arrangement, and the first and second body members being connected by a joint; the measurements are digitally processed such that the computing device at least: computes a length vector of a segment of the first body member based on a first orientation measurement of the first unit; defines a joint axis plane of the joint based on a second orientation measurement of the second unit; and computes a heading rotation value for making the first orientation measurement to be contained within the joint axis plane defined; and the method further comprising digitally modifying the first orientation measurement or the second orientation measurement by applying a rotation at least based on the heading rotation value computed. Also, a motion tracking system and a computer program product.
Claims
1. A method (100, 101) comprising: digitally processing (130), by a computing device (15) of a motion tracking system (10), orientation measurements (40, 41) provided by each of first and second inertial measurement units (20a, 20b) of the motion tracking system (10), each of the first and second units (20a, 20b) at least comprises an accelerometer (21) and a gyroscope (22), the first and second units (20a, 20b) being arranged on first and second body members (6a-6d) of a person (1), respectively, according to a predetermined unit arrangement, and the first and second body members (6a-6d) being connected by a joint (5a, 5b); characterized in that: the measurements are digitally processed (130) such that the computing device (15) at least: computes a length vector of a segment (7a, 7c) of the first body member (6a, 6c) based on a first orientation measurement (40) of the first unit (20a); defines a joint axis plane (50) of the joint (5a, 5b) based on a second orientation measurement (41) of the second unit (20b); and computes a heading rotation value for making the first orientation measurement (40) to be contained (60, 61) within the joint axis plane (50) defined; and the method further comprising digitally modifying (140), by the computing device (15), the first orientation measurement (40) or the second orientation measurement (41) by applying a rotation (55, 56) based on the heading rotation value computed.
2. The method (100, 101) of claim 1, wherein the step of digitally processing (130) comprises: providing a first vector (a, b, c) based on the second orientation measurement (41); providing value z.sub.orig based on the first orientation measurement (40); and solving the following system of three equations:
ax+by +cz=0,
y=√{square root over (1−z.sup.2−x.sup.2)}, and
z=z.sub.orig; and the heading rotation value is based on a solution to the system of three equations.
3. The method (100, 101) of claim 2, wherein: the step of digitally processing (130) further comprises selecting the solution to the system of three equations that at least one of: fulfills one or more predetermined biomechanical constraints, and results in an instantaneous heading error that: is determined not to be excessive by the computing device (15), and/or is less than an instantaneous heading error for the other solution to the system of three equations; and the heading rotation value is based on the selected solution to the system of three equations.
4. The method (100, 101) of claim 1, further comprising digitally providing (150), by the computing device (15), a movement sequence of the person (1) based on the orientation measurements (40, 41) provided by the first and second units (20a, 20b) wherein at least some orientation measurements (60) of one the first and second units (20a, 20b) have been digitally modified (140).
5. The method (100, 101) of claim 1, further comprising repeating at least the digital steps (130, 140, 150) one or more times with most recent orientation measurements (40, 41) provided by the first and second units (20a, 20b) for correction of heading error in the measurements provided by the units (20a, 20b).
6. The method (100, 101) of claim 5, wherein the rotation applied in the step of digitally modifying (140) is at least further based on one or more heading rotation values previously computed for digital modification (140) of previous orientation measurements.
7. The method (101) of claim 1, further comprising, prior to the step of digitally processing (130), arranging (110) the first and second units (20a, 20b) on the first and second body members (6a-6d) of the person (1), respectively.
8. The method (100, 101) of claim 1, wherein either the motion tracking system (10) or each of the first and second units (20a, 20b) comprises attaching devices for indicating or forcing arrangement of each unit (20a, 20b) with a predetermined orientation.
9. The method of claim 1, further comprising calibrating the first and second units (20a, 20b) after arranging (110) them on the first and second body members (6a-6d).
10. The method (101) of any one of the preceding claims, wherein at least the second unit (20b) is arranged on the second body member (6b, 6d) such that one of three axes of the second unit (20b) is aligned or substantially aligned with a segment of the second body member (6b, 6d).
11. The method (100, 101) of claim 1, wherein the first and second body members (6a-6d) respectively are an upper arm (6c) and a forearm (6d) or vice versa, or the first and second body members (6a-6d) respectively are a thigh (6a) and a shank (6b) or vice versa.
12. The method (101) of claim 1, further comprising providing (120), by each of the first and second units (20a, 20b), the orientation measurements (40, 41) to be digitally processed (130).
13. A motion tracking system (10) comprising: a computing device (15) comprising at least one processor (16) and at least one memory (17); and first and second inertial measurement units (20a, 20b) adapted to be arranged on first and second body members (6a-6d) of a person (1) and each at least comprising an accelerometer (21) and a gyroscope (22); characterized in that: the at least one processor (16) is configured, together with the at least one memory (17), to carry out a method (100, 101) according to claim 1.
14. A computer program product that has instructions which, when executed by a computing device (15) of a motion tracking system (10) comprising a first and second inertial measurement units (20a, 20b) adapted to be arranged on first and second body members (6a-6d) of a person (1) and each at least comprising an accelerometer (21) and a gyroscope (22), cause the computing device (15) to perform a method (100, 101) according to claim 1.
15. The method (100, 101) of claim 1, wherein each of the first and second units (20a, 20b) further comprises a magnetometer (23).
16. The motion tracking system (10) of claim 13, wherein each of the first and second units (20a, 20b) further comprises a magnetometer (23).
17. The computer program product of claim 14, wherein each of the first and second units (20a, 20b) further comprises a magnetometer (23).
Description
BRIEF DESCRIPTION OF THE DRAWINGS
[0074] To complete the description and in order to provide for a better understanding of the invention, a set of drawings is provided. Said drawings form an integral part of the description and illustrate embodiments of the invention, which should not be interpreted as restricting the scope of the invention, but just as examples of how the invention can be carried out. The drawings comprise the following figures:
[0075]
[0076]
[0077]
[0078]
DESCRIPTION OF WAYS OF CARRYING OUT THE INVENTION
[0079]
[0080] In this embodiment, the first IMU 20a is arranged on the thigh 6a, and the second IMU 20b is arranged on the shank 6b of the same leg. The thigh 6a and the shank 6b are connected by the knee 5a; the thigh 6a and the shank 6b each moves one relative to the other only around the axis of the knee 5a as a 1-DOF joint. In some other embodiments, the first IMU 20a is arranged on the shank 6b and the second IMU 20b is arranged on the thigh 6a. The first and second IMUs 20a, 20b are arranged on these body members 6a, 6b owing to first and second straps 30a, 30b even though other attaching means or devices for arranging the IMUs 20a, 20b on the person 1 are also possible.
[0081] In
[0082] From the standing still position, the person 1 performs a hip external rotation as illustrated in
[0083] Turning to
[0084] As it can be appreciated from the above examples, the heading estimates of the joint axis (knee axis in these examples) given by each of the first and second units 20a, 20b should be the same when no heading errors exist (the measurements of the sensing devices are error-free and tolerances of the sensing devices are disregarded), but in
[0085] Methods and systems according to the present disclosure make possible to correct heading errors, and in the following cases too, which are considered and, therefore, modelled as an additional degree of freedom in the movement of one of the body members: [0086] there are soft tissue artifacts (as in the above examples); [0087] there is supination/pronation of the forearm; and/or [0088] one of the units is misplaced.
[0089]
[0090] Each IMU 20a, 20b at least includes two sensing devices: an accelerometer 21, and a gyroscope 22. As shown with dashed lines, in some embodiments each of the IMUs 20a, 20b further includes a magnetometer 23.
[0091] The first and second IMUs 20a, 20b further include at least one processor 26, at least one memory 27, and a first communications module 28 for transmitting and receiving data that enables the IMUs to transmit (through a wired or wireless communications technology and protocol known by a skilled person, for instance but without limitation, Bluetooth communications, cellular network communications such as GSM, UMTS or LTE, wireless LAN communications, etc.) measurements of the sensing device(s) 21-23 to the computing device 15. The same first communications modules 28 enable the units 20a, 20b to receive data from the computing device 15.
[0092] In some preferred embodiments, the at least one processor 26 of the IMUs 20a, 20b runs a sensor fusion algorithm for processing the measurements of the sensing devices 21-23 within the respective IMU. The sensor fusion algorithm is intended to enhance the raw measurements of the sensing devices by correcting errors thereof due to drifts of the sensing devices and, thus, outputs processed measurements that are to be transmitted to the computing device 15. By way of example, biases of the gyroscopes 22 accumulate over time and worsen the accuracy of the measurements of the IMUs 20a, 20b, and magnetic disturbances also bias the measurements of the magnetometers 23 when the IMUs 20a, 20b comprise said sensing device. The sensor fusion algorithm may reduce the influence of these biases with the measurements provided by remaining sensing device(s), but may not completely cancel these biases out.
[0093] The computing device 15 includes at least one processor 16, at least one memory 17, and a second communications module 18 for transmitting and receiving data (through a wired or wireless communications technology and protocol known by a skilled person, for instance but without limitation, Bluetooth communications, cellular network communications such as GSM, UMTS or LTE, wireless LAN communications, etc.).
[0094]
[0095] The person 1 is performing a hip flexion. The first and second IMUs 20a, 20b provide respective orientation measurements such as first orientation measurement 40 and second orientation measurement 41.
[0096] The computing device (e.g. the computing device 15) of the motion tracking system receives the orientation measurements 40, 41 from the IMUs 20a, 20b and digitally processes them for determining whether heading errors are present in the orientation measurements 40, 41 and correcting the same.
[0097] To this end, the computing device processes the second orientation measurement 41 to establish the local axes of the second unit 20b and, based on these local axes and whose orientations can be obtained in global coordinates, define a joint axis plane 50 corresponding to the plane of the axis of the knee 5a. In this case, one of the local axes of the second unit 20b is used as a normal vector, e.g. (a, b, c) of the plane 50, which is sufficient to define it by equation: ax+by +cz=0.
[0098] In some embodiments, none of the local axes of the second unit 20b coincides with the axis of the knee 5a, for instance when the second unit 20b is misplaced on the shank 6b. In such situations, the computing device is to calibrate the units 20a, 20b by way of a calibration procedure for determining the placement of the units 20a, 20b on the body members 6a, 6b, and then a vector is provided by the computing device to define the joint axis plane 50; said vector does not coincide with the local axes of the second unit 20b but is based on the actual orientation of the (misplaced) second unit 20b. The calibration procedure triggered once the units 20a, 20b are arranged on the person 1 can be a calibration procedure of the state of the art whereby either the person 1 is requested to perform predetermined movement(s) for calibration purposes that involve motion of the body members with units 20a, 20b arranged thereon, and/or the units 20a, 20b have their sensing devices calibrated such that the headings of the measurements thereof are accurate at least for a certain amount of time, e.g. one minute, two minutes, five minutes, etc., without requiring any predetermined movement(s) from the person 1. By way of example, a calibration procedure of the state of the art is that described in PCT/PT2018/050044, which is incorporated by reference in its entirety herein.
[0099] The computing device computes a vector corresponding to a length direction of the segment 7a of the thigh 6a based on the first orientation measurement 40, for example said first orientation measurement 40 can be defined by vector (x.sub.orig, y.sub.orig, z.sub.orig). The computing device then determines that there is a heading error because the first orientation measurement 40 is not coplanar with the plane 50 defined by the computing device due to accumulated bias in the gyroscope and/or magnetic distortions affecting the magnetometer (if any) of the IMU; the orientation measurement 40 should be coplanar owing to the biomechanical constraints of the knee 5a.
[0100] The soft tissue artifacts do not change the orientation in the length direction of the thigh 6a (i.e. that of the segment 7a) but change those in the other two directions, namely the directions around the length direction, hence the orientation in the length direction is used in the correction of the heading error. The computing device computes how the orientation measurement 40 is to be rotated so that it ends up being coplanar with the joint axis plane 50. The computing device thus computes heading rotation(s) 55, 56 (e.g. angles, quaternions) whereby the orientation measurement 40 becomes coplanar with the joint axis plane 50, thereby providing first and second coplanar orientations 60, 61. The heading rotation(s) 55, 56 correspond to the heading error computed with inverse sign. For the sake of clarity only, a first heading rotation 55 is shown with solid line as an angle for providing the first coplanar orientation 60 also shown with solid line, whereas a second heading rotation 56 is shown with dashed line as an angle for providing the second coplanar orientation 61 also shown with dashed line. The heading rotations 55, 56 are obtained by digitally solving a system of three equations:
ax+by +cz=0,
y=√{square root over (1−z.sup.2−x.sup.2)}, and
z=z.sub.orig.
[0101] In this regard, the first coplanar orientation 60 can be defined by vector (x.sub.1, y.sub.1, z.sub.orig) and, accordingly, a first heading rotation value can be computed so that x.sub.orig and y.sub.orig become x.sub.1 and y.sub.1, respectively, thus the heading rotation value 55 is such that
Likewise, the second coplanar orientation 61 can be defined by vector (x.sub.2, y.sub.2, z.sub.orig) and, accordingly, the second heading rotation value can be computed so that x.sub.orig and y.sub.orig become x.sub.2 and y.sub.2, respectively, thus the heading rotation value is such that
[0102] The definition of the vertical component of the vectors as z.sub.orig, which is a value from the vector of the first orientation measurement 40, is due to the fact that heading is to be corrected by means of a rotation around the global vertical axis (i.e. Z axis illustrated), thus the vertical component is disregarded. As it can be appreciated, in some other examples, the first coplanar orientation 60 is defined by vector (x.sub.2, y.sub.2, z.sub.orig) and the second coplanar orientation 61 is defined by vector (x.sub.1, y.sub.1, z.sub.orig).
[0103] The computing device establishes that the first coplanar orientation 60 is the correct orientation out of the two orientations 60, 61 since it is the orientation that: [0104] fulfills one or more predetermined biomechanical constraints established for the concerned joint, and/or [0105] a difference between the computed heading error and a previously computed heading error fulfills one or both of: [0106] is smaller than the difference between the computed heading error for the other orientation and a previously computed heading error, and [0107] does not exceed a predetermined heading error threshold, namely the difference shall not exceed a threshold of e.g. 10°, 20°, etc.
[0108] In relation to the fulfillment of predetermined biomechanical constraints, for example the second coplanar orientation 61 results in a movement of the joint that is deemed impossible (e.g. knee hyperextension) or is caused by a soft tissue artifact deemed excessive for the concerned body member.
[0109] By way of example, when the IMUs 20a, 20b are arranged on the leg of the person 1, it has been found out that soft tissue artifacts should not result in a difference of more than 45° around the direction of the segment (i.e. length direction of the body member), and the solution being closer to the joint axis plane is the one deemed correct; positive Y axis is typically closer to the joint axis than negative Y axis, therefore, in these cases, the heading rotation value that is selected is that of positive Y of the thigh 6a joint axis, which is closer to the positive Y of the shank 6b.
[0110]
[0111] The person 1 is performing forearm pronation as illustrated with
[0112] The computing device (e.g. the computing device 15) of the motion tracking system receives the orientation measurements 40, 41 from the IMUs 20a, 20b and digitally processes them for determining whether heading errors are present in the orientation measurements 40, 41 and correcting the same.
[0113] With a digital processing as described in the present disclosure, in this example, the computing device processes the first orientation measurement 40 and provides a vector corresponding to a local axis of the first IMU 20a to define the axis plane 50 of the joint 5b, in this case the elbow 5b of the person 1.
[0114] The computing device solves the system of equations for a vector provided based on the second orientation measurement 41, and with the solutions it computes heading rotations 55, 56 for making the second orientation measurement 41 to be contained within the joint axis plane 50. One of the solutions can be selected over the other according to any one of the criteria described in relation to
[0115] It is noted that the vectors described for instance with reference to
[0116] Albeit in the embodiments of
[0117]
[0118] The method 100 comprises a step 130 whereby a computing device (e.g. the computing device 15 of
[0119] The orientation measurements that the computing device processes may be raw measurements, i.e. values measured by a sensing device of a respective unit, or processed measurements, i.e. values modified by the processor(s) of the IMU, for instance owing to a sensor fusion algorithm that is intended to improve the measurements of the IMU by combining the measurements of the different sensing devices.
[0120] The method 100 further comprises a step 140 whereby the computing device digitally modifies one or more measurements of one of the first and second IMUs for correcting the heading error. This is attained by applying a rotation at least based on a value or values computed in the step 130 in which the computing device digitally processes the measurements.
[0121] Preferably, the steps 130 and 140 are carried out in real time or almost in real time as the IMUs provide the measurements, even though, in some embodiments, the heading error correction is performed later on during the motion tracking procedure or after completion of the same. In this sense, the steps 130 and 140 can be carried out during physical exercising of the person, therefore the orientation measurements are provided by the units and the computing device processes and modifies them while the person moves one or both of the first and second body members; movement of the body members comprises linear motion and/or rotation of the respective body member.
[0122] In some embodiments, the steps 130 and 140 carried out by the computing device are repeated (as illustrated with a dashed line) a number of times so as to correct heading errors in orientation measurements provided by the IMUs during a motion tracking procedure. Therefore, upon providing more recent measurements, the computing device digitally processes the same and digitally modifies measurements of one of the IMUs so as to correct the heading error, thereby making possible to improve the precision of the movement sequence that is to be provided by the computing device.
[0123] In some embodiments, the method 100 further comprises one or more of steps 110, 120 and 150 as described next in relation to method 101 of
[0124]
[0125] The method 101 comprises a step 110 whereby the first and second units are arranged on the first and second body members, respectively. The units may be arranged 110 by way of attaching devices or means such as e.g. straps, Velcro, etc.
[0126] At least the unit whose orientation measurement(s) is/are used in the definition of the joint axis plane is preferably arranged on the respective body member (preferably the body member of the two body members that only rotates around the joint axis, or mostly rotates around the joint axis) such that a local axis of the unit out of the three local axes is aligned or substantially aligned with the segment of the body member, otherwise a calibration procedure is effected whereby the computing device determines how the IMUs are arranged on the body segments so that the orientation of the IMUs can be specified with respect to the segments of the body members (even if no local axis of one or both units is aligned or substantially aligned with the corresponding segment). More preferably, both units are arranged on the respective body members such that a respective local axis of the units out of the three local axes is aligned or substantially aligned with the segment of the respective body member, or the aforementioned calibration procedure is performed for determining the orientation of the IMUs as indicated.
[0127] The method 101 further comprises a step 120 whereby the first and second units, once arranged 110 on the body members of the person, provide the orientation measurements to be digitally processed 130 and modified 140 by the computing device. The orientation measurements may be provided to the computing device via wired or, preferably, wireless communications. The orientation measurements may be raw measurements or processed measurements, e.g. processed with a sensor fusion algorithm as known in the art.
[0128] The method 101 further comprises the steps 130, 140 of digitally processing and modifying the orientation measurements provided 120. The method 101 also comprises a step 150 whereby the computing device digitally provides a movement sequence of the person based on the orientation measurements provided by the first and second units, and where at least some (or all) orientation measurements of one the first and second units have been digitally modified 140.
[0129] The movement sequence comprises data of the motion tracking procedure that indicates how the person or at least the body members thereof with IMUs have behaved in terms of movements and orientations during said procedure. Thanks to the movement sequence provided 150, the computing device is capable of assessing whether the person has performed movements or physical exercises according to predetermined criteria, for instance for physical rehabilitation purposes. The correction of heading errors present in the orientation measurements enhances the accuracy of the movement sequence and, thus, of any additional evaluation, for example the mentioned movement or physical exercise assessment, which may be carried out according to state of the art procedures known in the art, for example but without limitation, a physical exercise evaluation procedure as described in PCT/EP2019/066237, which is incorporated by reference in its entirety herein.
[0130] A motion tracking system (e.g. the system 10 of
[0131] In this text, the terms first, second, third, etc. have been used herein to describe several devices, elements or parameters, it will be understood that the devices, elements or parameters should not be limited by these terms since the terms are only used to distinguish one device, element or parameter from another. For example, the first orientation measurement could as well be named second orientation measurement, and the second orientation measurement could be named first orientation measurement without departing from the scope of this disclosure.
[0132] In this text, the term “comprises” and its derivations (such as “comprising”, etc.) should not be understood in an excluding sense, that is, these terms should not be interpreted as excluding the possibility that what is described and defined may include further elements, steps, etc.
[0133] On the other hand, the invention is obviously not limited to the specific embodiment(s) described herein, but also encompasses any variations that may be considered by any person skilled in the art (for example, as regards the choice of materials, dimensions, components, configuration, etc.), within the general scope of the invention as defined in the claims.