MOTION CAPTURING GARMENTS AND SYSTEM AND METHOD FOR MOTION CAPTURE USING JEANS AND OTHER GARMENTS
20210247841 · 2021-08-12
Inventors
- Ozgur COBANOGLU (Inegol - Bursa, TR)
- Jitka ERYILMAZ (Inegol - Bursa, TR)
- Serkan MERT (Inegol - BURSA, TR)
- Fehim CAGLAR (Inegol - Bursa, TR)
Cpc classification
G06F3/011
PHYSICS
G06F3/0346
PHYSICS
A61B5/1121
HUMAN NECESSITIES
A61B2562/028
HUMAN NECESSITIES
International classification
A61B5/00
HUMAN NECESSITIES
A61B5/11
HUMAN NECESSITIES
Abstract
A real-time motion capture system and garment includes a wearable activity monitor that may be a pair of denim jeans. The wearable activity monitor includes multiple sensors such as accelerometers, gyrometers, magnetometers disposed within the seams of the garment. A microprocessor and wireless transmitter) communicate the motion data to an external device. The microprocessor and wireless transmitter may be included within one of the seams. An elastically stretchable ribbon or a flexible ribbon such as a kapton ribbon or a ribbon formed of textile, electrically couples the sensors and microprocessor and is also disposed inside the seams and the components within the seam are coated with a waterproof coating. The external device can store the data or display and analyze the data real-time, and may communicate the data to a further electronic device.
Claims
1. A garment comprising fabric and including a plurality of inertial measurement units (IMU's) contained within seams of said garment.
2. The garment according to claim 1, further comprising at least one microcontroller disposed on said garment or within at least one of said seams of said garment and electrically coupled to said plurality of IMU's.
3. The garment according to claim 1, further comprising a wireless transmitter disposed in or on said garment and adapted to wirelessly transmit data from said IMU's to at least one external data receiving and processing device.
4. The garment according to claim 1, wherein said plurality of IMU's are electrically coupled to one another along a ribbon of a flexible material further disposed within said seams, said ribbon including a plurality of electrical connections forming a communication bus.
5. The garment according to claim 4, wherein said ribbon is elastically stretchable by at least 10%, said electrical connections being arranged along said ribbon in a crooked path.
6. The garment according to claim 5, wherein said flexible material comprises plastic, textile or conductive yarn and further comprising a waterproof coating at least on said IMU's, within said seams.
7. The garment according to claim 6, wherein said garment is a pair of pants formed of denim.
8. The garment according to claim 7, wherein said seams include, for each pant leg of said pair of pants, an outer lateral seam extending in a longitudinal direction along a wearer's leg and/or an inner medial seam extending in said longitudinal direction along said wearer's leg, each of said outer lateral seams and/or each of said inner medial seams including at least one of said plurality of IMU's disposed along a wearer's femur, and at least one of said plurality of IMU's disposed along a lower portion of said wearer's leg.
9. The garment according to claim 8, wherein said seams include a waist seam extending along a waist of said pair of pants, said waist seam including at least one of said plurality of IMU's disposed therein.
10. The garment according to claim 1, wherein said plurality of IMU's includes a plurality of accelerometers, gyrometers and magnetometers.
11. The garment according to claim 1, further comprising at least one of an LED and one or more vibration members disposed within at least one of said seams.
12. A motion capture system comprising: a garment formed of fabric and including a plurality of inertial measurement units (IMU's) contained within seams of said garment, a wireless transmitter disposed in or on said garment and adapted to wirelessly transmit data from said IMU's to at least one external data receiving and processing device, and said external device including a processor adapted to receive and analyze said transmitted data and a display that displays said data, said external device adapted to control at least one further electronic device based upon said data.
13. A method for monitoring and analyzing real time motion, said method comprising: acquiring motion data from a plurality of inertial measurement units (IMU's) disposed within seams of a garment disposed on a non-stationary entity; wirelessly transmitting said motion data to a device that includes at least one of a processor and a display; and at least one of displaying and analyzing said data.
14. The method according to claim 13, further comprising one of said plurality of IMU's detecting a gravity vector, wherein said acquiring motion data is carried out using said gravity vector as reference.
15. The method according to claim 13, wherein said at least one of displaying and analyzing includes analyzing said motion data regarding angle, yaw, pitch, location and acceleration of a human wearer's femurs and said human wearer's tibias or fibulas, and visually displaying movement associated with said motion data on said display.
16. The method according to claim 15, further comprising a microcontroller receiving data wirelessly transmitted from said device and providing tactile feedback based on said data wirelessly transmitted from said device.
17. The method according to claim 16, wherein said at least one of displaying and analyzing includes collecting said motion data during a collecting period and displaying said collected data when desired, said displaying said collected data including displaying graphically a statistic distribution of the angles for each human wearer's femurs and for each human wearer's tibias or fibulas.
18. The method according to claim 16, wherein said providing tactile feedback comprises providing tactile feedback to different parts of the human wearer's body in response to said analyzing.
Description
BRIEF DESCRIPTION OF THE DRAWING
[0038] The present invention is best understood from the following detailed description when read in conjunction with the accompanying drawing. It is emphasized that, according to common practice, the various features of the drawing are not necessarily to scale. On the contrary, the dimensions of the various features are arbitrarily expanded or reduced for clarity. Like numerals denote like features throughout the specification and drawing.
[0039]
[0040]
[0041]
[0042]
[0043]
[0044]
[0045]
[0046]
DETAILED DESCRIPTION
[0047] The present disclosure provides a real-time motion capture system and garment. The garment is a wearable activity monitor and may be a shirt or a pair of jeans or any of various other garments and may be formed of denim or other suitable materials. The wearable activity monitor includes multiple sensors such as inertial measurement units, IMU's. The multiple inertial measurement units include accelerometers, gyrometers, magnetometers and other sensors. The IMU's are in electronic communication with one another and at least one microcontroller and also with one or more external data receiving and processing devices. The IMU's are advantageously all disposed within the seams of the garment. Stated alternatively, the IMU's are contained inside the seams formed between pieces of fabric, in the garment. A microprocessor may be included within one of the seams or at another location in or on the garment. A tactile notification device and/or an LED may also be included within one of the seams, in various embodiments. Ribbons formed of flexible material such as textile or plastic or conductive yarns, electrically couple the IMU's and microprocessor and any other devices, and are also disposed inside the seams to provide a true wearable activity tracker with the flexible circuit boards and the IMU's contained within the seams. The ribbon may be a flexible circuit board such as a kapton ribbon and may be advantageously coated with a waterproof coating as may be the other electronic components within the seam.
[0048] The motion data obtained by the IMU's may be wirelessly transmitted to an external data receiving and processing device such as a computer or other processor, by a wireless transmitter such as a radio antenna configured to transmit data generated by the IMU's. The wireless transmitter is disposed on or in the garment and may be part of the microprocessor. The disclosure provides a wearable activity tracker that is attractive, comfortable, fashionable, non-obstructive, lightweight and washable. No cables are required outside of the garment, the connections are invisible and the routing is not visible because the components are disposed within the seams where they are coated and thus isolated from sweat, water and any other liquid or other fluid.
[0049] The wearable activity monitor finds application in the fields of medicine, physical therapy and rehabilitation, sports coaching, sports performance analysis, body figure recording, posture and balance monitoring, sports injury prevention, differential analysis of walking and running, monitoring synchronized motions of groups of people, gait analysis for running and walking, throwing motion analysis, monitoring the activities of the elderly, sleep apnea analysis through body movement during sleep, and in various other fields.
[0050] The IMU's may be positioned in the seams and the seams positioned relative to a wearer's body so as to use the accelerometers, gyrometers and magnetometers to capture motion on a real-time basis by measuring the exact angle, yaw, location and pitch of body parts as well as the acceleration and other movements of the body parts. The IMU's are advantageously configured to detect motion in 360 degrees in all three axes. Multiple accelerometers may be utilized in a complementary fashion in cases in which accelerometers with limited ranges are used, to cover dead angles.
[0051] For example, and as will be shown in the drawings, the garments may be a pair of pants and include, for each pant leg, an outer lateral seam extending in a direction parallel a wearer's leg bones and an inner medial seam extending along the direction parallel a wearer's leg bones. Each of the outer lateral and inner medial seams may include each of an accelerometer, a gyrometer and a magnetometer disposed both along a wearer's upper leg (femur) portion and all three of an accelerometer, a gyrometer and a magnetometer also disposed along the wearer's lower leg (tibia and fibula) portion to measure and monitor each bone of the wearer's leg. As above, in some embodiments described herein, multiple accelerometers may be utilized in a complementary fashion.
[0052] The disclosure also provides an activity tracking system. The system includes the wearable motion-capturing garment as above and a wireless transmitter such as a radio antenna, disposed in or on the garment and adapted to transmit data from the IMU's to an external device. The external device may include a processor and a display. The external device may include a memory for storing the data. The display may visually display, on a screen or other visible interface such as a graphical user interface (GUI), the motion of multiple persons wearing the garments. The processor analyzes the data received from all of the IMU's. In some embodiments, the external device provides feedback messages or an alarm and in some embodiments, the external device sends a signal to another electronic device based on the data analysis. In some embodiments, the system also includes the garment acting as a controller to control other electronic devices such as home appliances, computer games, Xbox, other electronic devices and so forth. The data obtained from the IMU's is provided to a controller that, in turn, controls electronic devices such as the aforementioned examples.
[0053] The disclosure also provides a method for real time motion capture and the analysis of such motion. The method includes obtaining data from a wearable activity monitor such as the garments described herein and delivering the data to an external data receiving and processing device adapted to receive and analyze the data and provide a display of the data graphically or in various other forms. In other embodiments, the data is stored and displayed and analyzed later. The method includes obtaining data by measuring the exact angle, yaw, location and pitch of multiple body parts in time as well as the acceleration and other movements of the body parts by positioning the IMU's in appropriate locations along the seams and by positioning the seams at appropriate locations along the wearer's body. The method includes wirelessly or otherwise transmitting the data to an external data receiving and processing device that includes a processor. The external data receiving and processing device may control another electronic device and/or provide feedback based on the data analysis. Various algorithms and methods such as Kalman filtering, also known as linear quadratic estimation (LQE), may be used to analyze the data and present the data in a useful and user-friendly format.
[0054]
[0055] An advantage of the wearable activity monitor of the disclosure is that the electronic components that form the wearable activity monitor are disposed within, i.e. inside seams 4 and therefore not visible. Stated alternatively, the electronic components that form the wearable activity monitor are surrounded by fabric and inside a garment seam. A seam may be a portion of a garment in which two overlapping portions of fabric are joined together but the disclosed system may be formed in other types of garment seams in other embodiments. A close-up of seam 4 is shown in
[0056] The garment is advantageously a tight fitting or form fitting garment so that the inertial measurement units, IMU's or other sensors disposed within the seams are accurately positioned with respect to a wearer's body.
[0057]
[0058]
[0059] Some details of flexible connector 32 and the sensors 30 disposed on flexible connector, are shown in
[0060] In some embodiments, the IMU's, i.e. sensors 30 may be islands of electronics, formed on ordinary PCB's, and connected to each other by ribbons 32 that may be elastically stretchable flexible connectors. In particular, with respect to
[0061] In
[0062] In addition to sensors 30,
[0063] Referring again to
[0064] In some embodiments, terminals 42 may provide connection to a computer or other processor such as may be used to program microcontroller 40 and/or update firmware for microcontroller 40.
[0065]
[0066] The disclosure provides for obtaining data by using the aforementioned sensors to measure the exact angle, yaw, location and pitch of multiple body parts in time as well as the acceleration and other movements of the body parts by positioning the IMU's or other sensors in appropriate locations along the seams and by positioning the seams at appropriate locations along the wearer's body. Graph 48 shown in
[0067]
[0068] The IMU 30 disposed at the location 45 may be used for detecting the gravity vector so that the angle, yaw, location and pitch of multiple body parts can be obtained by taking the gravity vector detected at location 45 as reference.
[0069]
[0070] In general, in the case of a pair of pants, the seams 4 of the garment 2 include, for each pant leg 10 of the pair of pants, an outer lateral seam 4 extending in a longitudinal direction along a wearer's leg and/or an inner medial seam 4 extending in the longitudinal direction along the wearer's leg. Each of the outer lateral seam 4 and/or each of inner medial seams 4 include at least one IMU 30 disposed along a wearer's femur and at least one IMU 30 disposed along a lower portion 16 of the wearer's leg.
[0071] Various other numbers of sensors and locations in addition to those shown in
[0072] The disclosure provides a system and method in which the motion and orientation information is wirelessly transmitted such as by microcontroller 40 shown in
[0073]
[0074] In some embodiments, the step 104 of displaying and analyzing may comprise a step of collecting the motion data during a collecting period and a step of displaying the collected data when desired. The collected data may be displayed graphically as shown in
[0075] The user can switch between the display relating the motion data of a leg to the other leg by means of a graphical user interface, so that possible asymmetries of movements can be displayed graphically in a simple and intuitive manner for the user.
[0076] Various methods and algorithms may be used to process and analyze the transmitted motion and location data and to present a display or otherwise present the data in a useful form. In some embodiments, Kalman filtering may be used. Kalman filtering, also known as linear quadratic estimation (LQE), is an algorithm that uses a series of measurements observed over time, containing statistical noise and other inaccuracies, and produces estimates of unknown variables that tend to be more precise than those based on a single measurement alone, by using Bayesian inference and estimating a joint probability distribution over the variables for each timeframe.
[0077] The Kalman filtering embodiment is presented by way of example only and in various embodiments, various data analysis and presentation methods are used. In various embodiments, multiple garments 2 transmit data to an external device such as external device 64 and the data processing and analysis may include a comparison of data received from multiple wearers.
[0078] Referring again to
[0079] In various embodiments, the system may also provide an auditory, visual or other emergency alarm. In some embodiments, various movements or movement patterns such as excessive spinning, high impact or unmitigated freefall, or various other movements or more patterns, may trigger an emergency alarm. In some embodiments various other forms of visual and/or tactile feedback, i.e. non-emergency notifications, may be provided to the user. The alarm or other notification may be presented visually such as on various displays such as may be on external device 64. In other embodiments, the alarm or notification may be a tactile notification such as vibration on a wearable device such as a cellular telephone with a vibration engine. In some embodiments a light emitting diode, LED, or vibrational member within the seam (see feedback device 39 in
[0080] Some embodiments may provide that a feedback notification may be obtained by the external device 64 on the basis of the motion data acquired by the IMU's. The feedback notification may provided as signal 74 to transceiver 58, so that microcontroller 40 generates the notification and sends the same to the feedback device 39 (LED or vibrational member) within the seam. In some embodiments, the feedback device 39 may comprise a plurality of vibrational members arranged within the seam of the garment at different positions associated to particular parts of the wearer's body, so that a tactile feedback may be provided to one or multiple different parts of the wearer's body at the same time, in response to the motion data acquired by the IMU's and analyzed by the external device 64. For example, the tactile feedback can be used for notifying to the wearer wrong and/or correct movements of wearer's body parts during physical exercises.
[0081] In some embodiments, external device 64 may provide firmware updates received by wireless transceiver 58.
[0082] The disclosed garment and system enables the data to be analyzed and used for various advantageous purposes and applications. In some embodiments, the disclosed system provide for monitoring more than one garment such as two garments worn by a wearer, e.g, a pair of pants and an upper body garment. In one embodiment, the disclosed garment and system provide for monitoring group sports activities. For example, a single trainer can simultaneously check on multiple sportsmen to determine if, for example all the students are doing the right thing when they go through their training moves, or for comparative purposes. This can be done realtime or later off-line as the memory required to store the data is small in size and a trainer can later compare data among subjects and make various determinations. In other embodiments, the advantage of injury prevention is achieved by analyzing the movement of an athlete or other performer, for example. The motion data may be used for running or walking style and balance analysis. The data can demonstrate which leg is used more and whether the wearer's steps are equal over time. Gait analysis can also be used to suggest anatomical corrections and to determine a proper style of shoe, such as a running shoe. Various other types of activity tracking can be achieved. Sleep behavior can also be monitored based on the data analysis.
[0083] In some embodiments, the data can assist in calculating calories burned based on an integral of the sensor data and body measurements such as height, weight, leg length and the like. In still other embodiments, a performer such as a dancer or an athlete such as a skier or a skateboarder can develop a particular move and record the captured data provided by the disclosure instead of having to review a lengthy video that may include many irrelevant details.
[0084] The preceding merely illustrates the principles of the disclosure. It will thus be appreciated that those skilled in the art will be able to devise various arrangements which, although not explicitly described or shown herein, embody the principles of the invention and are included within its spirit and scope. Furthermore, all examples and conditional language recited herein are principally intended expressly to be only for pedagogical purposes and to aid in understanding the principles of the disclosure and the concepts contributed by the inventors to furthering the art, and are to be construed as being without limitation to such specifically recited examples and conditions. Moreover, all statements herein reciting principles, aspects, and embodiments of the disclosure, as well as specific examples thereof, are intended to encompass both structural and functional equivalents thereof. Additionally, it is intended that such equivalents include both currently known equivalents and equivalents developed in the future, i.e., any elements developed that perform the same function, regardless of structure.
[0085] This description of the exemplary embodiments is intended to be read in connection with the figures of the accompanying drawing, which are to be considered part of the entire written description. In the description, relative terms such as “lower,” “upper,” “horizontal,” “vertical,” “above,” “below,” “up,” “down,” “top” and “bottom” as well as derivatives thereof (e.g., “horizontally,” “downwardly,” “upwardly,” etc.) should be construed to refer to the orientation as then described or as shown in the drawing under discussion. These relative terms are for convenience of description and do not require that the apparatus be constructed or operated in a particular orientation. Terms concerning attachments, coupling and the like, such as “connected” and “interconnected,” refer to a relationship wherein structures are secured or attached to one another either directly or indirectly through intervening structures, as well as both movable or rigid attachments or relationships, unless expressly described otherwise.
[0086] Although the invention has been described in terms of exemplary embodiments, it is not limited thereto. Rather, the appended claims should be construed broadly, to include other variants and embodiments of the invention, which may be made by those skilled in the art without departing from the scope and range of equivalents of the invention.