Method and apparatus for exercise monitoring combining exercise monitoring and visual data with wireless devices

Abstract

Embodiments of the invention provide a method and apparatus for a wireless exercise monitoring system for interactively monitoring an aspect of exercise, sports, or fitness utilizing a wearable device, such as a watch, eyewear, or smart apparel. The device is equipped with, or connected to, a digital camera. Sensors integrated with, or wirelessly connected to, the wearable internet device record physiological data during exercise and data measuring the amount of exercise performed. The data and visual images from the camera are transmitted to one or more internet servers, and may be shared with other mobile internet devices.

Claims

1. A system for monitoring exercise, comprising: a mobile internet device configured to be carried on a person, the mobile internet device including: an integral digital camera, and a generic port providing a connection to an exercise monitoring device; a communications port providing an interactive wireless connection with one or more internet servers; a non-transitory computer readable medium having instructions stored thereon for providing a user interface for displaying visual images including graphics, photographs, and video clips; the non-transitory computer readable medium further having instructions stored thereon for performing a method comprising steps of: receiving a first type of data that is physiological data measured during exercise, and receiving a second type of data that is an amount of exercise work performed; receiving a third type of data representing visual images captured by the camera, and transmitting the first type of data measuring the physiological data, the second type of data measuring the quantitative amount of exercise work performed, and the third type of data representing the visual images, to one or more internet servers, and a non-transitory computer readable medium residing on one or more internet servers having instructions stored thereon defining a server application for performing a method comprising steps of: receiving the first type of data that is the physiological data, and receiving the second type of data that is a quantitative amount of exercise work performed, and receiving the third type of data representing visual images from the mobile internet device; processing the first, second, and third types of data; transmitting the processed data to the mobile internet device in response; and transmitting the processed data to a second mobile device, whereby a user of the second mobile device may review the transmitted data and images; wherein the non-transitory computer readable medium of the wireless internet device further has instructions for causing the wireless internet device to receive and display the processed data on the user interface of the wireless internet device.

2. The system of claim 1, wherein the mobile internet device is selected from the group comprising: a watch, eyewear, apparel, or other device configured to be worn by a person.

3. The system of claim 1, further comprising a GPS device, an accelerometer, or a timer, integral within the mobile internet device or connected to it, and wherein the data measuring a quantitative amount of exercise work performed is received from the GPS device, the accelerometer, or the timer.

4. The system of claim 1, wherein the first or second types of data, or both, are received from the generic port, and wherein the generic port for connection to an exercise monitoring device is adapted to receive signals from a device selected from the group comprising: a heart rate monitor, a respiration rate monitor, a blood pressure monitor, a temperature monitor, an accelerometer, a pedometer, a GPS device, a timer; a device tracking, rate, intensity, distance, duration, or total amount of exercise; a body weight scale, a body fat gauge, a biofeedback device, a treadmill, a rowing machine, an exercise bicycle, or a stepper.

5. The system of claim 1, wherein the generic port for connection to an exercise monitoring device is configured to provide a wired connection.

6. The system of claim 1, wherein the generic port for connection to an exercise monitoring device is configured to provide a wireless connection, that employs a protocol selected from a variety of 802.11, Bluetooth, or other short range wireless transmission methods.

7. The system of claim 1, wherein the communications port providing an interactive wireless connection to one or more internet servers is configured to provide a wireless link that employs a protocol selected from a variety of 802.11, other IEEE family protocols, cellular, or other RF protocols.

8. A system for monitoring exercise, comprising: a mobile internet device configured to be carried on a person, the mobile internet device including: an integral digital camera, and a generic port providing a connection to an exercise monitoring device; a communications port providing an interactive wireless connection with one or more internet servers; a non-transitory computer readable medium having instructions stored thereon for providing a user interface for displaying visual images including graphics, photographs, and video clips; the non-transitory computer readable medium further having instructions stored thereon for performing a method comprising steps of: receiving a first type of data that is physiological data measured during exercise, and receiving a second type of data that is an amount of exercise work performed; receiving a third type of data representing visual images captured by the camera, and transmitting the first type of data measuring the physiological data, the second type of data measuring the quantitative amount of exercise work performed, and the third type of data representing the visual images, to one or more internet servers, and a non-transitory computer readable medium residing on one or more internet servers having instructions stored thereon defining a server application for performing a method comprising steps of: receiving the first type of data that is the physiological data, and receiving the second type of data that is a quantitative amount of exercise work performed, and receiving the third type of data representing visual images from the mobile internet device; processing the first, second, and third types of data; transmitting the processed data to the mobile internet device in response; and sharing the physiological data related to exercise, or the data measuring a quantitative amount of exercise work performed, or the visual images related to exercise, or combinations thereof, with multiple other mobile internet devices; wherein the non-transitory computer readable medium of the wireless internet device further has instructions for causing the wireless internet device to receive and display the processed data on the user interface of the wireless internet device.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

(1) FIG. 1 illustrates a general embodiment of a wireless health-monitoring system according to the present invention;

(2) FIG. 2 illustrates an embodiment of a wireless health-monitoring apparatus according to the present invention, showing the system of FIG. 1 up to a point of a wireless antenna 60;

(3) FIG. 3 illustrates an embodiment of a WID employing a camera, which may be integral or not;

(4) FIG. 4 illustrates an embodiment of a back end of an exercise monitoring system according to the present invention;

(5) FIG. 5 illustrates a data flow diagram according to an embodiment of the present invention;

(6) FIG. 6 illustrates an embodiment of a method of use for a wireless application and a server application according to the present invention, in which the same is implemented for exercise monitoring;

(7) FIG. 7 illustrates an embodiment of a wired connection between a HMD and a WID, also showing an optional adaptor; and

(8) FIG. 8 illustrates an embodiment of a wireless connection between a HMD and a WID, showing a different type of optional adaptor than that in FIG. 7.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

(9) Various acronyms are used for clarity herein. Definitions are given below. The term HMD may encompass not only devices with physiological and exercise data sensors but also devices with a keypad, keyboard, mouse, touch screen, pointer, pressure sensor, or other such inputs that the patient or user may employ to perform data entry of the desired parameters. In general, HMDs include some means for determining an exercise or health parameter. For receiving physiological data, an HMD may be a heart rate monitor, a blood pressure monitor, an ambulatory ECG recorder, a respiratory monitor, a temperature monitor and so on.

(10) For receiving exercise data corresponding to the amount of exercise performed, an HMD may be an exercise monitor such as a pedometer, accelerometer, biofeedback device, or other sensor tracking duration, rate, intensity or total amount of exercise, such as number of miles traveled. In a healthy lifestyle management embodiment, an HMD may be incorporated within an exercise machine, including treadmills, rowers, steppers, exercise cycles, or other aerobic or anaerobic exercisers, the same providing exercise data corresponding to the amount of exercise performed such as number of miles traveled, duration of exercise or rate of work performed, and the same may provide physiological data such as heart rate, and additional derived data such as the number of calories.

(11) The term subject as used herein primarily indicates a human subject. The same may be, a person interested in maintaining health via accurate recording of exercise, fitness, and nutrition, and so on, or a medical patient under a physician's care or the care of another healthcare professional The term user is generally used to refer to a user of the WID, which may be synonymous with the subject or may alternatively be another person using a similar WID to review the data from the first user. A user may also be a trainer, fitness coach, physician monitoring an exercise program, or the like. The term patient is used, in addition to a person under the care of a physician, to also refer to a normal or healthy individual who is interested in maintaining a healthy physiological balance.

(12) The term signal communication is used to mean any type of connection between components where the connection is, e.g., electromagnetic, and where the connection allows information to be passed from one component to another. This term may be used in a similar fashion as coupled, connected, information communication, data communication, etc. The following are examples of signal communication schemes. As for wired techniques, a standard USB or serial cable may be used if the input/output ports are compatible and an optional adaptor may be employed if they are not. As for wireless techniques, examples of employable techniques include: infrared (IR), microwaves, radio frequency (RF), a variety of cellular protocols, a variety of 802.11 protocols, 802.15 protocols, 802.16 protocols, 802.20 protocols, other IEEE 802 family protocols, short range wireless transmission, wide area network or broadband transmission methods, such as Bluetooth, wireless universal serial bus protocols (W-USB), WiFi, WiMax, WiFiMax, Long Term Evolution (LTE), VOIP, ultrawideband protocols (UWB), satellite communication protocols or other wireless protocols, or optical techniques including lasers. The user may also input data manually, such as by a stylus or keypad, touchpad or by various other techniques discussed above and below.

(13) The term generic input/output port is used to mean any type of conventional, standard, universal, stock, consumer, or off-the-shelf port for data input and output. These may include both wired and wireless ports and may be located externally or internally to the WID. A further description is given below.

(14) Various embodiments of the invention are now described in more detail.

(15) Referring to FIG. 1, a system is shown for monitoring exercise data from a patient or subject 38. The system includes a wireless exercise monitoring apparatus (WEMA) 10 described in further detail below. WEMA 10 is linked in a wireless fashion to a wireless connection point of presence (POP) 19, the same including at least a base station antenna 15 coupled to a server 17. Server 17 is in turn connected to the wired, or even a wireless (not shown) Internet 21, which may include the World Wide Web.

(16) It should be noted that the base station embodiment shown in FIG. 1 may be replaced or removed for mobile phones which connect via satellite i.e., satellite phones, rather than via a cellular network or other wireless communication.

(17) Referring to FIG. 2, a first embodiment of WEMA 10 is shown. WEMA 10 includes an HMD 11, which may include an optional monitor screen 40, coupled via an optional adaptor 42 to a WID 12. WID 12 connects wirelessly via an antenna 60 to base station 15 (see FIG. 1). One function of WID 12 may be to provide the user interface; other functions are described below.

(18) As noted above, HMD 11 may include a physiological sensor 24 which connects to the HMD via a connection 44 (or which may alternatively connect directly to the WID) or a sensor 24 tracking an amount of exercise performed that connects to the WID (or HMD) via a connection 44; or may include a manual system 36 for input of physiological and exercise data via a connection 34. Manual system 36 may also be used to input data directly into WID 12 via a connection 32. Manual system 36 may include, e.g., a keyboard or keypad 30, a mouse 26, a pen-type device 28, touch screen 28 and may also employ a separate monitor (not shown). Of course, the user may also view information on monitor 40 or on a screen 41 of WID 12. In many embodiments, the touch screen or keypad system employed by many current mobile phones such as the iPhone, may be preferred for such manual data input.

(19) Data may also be input via entry on a computer 37. This data may then be synchronized to WID 12 in known fashion. Alternatively, computer 37, or another computer may be used to connect to a server using the wired internet. This use may be particularly advantageous when entering a large amount of data. As noted above, in this way the user may be afforded a more convenient environment in which to manipulate data to supplement the data input to the WID.

(20) It will be clear to one of skill in the art given this teaching that cable 32, as well as cables 34 and 44, may be replaced with wireless circuitry to communicate signals wirelessly. Alternatively, cables 34 or 44 may be replaced by a direct plug and socket connection. In this connection, adaptor 42 may be a direct plug and socket connection.

(21) For exercise devices and applications, physiological sensor 24 may include, e.g., a sensor appropriate for measuring heart rate, respiration, blood glucose levels, blood oxygen levels, blood pressure, electrocardiograms (ECG), or any other desired physiological parameter. Sensor 24 may also include a camera configured as a heart pulse monitor by detecting color or heat changes. Sensor 24 may connect via an optional cable 44 to subject 38. Alternatively, sensor 24 may be distal of HMD 11, i.e., at or within subject 38. In other words, if cable 44 is employed, sensor 24 may be proximal or distal of cable 44. In some applications, such as some types of cardiac monitoring, sensor 24 is implanted within the patient. The sensor 24 tracking the exercise performed may include, e.g. a pedometer, accelerometer, biofeedback device, timer, GPS device, or other sensor tracking duration, rate, intensity or total amount of exercise. Either or both the physiological sensor and/or sensor tracking the amount of exercise may be incorporated in exercise machines such as a treadmill, exercise bicycle, stepper and so forth. Alternatively, other exercise monitors may also be employed so long as the measured data may either be transferred to WID 12, e.g., via optional adaptor 42, described in further detail below, or by being read by a user, e.g., from a display, and manually input to the WID.

(22) If a wireless communications capability is added, sensor 24 or sensor 24 need not physically connect with HMD 11 or WID 12 at all. That is, the same may measure an exercise parameter and may communicate the same to WEMA 10 wirelessly. The short-range wireless communications schemes which may be employed include infrared, radio frequency including a variety of 802.15 protocols such as Bluetooth, a variety of 802.11 protocols such as WiFi or any other such wireless transmission schemes, including those described above.

(23) As examples of sensor types, to measure heart rate, sensor 24 may be placed via a chest band or an adhesive sensor disposed on the chest. As an example of sensor type 24 to measure the amount of exercise performed, a pedometer may be disposed on the user to track the number of miles traveled. Other exercise monitors may also be employed so long as the measured data may either be transferred to WID 12, e.g., via optional adaptor 42, described in further detail below, or by being read by a user, e.g., from a display, and manually input to WID 12. Alternatively, the measured data may be transferred to WID 12 via wireless communication schemes, using a variety of 802.15 protocols such as Bluetooth, a variety of 802.11 protocols such as WiFi, infrared, optical, microwaves, etc., directly from sensor 24 or sensor 24 or from HMD 11 as described in greater detail below. HMD 11 may be a wearable device, such as a watch in wireless communication with sensor 24, e.g. a heart rate chest band; or sensor 24 can be integrated with the watch, e.g. a resident pulse monitor. Sensor 24 may also be integrated in an article of clothing, e.g. smart apparel. The wearable device may communicate wirelessly with WID 12, e.g. a mobile phone, or with the Internet 11 directly via other transceiver means.

(24) The user, who may or may not be the same person as subject 38, may input data to WID 12 to supplement the measured data. For example, in a health or exercise system, if subject 38 consumes a known number of calories, this information may be entered via manual system 36 directly into WID 12 or into HMD 11. Further, the user, the subject, and the sensor are not necessarily the sole sources of information. Data stored on the server, or on a separate server operated for health management, may also be employed to result in a health benefit to subject 38.

(25) The data used to benefit the subject 38 may be from a camera as well as from an HMD. Referring to FIG. 3, an embodiment of a WID 244 is shown equipped with a camera 224 and an optional memory device 246. The camera may be integral to the WID, or may be separate and connected via a cable. The camera may be incorporated in a wearable WID, e.g smart

(26) Particularly important types of visual data about a user that may be wirelessly transmitted from the WID are those corresponding to photographic pictures, both still and video, and graphical or visual data output images from an HMD, e.g., an exercise machine. In either case, a trainer, coach, or medical personnel may use such data to take appropriate action.

(27) To obtain the first type of data noted above, photographic data, a camera may be employed, which is either resident on the WID or is otherwise available by way of a wired or wireless link. The WID may store photographic data, either still or video, and transmit the same wirelessly to a point of collection, e.g., a server application, or may save the photographic data on the memory device or in other device storage for contemporaneous or later transmission, again either via a streamed, non-streamed, or other transmission method. The WID may be in the form of a wearable device equipped with a camera, e.g. eyewear or watch. As noted above, the WID may connect with a physiological sensor 24, e.g. a distal or integrated pulse monitor, and with a sensor 24 to measure the amount of exercise performed, e.g. a distal or integrated GPS device.

(28) There are various ways in which visual data may be communicated to a remote reviewer. A simple method is to send the visual data via an email message. In more advanced methods, the visual data may be integrated with a data stream of other medical information. Current systems may be used in combination with the present invention to facilitate the handling and transmission of visual data by a WID. In all cases, the visual data may be stored, e.g., as a separate file or may be included as an embedded object in a data file on the memory device or in an email.

(29) In more detail, a data port from a HMD that is coupled to a user may be employed to send visual information from the same to an input port on the WID. This transmission may be accomplished via the techniques described above. Such data may be in either a raw form or in a preformatted-for-video form, and may be stored in the WID or on the memory device. In either case, it may be required to format the data in a way suitable for the display screen of the WID. In some cases, viewing on the WID is not necessary, and the data may be sent in its original form, optionally undergoing some intermediate processing, directly on to the trainer, coach or other remote reviewer's system for viewing or analysis. Examples of this type of data may pertain to the performance of exercise data or other sources of data. The remoter reviewer may include another user, similarly equipped with a WID, e.g. when the users are engaged in team activities.

(30) Alternatively, a subset of the data may be sent, such as a compressed version, while the remainder, i.e., the complete version of the user data, may be maintained on the WID and/or memory device for purposes of maintaining a complete user record.

(31) The memory device 246 may be a smartcard, a smartmedia card, a memory card, memory stick, compact flash card, memory cubes, micro-drives, disk-on-keys, flash memory-keys, micro-laser disks, nano-storage devices, bio-memories, battery/memory combination device, USB flash drives, and so on, or indeed any other type of removable media that may be connected to a WID to store information. Typically, these memory devices are capable of storing substantial amounts of data. The same may also include a memory and power source or combination device. In another embodiment, the memory device 246 may be inserted (not shown) in a memory device reader, which is in turn connected to a WID via a link.

(32) Of course, in some devices, including some current smart mobile phones or other wireless mobile devices such as netbooks, there is no need for a separate memory device 246 as the internal storage capacity, e.g. in the form of solid state memory, microdrive or other memory storage devices, is sufficient to store all applications and data. Whether via a memory device 246 or internal storage, enhanced functionality and storage are provided for the WIDs 244 or 250. This may be particularly important for exercise data, as certain physiological monitoring apparatuses produce copious amounts of data, e.g., cardiac monitoring equipment, and thus require substantial storage capabilities. This is particularly true for memory-intensive video and multimedia content.

(33) Another reason such memory devices are particularly pertinent in medical device monitoring is that they store data which can then be wirelessly transmitted in a streamed or non-streamed fashion. In the event of drop-outs, interruptions, or unavailability of the wireless network, no loss of data occurs, as the data has been stored on the memory device and may be wirelessly transmitted at a later time when cellular or mobile service is again available. The memory device thus serves as a backup storage media. In the event of an extended period of unavailability of a wireless network, the memory device may be replaced or overwritten to provide practically unlimited storage until such time as the network is available and the data can be uploaded. In more detail, in the case of a dropout or other disruption of wireless service, the data may be stored on the memory device or in the WID if it has not already been, as may be the case for streamed data. The WID may periodically test for the availability of the wireless network, and may wait until the network is available. Once the system is again available, the advice from the trainer, coach or other reviewer may be sent to the WID and the user may again take action.

(34) Of course, even if the wireless network is available, the memory device or on-board WID memory may store the data for various purposes. This real-time capability and robustness is often very important in ensuring user safety and ensuring that a high level of care is being delivered to the user by the trainer, coach, or medical personnel, particularly in field situations, such as running or, bicycling, where the wireless connection may be the only source of communication. A related reason why memory devices are particularly pertinent in exercise device monitoring is that they allow a greater level of buffering for real-time data monitoring, thus allowing more pre-analysis and filtering of data.

(35) A further benefit of the use of memory devices is that they provide for easy application downloading onto a WID. For example, a memory device may be inserted into a WID and a large application program may be easily downloaded onto the computer-readable media of a WID from the memory device rather than through a wired or wireless synchronization or downloading process via a PC or the internet or both. Appropriate types of computer-readable media have been described above. Downloading in this fashion may be particularly rapid and complete. The downloaded data may include visual data, such as still or video photographic images, that instruct a user on the operation of a device. In an alternative embodiment, visual data need not be downloaded but may rather be streamed, either from a stored video on a server or in real-time via a user with a webcam. In downloaded or streamed but generally not live systems, the user interface may be interactive, allowing the user to access a knowledge database resident on the server or memory device or previously downloaded onto the memory of the WID.

(36) Referring to FIG. 4, WEMA 10 is shown communicating wirelessly with the Internet. In doing so, WEMA 10 generally sends a wireless signal to a base station 14 (in known fashion) that is connected to a server 18 that is in signal communication (in known fashion) with the internet 20. Server 18 communicates via a protocol (in known fashion) to Internet 20, which also communicates via a protocol (in known fashion) to a server 22 running an application 62. Server 22 may be accessed (in known fashion) by a client computer 44 through a connection 64.

(37) As noted, the protocols for data communication are known. They include a variety of cellular protocols, a variety of 802.11 protocols, 802.15 protocols, 802.16 protocols, 802.20 protocols, other IEEE 802 family protocols, wide area network or broadband transmission methods, WiFi, WiMax, WiFiMax, Long Term Evolution (LTE), VOIP, ultrawideband protocols (UWB), or other wireless communication protocols, and may include a satellite instead of ground-based communication systems. However, they currently vary amongst known techniques. The present invention is not limited to any particular protocols, and may be implemented in any languages supported by the WID and server. In particular, the wireless communications schemes envisioned by the present invention include cellular, mobile, satellite, and other such wireless techniques. In such wireless communication systems, an additional security layer may be employed, including industry-standard encryption and decryption of the transmitted data, especially as health information is highly sensitive and private data.

(38) Of course, as computing capabilities continue to increase, it is expected that the capabilities of WEMA 10, servers 18 and 22, as well as application 62 and client 44, and other components, will correspondingly increase.

(39) Application 62 running on server 22 which is stored in computer readable media as described above may interact with WEMA 10 in a number of ways. Referring to FIG. 5, WEMA 10 is shown in signal communication with server 22 via a connection 72. Connection 72 schematically represents the wireless Internet connection and intervening pathways. WEMA 10 includes an application that may be viewed as having two components: a base wireless or device application 70 and an application presentation layer or user interface 68. User interface 68 is employed to, e.g., present a menu of options to the user, to allow the user to choose inputs, and to generally operate the device. User interface 68 may vary widely in sophistication, e.g., from a simple data entry field to a full graphical user interface. These applications may accept as inputs data from a sensor 24 and sensor 24 as well as from a manual input 36.

(40) Server 22 has a base server application 62 stored on computer-readable media as described above with which the same calculates or provides a response based at least in part on data from WEMA 10. Application 62 may include an algorithm 63 for analyzing data from the HMD, and either application 62 or algorithm 63 may optionally access data from an external data source 74 and may further consult an artificial intelligence system 76. Server 22 may be accessed by a remote computing device 37 by the subject user, a trainer, coach, or any other reviewer.

(41) External data source 74 may be a memory or disk or other such storage that stores health data, such as healthy and unhealthy weight/height ranges, healthy and unhealthy cholesterol counts, the user's prior medical or health history, healthy and unhealthy heart rate values, information corresponding to the caloric and other nutritional content of foods, information corresponding to the caloric expenditure values of various exercises, algorithms for calculating various health parameters, etc. In general, any data that may benefit the health of a user may be stored in external data source 74. External data source 74 may be a memory device or other such storage that stores supplemental data such as treatment protocols. In general, any data that may benefit or otherwise affects the health condition of a user may be stored in external data source 74. External data source 74 may also include online access of health information from external databases or other sources.

(42) As noted, application 62 or algorithm 63 may also consult AI system 76 for suggestions as to health benefits. AI system 76 may even interact with external data source 74 to extract useful information from the same. AI system 76 may employ, e.g., case-based reasoning, rules-based systems, collaborative filtering, neural networks, expert systems, or other such systems as are known.

(43) It should also be noted that each of application 62, algorithm 63, external data source 74, or AI system 76, may physically reside on more than one server, e.g., on an array of servers for, e.g., storage or multiple processing purposes. Each of application 62, algorithm 63, external data source 74, or AI system 76, or combinations of each, may also respectively reside on different servers.

(44) The extent to which server application 62 interacts with wireless application 70 depends on the use to which the system is put. For example, in a less interactive embodiment, device application 70 may act to measure a user's heart rate during exercise and report the same to server application 62. In this case, a trainer may simply review the measured value and send the user a response reporting that the value is acceptable or not. In a highly interactive embodiment, a user may have numerous HMDs 11 measuring a variety of physiological data and/or other data tracking the amount of exercise performed, connected via optional adaptors to a WID 12, and wireless application 70 may correspondingly send a large amount of exercise data to server application 62. The trainer, coach, or physician accessing server application 62, may in turn send detailed plans for an exercise regimen via connection 72. The received data may be analyzed using algorithm 63, external data source 74, and AI system 76. In this sense, the two applications may be highly interactive.

(45) It is noted that an Application Service Provider (ASP) may operate application 62. That is, application 62 may be leased by an ASP to a company specializing in fitness or lifestyle management, and the ASP may perform all necessary upgrades and maintenance to application 62 and its associated components.

(46) To initialize the system, the program starts and a wireless application is loaded into computer readable media in the WID, if it has not been pre-loaded. The initial loading of the wireless application may occur via synchronization from a desktop or via downloading from a server over the internet. The server application may be loaded into computer readable media in an appropriate internet-connected server. Subject data may be loaded into the WID or into the server application. In the latter case, the subject information may later be transferred to the WID. The initialization scheme then ends.

(47) The wireless application may access the server and server application, or vice-versa, as determined by the respective program instructions.

(48) Referring to FIG. 6, a flowchart of a method is shown for a system of exercise, fitness, nutrition or health monitoring, and/or exercise management. In this example, the HMD is in the form of a general exercise machine such as a treadmill, stepper, exercise bicycle and so forth. An exercise monitor may be integral to the exercise machine or may be separate.

(49) In the method, both exercise data and visual data may be wirelessly transmitted from the WID. Visual data include that corresponding to photographic pictures, both still and video, and graphical or visual data output images from an HMD, e.g., an exercise machine or equipment display screen, or other images related to the exercise being performed. A trainer or coach may review these images and use them to guide the user as appropriate as described below.

(50) After the start (step 242) of the method, a user interacts with a WID (step 240). By the interaction, visual data may be captured (step 241), and/or the user may send supplemental data to the server and server application (step 241).

(51) To obtain the visual data noted above, photographic data, a camera may be employed, which is either resident on the WID (camera 224 of FIG. 3) or is otherwise available by way of a link. The WID may store photographic data, either still or video, and transmit the same wirelessly to a point of collection, e.g., a server application, or may save the photographic data on the memory device for contemporaneous or later transmission, again either via a streamed, non-streamed, or other transmission method (step 208 of FIG. 6).

(52) Also after the program is started (step 242), a sensor measures an exercise parameter (step 216), where the exercise parameter corresponds to physiological data or to data corresponding to the amount of exercise performed.

(53) The exercise monitor sensor may send the parameter to the exercise machine (step 218) or the parameter may be sent directly to the WID (step 218). It is understood here that the sensor may pertain to any of the exercise monitors discussed above.

(54) If the parameter is sent to the exercise machine as an intermediate step, the exercise machine then sends the parameter to the WID (step 220). In any case, the WID wirelessly communicates the parameter to the application server (step 222), e.g., via the wireless web.

(55) An alternative and complementary way of entering the parameter is by user input. For example, the user may enter the parameter into the exercise machine or into the WID. This parameter may correspond to an amount of exercise performed, an amount of food consumed, etc. Such data and other types are termed supplemental data, and may be transmitted to the server and server application (step 241). Calculations by the server application may take into account the supplemental data as well as the visual data and exercise data.

(56) The server application processes the parameter (step 224 and optionally step 225), and calculates a response (step 226) based at least in part on the parameter. The server application may optionally employ external data (step 232) or an AI system (step 234) in the calculation. The application server then sends the response to the WID (step 228), where the response is displayed (step 230). Alternatively, the application server may sends the response to an alternate WID (step 228), such as that of a coach or teammate, where the response is displayed (step 230).

(57) The same definitional statements regarding the terms response, calculate, sensor, etc., as given before, apply in this embodiment as well.

(58) As an optional step, a trainer, coach, or other specialist may notify the user of the response (step 238) after having the same displayed on their client computer (step 236). The specialist may be replaced in this example by an application that may also include an algorithm.

(59) To devise the exercise data mentioned above, a data port from a HMD that is coupled to a user may be employed to send information from the same to an input port on the WID. This transmission may be accomplished via the techniques described above. Such data may be in either a raw form or in a preformatted-for-video form, and may be stored in the WID or on the memory device. In either case, it may be required to format the data in a way suitable for the display screen of the WID. In some cases, viewing on the WID is not necessary, and the data may be sent in its original form, optionally undergoing some intermediate processing, directly on to the remote reviewer's system for viewing or analysis. Examples of this type of data may be the display output of an exercise machine or other sources of data. Alternatively, a subset of the data may be sent, such as a compressed version, while the remainder, i.e., the complete version of the user's data, may be maintained on the WID and/or memory device for purposes of maintaining a complete user record.

(60) The WID may store the HMD data, and transmit the same wirelessly to a point of collection or may save the data on the memory device for contemporaneous or later transmission, again either via a streamed or other transmission method. The memory device may be any type of computer-readable media, including, e.g., a smartcard, a smartmedia card, a memory card, memory stick, compact flash card, memory cubes, micro-drives, disk-on-keys, flash memory-keys, micro-laser disks, nano-storage devices, bio-memories, battery/memory combination device, USB flash drives, and so on, or indeed any other type of removable media that may be connected to a WID to store information.

(61) In the case of a dropout or other disruption of wireless service, the data may be stored on the memory device or in the WID if it has not already been, as may be the case for streamed data. The WID may periodically test for the availability of the wireless network, and may wait until the network is available. Once the system is again available, the advice from the caregiver may be sent to the WID and the on-site personnel may again take action.

(62) Using such data, an off-site trainer, coach, or other health care professional may give the user valuable guidance and advice. Moreover, no wired or dedicated connection is necessary.

(63) More particularly, for visual data, there are various ways in which such data may be communicated to an off-site reviewer. A simple method is to send the visual data via an email message. In more advanced methods, the visual data may be integrated with a data stream of other information. Current systems may be used in combination with the present invention to facilitate the handling and transmission of visual data by a WID. In all cases, the visual data may be stored, e.g., as a separate file or may be included as an embedded object in a data file on the memory device or in an email.

(64) Adaptor Hardware

(65) A description is given below of a particular type of adaptor hardware. As noted above, the adaptor may optionally be used to connect a HMD to a WID.

(66) In general, a connection is necessary between a HMD 11 and a WID. The nature of this connection may vary. For example, the connection may be wired or wireless. For wired systems, the connection may be direct or an adaptor may be employed, either on one or both ends of the direct wired connection, to adapt the signal appropriately. In the same way, for wireless systems, the connection may be direct, if both HMD and WID employ the same wireless protocol, or an adaptor may be involved to modify the signal of one or both devices. These connections, all of which are encompassed by the present invention, are discussed in more detail below.

(67) Referring to FIG. 7, an embodiment of the connection is shown. In this figure, a hardware (or wired) connection is shown, i.e., an adaptor 168, disposed between a HMD 160 and a WID 162. The HMD 160 may connect to an exercise machine via a connector 52 with optional adaptor 124. This connection may also be wireless as has been described. In most circumstances, it is assumed that the varieties of HMDs will vary more widely than the varieties of WIDs. Accordingly, HMD 160 will likely have one of a variety of types of connectors for input/output purposes, here shown as a connector 164. Connector 164 mates with a connector 172 of adaptor 168. At another point on adaptor 168 is a connector 172 for connecting to a generic input/output port 164 on WID 162. A cable 170 is disposed between the two connectors, cable 170 capable of including adaptor circuitry if desired.

(68) Of course, the use and structure of adaptor 168, between HMD 160 and WID 162, depends on factors such as the prevalence of an industry standard for such communications. In other words, if the output of HMD 160 is readily acceptable to WID 162, then the same may be directly connected or may be connected via a simple cable, the same basically extending from pin-to-pin. For example, a standard Universal Serial Bus (USB) or serial cable (RS232) may be used if the input/output ports between the HMD and the WID are compatible. Otherwise, a suitable adaptor circuit that provides for appropriate signal and pin conversion may be employed. For example, a standard USB-to-serial (RS232) cable or the like may be used as required. The connection may also be an input for data, e.g. a USB port or phone jack or other such wired input, or a media storage device, again employing an adaptor circuit if required.

(69) Port 164 can be used to communicate with HMD 160 and connector 164 via a number of types of wired connections, including USB, or Firewire. In some cases, optional adaptor 168 may also be employed.

(70) In other embodiments, such as for devices connected to proprietary connectors, a less standard and perhaps proprietary pin-out may be required.

(71) Referring to FIG. 8, an embodiment of a wireless implementation of the WEMA 10 is shown. In FIG. 8, a wireless connection is shown between HMD 160 and WID 162. HMD 160 may have an integral wireless modulator/demodulator disposed within (not shown). More likely, however, is that HMD 160 has an adaptor 154 connectable thereto which performs these functions. Adapter 154 may plug into a connector 165 on HMD 160. WID 162 may have an integral wireless modulator/demodulator (not shown), although an adaptor can also be used in this context.

(72) While the device shown in FIG. 8 is described in the context of general wireless communications, various protocols may be employed. For radio frequency communications, a variety of 802.11 protocols, 802.15 protocols, other IEEE 802 family protocols, short-range wireless transmission methods such as Bluetooth, wireless universal serial bus protocols (W-USB), or other wireless transmission methods may be advantageously employed. Other techniques employing a similar configuration include those employing IR, microwaves, optical techniques including lasers, and so on.

(73) It should be understood that the above is merely exemplary, and that the form of the adaptor may vary widely between HMDs and WIDs.

(74) As noted above, besides the exercise data transmission from HMDs, other sorts of transmissions may also occur. For example, visual data, such as photographs or videos, may be transferred as an indication of the user's performance condition and to aid remote analysis. Alternatively, other visual indications of a user's status, such as graphical or other outputs of HMDs, may provide information useful for a trainer, coach, or other reviewer.

(75) In certain embodiments, a set of visual data from a camera or from an HMD and voice communication may be transmitted via the telecommunications infrastructure from the WID. The visual data may thus be sent via an appropriate protocol to a server for retrieval and analysis by a trainer, coach or other reviewer.

(76) The advent of multimedia mobile phones and other WIDs that include a digital camera (or are equipped with a link to one) allow the capture and transmission of photographic images using low-cost consumer devices. Embodiments of the invention may employ these in combination with HMDs.

(77) It will be understood that the above description of a Method And Apparatus For Exercise Monitoring Combining Exercise Monitoring and Visual Data With Wireless Internet Connectivity has been with respect to particular embodiments of the invention. While this description is fully capable of attaining the objects of the invention, it is understood that the same is merely representative of the broad scope of the invention envisioned, and that numerous variations of the above embodiments may be known or may become known or are obvious or may become obvious to one of ordinary skill in the art, and these variations are fully within the broad scope of the invention. For example, while certain wireless technologies have been described herein, other such wireless technologies may also be employed. Furthermore, while various types of exercise monitors have been mentioned, numerous other types may also be used in the embodiments of the invention, including types of devices that are incorporated within the WID. Accordingly, the scope of the invention is to be limited only by the claims appended hereto, and equivalents thereof. In these claims, a reference to an element in the singular is not intended to mean one and only one unless explicitly stated. Rather, the same is intended to mean one or more. All structural and functional equivalents to the elements of the above-described preferred embodiment that are known or later come to be known to those of ordinary skill in the art are expressly incorporated herein by reference and are intended to be encompassed by the present claims. Moreover, it is not necessary for a device or method to address each and every problem sought to be solved by the present invention, for it to be encompassed by the present claims. Furthermore, no element, component, or method step in the present invention is intended to be dedicated to the public regardless of whether the element, component, or method step is explicitly recited in the claims. No claim element herein is to be construed under the provisions of 35 U.S.C. 112, para. 6, unless the element is expressly recited using the phrase means for.