SYSTEMS FOR OPTIMIZING POWER CONSUMPTION OF A WEARABLE DEVICE USING SENSOR-BASED DETERMINATIONS OF POSITIONAL STATES OF A PORTION OF THE WEARABLE DEVICE, AND METHODS OF USE THEREOF
20230337981 · 2023-10-26
Inventors
- Nishant Srinivasan (San Francisco, CA, US)
- Nagalakshmi Rajagopal (Sunnyvale, CA, US)
- Derek William Wright (San Francisco, CA, US)
- Edwin Corona Aparicio (Wahiawa, HI, US)
- Szymon Michal TANKIEWICZ (San Jose, CA, US)
- Ravi Krishna Shaga (Fremont, CA, US)
- Ramiro CALDERON (New York, NY, US)
- Shan Chu (Syosset, NY, US)
- Priyanka SHARMA (Cupertino, CA, US)
- Lei Yin (Santa Clara, CA, US)
- Lidu Huang (Danville, CA)
Cpc classification
H02J3/00
ELECTRICITY
A61B5/721
HUMAN NECESSITIES
International classification
Abstract
Methods and wearable devices for optimizing power consumption using sensor-based position and use determinations are described here. One example method is performed at a device that includes a first sensor configured to operate with a first power consumption rate and a second sensor configured to operate with a second power consumption rate. The method includes, while a component associated with the second sensor is in an inactive state, receiving first sensor data, and determining whether the first sensor data indicates movement of the device. The method also includes, when movement of the device is indicated, operating the second sensor in an active state. The method further includes, after activating the second sensor, when second sensor data from the second sensor indicates that the device has been placed on a user’s body, continuing to operate the second sensor in the active state.
Claims
1. A method of optimizing power consumption in a wearable device, the method comprising: at a wearable device that includes a first sensor configured to operate with a first power consumption rate and a second sensor configured to operate with a second power consumption rate that is greater than the first power consumption rate: while a component associated with the second sensor is in an inactive state: receiving, from the first sensor, first sensor data; determining whether the first sensor data indicates movement of the wearable device; in accordance with a determination that the first sensor data indicates movement of the wearable device: operating the component of the second sensor in an active state in which the component associated with the second sensor is used to actively sense or process sensor data; after activating the component associated with the second sensor, receiving, from the second sensor, second sensor data; and in accordance with a determination that the second sensor data indicates that the wearable device has been placed on a user’s body, continuing to operate the component associated with the second sensor in the active state.
2. The method of claim 1, wherein: the first sensor is an inertial measurement unit (IMU) sensor that is configured to detect data indicating movements of the wearable device; and the second sensor is an electromyography (EMG) sensor that is configured to detect neuromuscular signals indicating muscular movements of a user.
3. The method of claim 1, wherein: the first sensor is a hall effect sensor (HES) that is configured to be used to check if a display portion of the wearable device is attached or detached to a cradle portion of the wearable device; and the second sensor is photoplethysmography (PPG) sensor that is configured to be used to detect changes in blood flow in a user.
4. The method of claim 1, wherein: the first sensor is a hall effect sensor (HES) that is configured to be used to check if a display portion of the wearable device is attached or detached to a cradle; and the second sensor is an inertial measurement unit (IMU) sensor that is configured to detect data indicating movements of the wearable device.
5. The method of claim 1, wherein the first sensor has a first power consumption rate between 2 to 5 mW/s (milliwatts per second).
6. The method of claim 1, wherein the second sensor has a second power consumption rate between 7 to 17 mW/s.
7. The method of claim 1, wherein the component associated with the second sensor is operated in the active state for a period of time, and, after the period of time, the method further comprises: while the component associated with the second sensor is in the inactive state: receiving, from the first sensor, new first sensor data; determining whether the new first sensor data indicates movement of the wearable device; in accordance with a determination that the first sensor data indicates that the wearable device has not moved, continuing to operate the component of the second sensor in the inactive state.
8. The method of claim 1, wherein the continuing to operate the component associated with the second sensor in the active state includes continuing to operate the component associated with the second sensor in the active state until a deactivation trigger is detected.
9. The method of claim 8, wherein the deactivation trigger is detected when data from the first sensor that indicates that the wearable device has been removed from the user’s body.
10. The method of claim 8, wherein the deactivation trigger is detected when data from the second sensor indicates that the wearable device has been remove from the user’s body.
11. The method of claim 1, wherein the determination that the second sensor data indicates that the wearable device has been placed on the user’s body includes determining that the second sensor data, as compared to subsequent sensor data from the second sensor, reflects a change in data sensed by the second sensor.
12. The method of claim 11, wherein: the second sensor data indicates that one or more sensing channels of the second sensor are receiving data at or above a noise threshold, and the subsequent sensor data indicates that the one or more sensing channels of the second sensor are receiving data below the noise threshold.
13. The method of claim 1, wherein operating the component of the second sensor in the active state includes causing an interrupt signal to be sent to the component associated with the second sensor to cause it to transition from the inactive to the active state.
14. The method of claim 1, wherein determining whether the first sensor data indicates movement of the wearable device includes determining whether the movement is consistent with one or more known movements associated with the user placing the wearable device on the user’s body.
15. The method of claim 1, wherein the wearable device is a wrist-wearable device or a head-worn wearable device.
16. The method of claim 1, comprising: while the wearable device has been placed on the user’s body: in accordance with a determination that a display portion of the wearable device has been detached from a cradle, locking the wearable device until an authentication input is received.
17. The method of claim 16, wherein the authentication input is a biometric input.
18. The method of claim 17, wherein the biometric input is a predefined movement pattern of a finger of the user in front of a camera of the wearable device or an image of the finger of the user that is captured by the camera of the wearable device to detect at least one unique biometric characteristic of the finger.
19. A wrist-wearable device that includes a first sensor configured to operate with a first power consumption rate and a second sensor configured to operate with a second power consumption rate that is greater than the first power consumption rate, and the wrist-wearable device is configured to perform or cause performance of: while a component associated with the second sensor is in an inactive state: receiving, from the first sensor, first sensor data; determining whether the first sensor data indicates movement of the wearable device; in accordance with a determination that the first sensor data indicates movement of the wearable device: operating the component of the second sensor in an active state in which the component associated with the second sensor is used to actively sense or process sensor data; after activating the component associated with the second sensor, receiving, from the second sensor, second sensor data; and in accordance with a determination that the second sensor data indicates that the wearable device has been placed on a user’s body, continuing to operate the component associated with the second sensor in the active state.
20. A non-transitory, computer-readable storage medium including instructions that, when executed by a wrist-wearable device that includes a first sensor configured to operate with a first power consumption rate and a second sensor configured to operate with a second power consumption rate that is greater than the first power consumption rate, cause the wrist-wearable device to: while a component associated with the second sensor is in an inactive state: receive, from the first sensor, first sensor data; determine whether the first sensor data indicates movement of the wearable device; in accordance with a determination that the first sensor data indicates movement of the wearable device: operate the component of the second sensor in an active state in which the component associated with the second sensor is used to actively sense or process sensor data; after activating the component associated with the second sensor, receive, from the second sensor, second sensor data; and in accordance with a determination that the second sensor data indicates that the wearable device has been placed on a user’s body, continue to operate the component associated with the second sensor in the active state.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
[0073] For a better understanding of the various described embodiments, reference should be made to the Detailed Description below, in conjunction with the following drawings in which like reference numerals refer to corresponding parts throughout the figures.
[0074]
[0075]
[0076]
[0077]
[0078]
[0079]
[0080]
[0081]
[0082]
[0083]
[0084]
[0085]
[0086]
[0087] In accordance with common practice, the various features illustrated in the drawings may not be drawn to scale. Accordingly, the dimensions of the various features may be arbitrarily expanded or reduced for clarity. In addition, some of the drawings may not depict all of the components of a given system, method or device. Finally, like reference numerals may be used to denote like features throughout the specification and figures.
DETAILED DESCRIPTION
[0088] Reference will now be made in detail to embodiments, examples of which are illustrated in the accompanying drawings. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the various described embodiments. It will be apparent to one of ordinary skill in the art that the various described embodiments may be practiced without these specific details. In some instances, well-known methods, procedures, components, circuits, and networks have not been described in detail so as not to unnecessarily obscure aspects of the embodiments.
[0089] As will become apparent to a person of skill in this art upon reading this disclosure, the various embodiments provide systems and methods of providing optimized power (activating and deactivating sensors as needed to conserve power) consumption of a wearable device based on determined states of the wearable device.
[0090] An example method of optimizing power consumption in a wearable device, is performed at a wearable device (or at least a portion of the wearable device) that includes a first sensor configured to operate with a first power consumption rate and a second sensor configured to operate with a second power consumption rate that is greater than the first power consumption rate. The method includes, while a component associated with the second sensor is in an inactive state: receiving from the first sensor, first sensor data, and the wearable device determines whether the first sensor data indicates movement of the wearable device. The method also includes, in accordance with a determination that the first sensor data indicates movement of the wearable device, operating the component of the second sensor in an active state in which the component associated with the second sensor is used to actively sense or process sensor data. The method includes, after activating the component associated with the second sensor, receiving, from the second sensor, second sensor data. The method further includes, that in accordance with a determination that the second sensor data indicates that the wearable device has been placed on a user’s body, continuing to operate the component associated with the second sensor in the active state.
[0091] Another example method uses three types of sensors to determine cradle, wrist, and activity states of a wrist-wearable device, and is performed at a wrist-wearable device that includes a capsule portion that is configured to be removably coupled to a cradle portion of the wrist-wearable device, the wrist-wearable device including a first sensor of a first sensor type and a second sensor of a second sensor type distinct from the first sensor type. The method includes, determining, using sensor data from the first sensor of the first sensor type, whether the capsule portion of the wrist-wearable device is coupled with the cradle portion of the wrist-wearable device. The method includes, in accordance with a determination that the data from the first sensor of the first sensor type indicates that the capsule portion of the wrist-wearable device is coupled with the cradle portion of the wrist-wearable device, obtaining data from the second sensor of the second sensor type. The method also includes, in accordance with a determination that the data from the second sensor of the second sensor type indicates that the wrist-wearable device is worn on a wrist of a user, monitoring data from a third sensor of a third sensor type that is distinct from the first and second sensor types to determine an activity being performed by the user while the wrist-wearable device is worn on the user’s wrist.
[0092] In yet another example method, the method includes repurposing a metallic data-transmission component included in an electronic device, and the electronic device includes one or more processors. The method includes that the electronic device receives, via the metallic data-transmission component, operational data indicative of a current operation state of the metallic data-transmission component. The method includes, determining, based on the operational data, whether data-transmission criteria are present. The method includes, in accordance with a determination that the data-transmission criteria are present, operating the metallic data-transmission component in a first mode. In some embodiments, while in the first mode, the metallic data-transmission component is configured to be used as antenna in conjunction with transferring data between the electronic device and at least one other electronic device using the metallic data-transmission component. The method also includes, in accordance with a determination that the data-transmission criteria are not present, operating the metallic data-transmission component in a second mode, in which the metallic data-transmission component functions as a capacitive sensor.
[0093] Small portable devices, such as wearable devices, need to optimize their power usage to provide the best user experience, i.e., long periods of continuous use without needing a charge. The following figures illustrate how a wearable device activates and deactivates various sensors to improve battery life of small wearable devices.
[0094]
[0095]
[0096]
[0097] Using a combination of the first sensor data and the second sensor data, the state determination logic 110 is configured to determine whether the wearable device 100 is on the wrist of the user 104. In accordance with a determination by the state determination logic 110 that the wearable is on the wrist of the user 104, the wearable device 100 continues to keep the second sensor activated. In some embodiments, having the second sensor activated is referred to as a high-power mode. In some embodiments, the high-power mode includes activating one or more sensors that draw more power than other sensors. In some embodiments, the low power sensor (e.g., the first sensor) can be configured to shut-off when the higher-power sensor (e.g., the second sensor) is in use and would otherwise be supplying redundant data.
[0098] While
[0099]
[0100]
[0101] While
[0102]
[0103]
[0104]
[0105] In some embodiments, the chart 210 shows another magnetic field being detected by a hall effect sensor. In some embodiments, the hall effect sensor is relied on to improve the security of the wearable device, by locking the device once the magnetic field is no longer present (e.g., when the capsule portion of the wearable device is detached from the cradle portion of the wearable device). By immediately locking, it ensures that if someone were to steal the capsule portion, they would not be able to access data, unless they unlocked the capsule device by providing authenticating information.
[0106]
[0107]
[0108]
[0109]
[0110]
[0111]
[0112]
[0113] When the hall-effect sensor provides data indicating that the HES is on the cradle, as shown in step 306, the logic controlling the wearable device deactivates the rear-facing camera, enables the front-facing camera, and AFE remains (the ambient light sensor and infrared sensor of the AFE remain enabled).
[0114] Alternatively, when a wearable device is not equipped with a HES, it can rely on inertial measurements sourced from an inertial measurement unit to determine if the wearable device is on or off the cradle.
[0115] Step 308A and 308B illustrate two possible paths the logic of the wearable device can follow, based on the configuration of the wearable device. Step 308A illustrates a first path that the state determination logic follows when a device is equipped with an IMU sensor and an EMG sensor, but not a AFE or a HES. Step 308B illustrates another branch of logic for a wearable device that includes an AFE and a HES.
[0116] In step 308A, the state determination logic uses IMU sensor information to detect active movement of the wearable device to determine whether the watch is on or off the wrist of a user. In some embodiments, the IMU data corresponds to a predetermined pattern that indicates a wrist off state and/or an off-charger state. After the state determination logic has determined that the wearable device is off wrist of a user, using data from the IMU sensor, the state determination logic activates the EMG. Using data from the activated EMG, the state determination logic determines, as shown in step 310, that the wearable device is on a wrist of a user.
[0117] Step 308B illustrates another branch of logic for a wearable device that includes an AFE and a HES. As shown in step 308B, when the data from the HES indicates that is off cradle, the state determination logic initiates an infrared (IR) sensor to determine if the watch is proximate to a wrist of a user.
[0118] Step 312 shows that when the state determination logic determines, using data from the IR sensor, that the wearable device is proximate to something, the state determination logic causes a PPG and/or EMG sensor to be activated. The PPG and/or EMG sensors are capable of providing data that can be used to determine if the wearable device is on a wrist of a user.
[0119] After step 312, the state determination logic can follow two branches depending on what the data from the ppg and/or EMG sensor indicate. Step 314 indicates the first logical branch, which is followed when the state determination logic determines, using data from the PPG and/or EMG sensor to determine if the wearable device is proximate to a wrist or something else (e.g., a tabletop, floor, desk mat (e.g., a surface that could be interpreted as wrist, unless a biometric sensor is used to confirm), etc.). Once the state determination logic determines, using data from the PPG and/or EMG sensor, that the wearable device is on a surface other than a wrist of the user, it forgoes any further operation until it detects that is on a wrist of a user.
[0120] Step 316 indicates the second logical branch, which is followed when the state determination logic determines, using data from the PPG and/or EMG sensor(s), that the wearable device is on a user’s wrist. When the wearable device is on the wrist of the user, the PPG and/or EMG sensor(s) remain operational, and track biometric data. In some embodiments, the sensors run at frequency of 25 Hz. In some embodiments, the frequency is variable depending on determined activity states and runs between 20 Hz-200 Hz.
[0121] After it has been determined that the wearable device has been place on a wrist of a user, the state determination logic then determines, using data from an IMU sensor, an activity being performed by a user. As shown in step 318, when the IMU sensor provides data that indicates the user is sleeping (e.g., the data provided by the IMU corresponds to a predetermined pattern that indicates a user wearing the wearable device is sleeping), the rear-facing camera is disabled, the front-facing camera is disabled, the red LED, the green LED, and the IR on the AFE are enabled, and the sensors are configured to operate at 128 Hz. Step 320 indicates another logical branch using the IMU sensor. When the IMU sensor provides data that indicates that the user is exercising (e.g., the data provided by the IMU corresponds to a predetermined pattern that indicates a user wearing the wearable device is exercising), one or more sensors are operated at 25 Hz, the green LED, and the IR on the AFE are enabled.
[0122]
[0123]
[0124] The following steps in
[0125] Step 422 indicates that a proximity sensor is used to confirm whether an interrupt in receipt of biometric information from a biometric sensor is a wrist off event. In accordance with a determination that the interrupt occurs, and the proximity sensor indicates that the wearable device is not proximate to a surface (e.g., is beyond a threshold distance away from a surface), the state determination logic confirms that the wearable device is off the wrist of a user. Step 422 also shows that in accordance with a determination that the interrupt occurs, and the proximity sensor indicates that the wearable device is proximate to a surface (e.g., is not beyond a threshold distance away from a surface), the state determination logic confirms that the wearable device is still potentially on the wrist of a user and reverts to step 418.
[0126]
[0127]
[0128]
[0129]
[0130]
[0131]
[0132]
[0133] Functions executed by the wrist-wearable device 650 can include, without limitation, display of visual content to the user (e.g., visual content displayed on display screen 115), sensing user input (e.g., sensing a touch on button 658, sensing biometric data on sensor 664, sensing neuromuscular signals on neuromuscular sensor 665, etc.), messaging (e.g., text, speech, video, etc.), image capture, wireless communications (e.g., cellular, near field, WiFi, personal area network, etc.), location determination, financial transactions, providing haptic feedback, alarms, notifications, biometric authentication, health monitoring, sleep monitoring, etc. The wrist-wearable device 650 is configured to perform, without limitation, the functions described above in reference to
[0134] The watch band 662 can be configured to be worn by a user such that an inner surface of the watch band 662 is in contact with the user’s skin. When worn by a user, sensor 664 is in contact with the user’s skin. The sensor 664 can be a biosensor that senses a user’s heart rate, saturated oxygen level, temperature, sweat level, muscle intentions, or a combination thereof (any of these can be examples of the biometric sensor described above and used in conjunction with the positional-state determinations described herein, and can also be associated with the capsule portion instead of the band portion). The watch band 662 can include multiple sensors 664 that can be distributed on an inside and/or an outside surface of the watch band 662. Additionally, or alternatively, the watch body 654 can include the same or different sensors than the watch band 662 (or the watch band 662 can include no sensors at all in some embodiments). For example, multiple sensors can be distributed on an inside and/or an outside surface of watch body 654. The watch body 654 (e.g., a capsule portion) can include, without limitation, a magnetic field sensor 120 (as shown in
[0135] The watch band 662 and/or watch body 654 can include a haptic device 666 (e.g., a vibratory haptic actuator) that is configured to provide haptic feedback (e.g., a cutaneous and/or kinesthetic sensation, etc.) to the user’s skin. The sensor 664 and/or haptic device 666 can be configured to operate in conjunction with multiple applications including, without limitation, health monitoring, social media, game playing, and artificial reality (e.g., the applications associated with artificial reality).
[0136] In some examples, the watch band 662 can include a neuromuscular sensor 665 (e.g., an electromyography (EMG) sensor, a mechanomyogram (MMG) sensor, a sonomyography (SMG) sensor, etc.). Neuromuscular sensor 665 can sense a user’s intention to perform certain motor actions (this sensor 665 can be another example of a sensor used as the biometric sensor in conjunction with the positional-state determinations described herein). The sensed muscle intention can be used to control certain user interfaces displayed on the display 115 of the device 102 and/or can be transmitted to device responsible for rendering an artificial-reality environment (e.g., a head-mounted display) to perform an action in an associated artificial-reality environment, such as to control the motion of a virtual device displayed to the user.
[0137] Signals from neuromuscular sensor 665 can be used to provide a user with an enhanced interaction with a physical object and/or a virtual object in an artificial-reality application generated by an artificial-reality system (e.g., user interface objects presented on the display 115, or another computing device (e.g., a head-mounted display)). Signals from neuromuscular sensor 665 can be obtained (e.g., sensed and recorded) by one or more neuromuscular sensors 665 of watch band 662. Although
[0138] The wrist-wearable device 650 can include a coupling mechanism (also referred to as a cradle) for detachably coupling watch body 654 to watch band 662. A user can detach watch body 654 from watch band 662 to reduce the encumbrance of the wrist-wearable device 650 to the user. The wrist-wearable device 650 can include a coupling surface 656 on the watch body 654 and/or coupling mechanism(s) 660 (e.g., a cradle, a tracker band, a support base, a clasp). A user can perform any type of motion to couple watch body 654 to watch band 662 and to decouple watch body 654 from watch band 662. For example, a user can twist, slide, turn, push, pull, or rotate watch body 654 relative to watch band 662, or a combination thereof, to attach watch body 654 to watch band 662 and to detach watch body 654 from watch band 662.
[0139] As shown in the example of
[0140] The wrist-wearable device 650 can include a single release mechanism 670 or multiple release mechanisms 670 (e.g., two release mechanisms 670 positioned on opposing sides of the wrist-wearable device 650 such as spring-loaded buttons). As shown in
[0141]
[0142] In some examples, watch body 654 can be decoupled from watch body interface 680 by actuation of a release mechanism. The release mechanism can include, without limitation, a button, a knob, a plunger, a handle, a lever, a fastener, a clasp, a dial, a latch, or a combination thereof. In some examples, the wristband system functions can be executed independently in watch body 654, independently in watch body interface 680, and/or in communication between watch body 654 and watch body interface 680. Watch body interface 680 can be configured to operate independently (e.g., execute functions independently) from watch body 654. Additionally, or alternatively, watch body 654 can be configured to operate independently (e.g., execute functions independently) from watch body interface 680. Watch body interface 680 and/or watch body 654 can each include the independent resources required to independently execute functions. For example, watch body interface 680 and/or watch body 654 can each include a power source (e.g., a battery), a memory, data storage, a processor (e.g., a CPU), communications, a light source, and/or input/output devices.
[0143] In this example, watch body interface 680 can include all of the electronic components of watch band 662. In additional examples, one or more electronic components can be housed in watch body interface 680 and one or more other electronic components can be housed in portions of watch band 662 away from watch body interface 680.
[0144]
[0145] For example, watch body 704 can include capacitive sensor 122 (or a device such as an NFC sensor 715 that can in some embodiments act as a capacitive sensor), magnetic field sensor 120, antenna return-loss (RL) sensor 124, biometric sensor 126, battery 728, CPU 726, storage 702, heart rate sensor 758, EMG sensor 746, SpO.sub.2 sensor 754, altimeter 748, IMU 742, random access memory 703, charging input 730 and communication devices NFC 715, LTE 718, and WiFi/Bluetooth 720. Similarly, watch band 712 can include battery 738, microcontroller unit 752, memory 750, heart rate sensor 758, EMG sensor 746, SpO.sub.2 sensor 754, altimeter 748, IMU 742, charging input 734 and wireless transceiver 740. Memory 750 (and/or storage 702) may further include device state table, an example of which is shown in
[0146] Watch body 704 and watch band 712 can further execute instructions stored in storage 702 and memory 750 respectively that enables watch band 712 to transmit the acquired data to watch body 704 (or other computing device such as a head mounted display or other computing device) using wired communications 727 and/or wireless transceiver 740. For example, watch body 704 can display visual content to a user on touchscreen display 713 (e.g., an instance of display 115) and play audio content on speaker 125. Watch body 704 can receive user inputs such as audio input from microphone 127 and touch input from buttons 724. Watch body 704 can also receive inputs associated with a user’s location and/or surroundings. For example, watch body 704 can receive location information from GPS 716 and/or altimeter 748 of watch band 712.
[0147] Watch body 704 can receive image data from at least one image sensor 135 (e.g., a camera). Image sensor 135 can include front-facing image sensor 625A (
[0148] Watch body 704 and watch band 712 can receive a charge using a variety of techniques. In some embodiments, watch body 704 and watch band 712 can use a wired charging assembly (e.g., power cords) to receive the charge. Alternatively, or in addition, watch body 704 and/or watch band 712 can be configured for wireless charging. For example, a portable charging device can be designed to mate with a portion of watch body 704 and/or watch band 712 and wirelessly deliver usable power to a battery of watch body 704 and/or watch band 712.
[0149] Watch body 704 and watch band 712 can have independent power and charging sources to enable each to operate independently. Watch body 704 and watch band 712 can also share power (e.g., one can charge the other) via power management IC 732 in watch body 704 and power management IC 736 in watch band 712. Power management IC 732 and power management IC 736 can share power over power and ground conductors and/or over wireless charging antennas.
[0150] Wrist-wearable device system 700 can operate in conjunction with a health monitoring application that acquires biometric and activity information associated with the user. The health monitoring application can be designed to provide information to a user that is related to the user’s health. For example, wrist-wearable device system 700 can monitor a user’s physical activity by acquiring data from IMU 742 while simultaneously monitoring the user’s heart rate via heart rate sensor 758 and saturated blood oxygen levels via SpO.sub.2 sensor 754. CPU 726 can process the acquired data and display health related information to the user on touchscreen display 713.
[0151] Wrist-wearable device system 700 can detect when watch body 704 and watch band 712 are connected to one another (e.g., mechanically connected and/or electrically or magnetically connected) or detached from one another. For example, pin(s), power/ground connections 760, wireless transceiver 740, and/or wired communications 727, can detect whether watch body 704 and watch band 712 are mechanically and/or electrically or magnetically connected to one another (e.g., detecting a disconnect between the one or more electrical contacts of power/ground connections 760 and/or wired communications 727). In some examples, when watch body 704 and watch band 712 are mechanically and/or electrically disconnected from one another (e.g., watch body 712 has been detached from watch band 712 as described with reference to
[0152] Modifying the level of functionality (e.g., reducing the functionality in watch body 704 and/or watch band 712) can reduce power consumption in battery 728 and/or battery 738. For example, any of the sensors (e.g., heart rate sensor 758, EMG sensor 746, SpO.sub.2 sensor 754, altimeter 748, etc.), processors (e.g., CPU 726, microcontroller unit 752, etc.), communications elements (e.g., NFC 715, GPS 716, LTE 718, WiFi/BluetoothTM 720, etc.), or actuators (e.g., haptics 722, 749, etc.) can reduce functionality and/or power consumption (e.g., enter a sleep mode) when watch body 704 and watch band 712 are mechanically and/or electrically disconnected from one another. Watch body 704 and watch band 712 can return to full functionality when watch body 704 and watch band 712 are mechanically and/or electrically connected to one another. The level of functionality of each of the sensors, processors, actuators, and memory can be independently controlled.
[0153] As described above, wrist-wearable device system 700 can detect when watch body 704 and watch band 712 are coupled to one another (e.g., mechanically connected and/or electrically connected) or decoupled from one another. In some examples, watch body 704 can modify a level of functionality (e.g., activate and/or deactivate certain functions) based on whether watch body 704 is coupled to watch band 712. For example, CPU 726 can execute instructions that detect when watch body 704 and watch band 712 are coupled to one another and activate front-facing image sensor 625A. CPU 726 can activate front-facing image sensor 625A based on receiving user input (e.g., a user touch input from touchscreen display 713, a user voice command from microphone 127, a user gesture recognition input from EMG sensor 746, etc.).
[0154] When CPU 726 detects that watch body 704 and watch band 712 are decoupled from one another, CPU 726 can modify a level of functionality (e.g., activate and/or deactivate additional functions). For example, CPU 726 can detect when watch body 704 and watch band 712 are decoupled from one another and activate rear-facing image sensor 625B. CPU 726 can activate rear-facing image sensor 625B automatically (e.g., without user input) and/or based on receiving user input (e.g., a touch input, a voice input, an intention detection, etc.). Automatically activating rear-facing image sensor 625B can allow a user to take wide-angle images without having to provide user input to activate rear-facing image sensor 625B.
[0155] In some examples, rear-facing image can be activated based on an image capture criterion (e.g., an image quality, an image resolution, etc.). For example, rear-facing image sensor 625B can receive an image (e.g., a test image). CPU 726 and/or rear-facing image sensor 625B can analyze the received test image data and determine whether the test image data satisfies the image capture criterion (e.g., the image quality exceeds a threshold, the image resolution exceeds a threshold, etc.). Rear-facing image sensor 625B can be activated when the test image data satisfies the image capture criterion. Additionally, or alternatively, rear-facing image sensor 625B can be deactivated when the test image data fails to satisfy the image capture criterion.
[0156] In some examples, CPU 726 can detect when watch body 704 is coupled to watch band 1012 and deactivate rear-facing image sensor 625B. CPU 726 can deactivate rear-facing image sensor 625B automatically (e.g., without user input) and/or based on receiving user input (e.g., a touch input, a voice input, an intention detection, etc.). Deactivating rear-facing image sensor 625B can automatically (e.g., without user input) reduce the power consumption of watch body 704 and increase the battery charge time in watch body 704. In some examples, wrist-wearable device system 700 can include a coupling sensor 707 that senses whether watch body 704 is coupled to or decoupled from watch band 712. Coupling sensor 707 can be included in any of watch body 704, watch band 712, or watch band coupling mechanism 660 of
[0157] Various operations described herein can be implemented on computer systems.
[0158] Network interface 820 can provide a connection to a wide area network (e.g., the Internet) to which WAN interface of a remote server system is also connected. Network interface 820 can include a wired interface (e.g., Ethernet) and/or a wireless interface implementing various RF data communication standards such as Wi-Fi, Bluetooth, or cellular data network standards (e.g., 3G, 4G, 5G, 60 GHz, LTE, etc.).
[0159] User input device 822 can include any device (or devices) via which a user can provide signals to computing system 814; computing system 814 can interpret the signals as indicative of particular user requests or information. User input device 822 can include any or all of a keyboard, touch pad, touchscreen, mouse or other pointing device, scroll wheel, click wheel, dial, button, switch, keypad, microphone, sensors (e.g., a motion sensor, an eye tracking sensor, etc.), and so on.
[0160] User output device 824 can include any device via which computing system 814 can provide information to a user. For example, user output device 824 can display images generated by or delivered to computing system 814 using a display. The display can incorporate various image generation technologies, e.g., a liquid crystal display (LCD), light-emitting diode (LED) including organic light-emitting diodes (OLED), projection system, cathode ray tube (CRT), or the like, together with supporting electronics (e.g., digital-to-analog or analog-to-digital converters, signal processors, or the like). A device such as a touchscreen that functions as both an input and output device can be used. Output device 824 can be provided in addition to or instead of a display. Examples include indicator lights, speakers, tactile “display” devices, printers, and so on.
[0161] Some implementations include electronic components, such as microprocessors, storage and memory that store computer program instructions in a computer readable storage medium (e.g., non-transitory computer readable medium). Many of the features described in this specification can be implemented as processes that are specified as a set of program instructions encoded on a computer readable storage medium. When these program instructions are executed by one or more processors, they cause the processors to perform various operations indicated in the program instructions. Examples of program instructions or computer code include machine code, such as is produced by a compiler, and files including higher-level codes that are executed by a computer, an electronic component, or a microprocessor using an interpreter. Through suitable programming, processor 816 can provide various functionalities for computing system 1814, including any of the functionalities described herein as being performed by a server or client, or other functionality associated with message management services.
[0162] It will be appreciated that computing system 814 is illustrative and that variations and modifications are possible. Computer systems used in connection with the present disclosure can have other capabilities not specifically described here. Further, while computing system 814 is described with reference to particular blocks, it is to be understood that these blocks are defined for convenience of description and are not intended to imply a particular physical arrangement of component parts. For instance, different blocks can be located in the same facility, in the same server rack, or on the same motherboard. Further, the blocks need not correspond to physically distinct components. Blocks can be configured to perform various operations, e.g., by programming a processor or providing appropriate control circuitry, and various blocks might or might not be reconfigurable depending on how the initial configuration is obtained. Implementations of the present disclosure can be realized in a variety of apparatus including electronic devices implemented using any combination of circuitry and software.
[0163]
[0164] Operations 1018 and the other operations shown on
[0165] In some embodiments, the first sensor is an inertial measurement unit (IMU) sensor that is configured to detect data indicating movements of the wearable device, and the second sensor is an electromyography (EMG) sensor that is configured to detect neuromuscular signals indicating muscular movements of a user (1018). In some embodiments, the first sensor is a different (i.e., not the same) sensor than the second sensor, and in some other embodiments the first sensor and second sensor are different components of the same sensor.
[0166] In some embodiments, the first sensor is a hall effect sensor (HES) that is configured to be used to check if a display portion of the wearable device is attached or detached to a cradle portion of the wearable device, and the second sensor is photoplethysmography (PPG) sensor that is configured to be used to detect changes in blood flow in a user (1020).
[0167] In some embodiments, the first sensor is a hall effect sensor (HES) that is configured to be used to check if a display portion of the wearable device is attached or detached to a cradle, and the second sensor is an inertial measurement unit (IMU) sensor that is configured to detect data indicating movements of the wearable device (1022).
[0168] In some embodiments, the first sensor has a first power consumption rate between 2 to 5 mW/s (milliwatts per second) (1024).
[0169] In some embodiments, the second sensor has a second power consumption rate between 7 to 17 mW/s (1026).
[0170] In some embodiments, the component associated with the second sensor is operated in the active state for a period of time, and, after the period of time, the method further comprises (1028): while the component associated with the second sensor is in the inactive state: receiving, from the first sensor, new first sensor data; and the wearable device determines whether the new first sensor data indicates movement of the wearable device. In some embodiments, the method includes in accordance with a determination that the first sensor data indicates that the wearable device has not moved, continuing to operate the component of the second sensor in the inactive state (e.g.,
[0171] In some embodiments, the continuing to operate the component associated with the second sensor in the active state includes continuing (1030) to operate the component associated with the second sensor in the active state until a deactivation trigger is detected.
[0172] In some embodiments, the deactivation trigger is detected when data from the first sensor that indicates that the wearable device has been removed from the user’s body (1032) (e.g.,
[0173] In some embodiments, the deactivation trigger is detected when data from the second sensor indicates that the wearable device has been remove from the user’s body (1034) (e.g.,
[0174] In some embodiments, the determination that the second sensor data indicates (1036) that the wearable device has been placed on the user’s body includes determining that the second sensor data, as compared to subsequent sensor data from the second sensor, reflects a change in data sensed by the second sensor.
[0175] In some embodiments, the second sensor data indicates (1038) that one or more sensing channels of the second sensor are receiving data at or above a noise threshold (e.g., noise can include one or more of: inherent noise in the electronics equipment, ambient noise, motion artifact, inherent instability of signal, ECG artifacts, cross talk, electrode contact, transducer noise, or baseline shifts) (In some embodiments, receiving data above a noise threshold includes identifying a pattern identified as not noise for a predetermined amount of time), and the subsequent sensor data indicates that the one or more sensing channels of the second sensor are receiving data below the noise threshold (e.g., one or more sensing channels reflect stable signals (while device is on wrist) as compared to noisy signals (while device is moving and being attached to wrist)). For example,
[0176] In some embodiments, operating (1040) the component of the second sensor in the active state includes causing an interrupt signal to be sent to the component associated with the second sensor to cause it to transition from the inactive to the active state.
[0177] In some embodiments, determining (1042) whether the first sensor data indicates movement of the wearable device includes determining whether the movement is consistent with one or more known movements associated with the user placing the wearable device on the user’s body. For example, the description associated with
[0178] In some embodiments, the component associated with the second sensor is a system-on-a-chip configured to process the second sensor data (1044).
[0179] In some embodiments, the component associated with the second sensor is a machine-learning model used to process and/or analyze the second sensor data (1046) (e.g., the state determination logic 110 in
[0180] In some embodiments, the component is an electrode configured to sense the second sensor data (1048). In some embodiments, the component includes a channel of at least two electrodes configured to sense the second sensor data (1050). For example,
[0181] In some embodiments, the component includes both at least one electrode configured to sense the second sensor data and a machine-learning model configured to process the second sensor data (1052). For example, this is similar to the different power-consumption levels for different types of machine-learning models, the underlying components used to sense the signals themselves (as compared to processing the signals after they are sensed) can also be activated at appropriate times to make efficient use of limited power and computing resources. These include individual electrodes, as well as groups of multiple electrodes (e.g., pairs) operating as sensor channels), such that individual sensors or groups of sensors can be activated at different points in time in accordance with the movement determinations made using the first sensor data. For example, the system can operate with a predetermined number of groups of sensors (e.g., 4 or 6) in an always on/active state and can determine when to activate a remaining number of groups of sensors (e.g., a remining 2-10 additional groups) for assisting with detection of a more complex set of gestures.
[0182] In some embodiments, the wearable device is a wrist-wearable device or a head-worn wearable device (1054).
[0183] In some embodiments, while the wearable device has been placed on the user’s body: in accordance with a determination that a display portion of the wearable device has been detached from a cradle, locking (1056) the wearable device until an authentication input is received (e.g., a waving of a finger detected by the rear-facing camera, as shown in
[0184]
[0185] The method includes determining (1104) (e.g., at the capsule portion of the wrist-wearable device or a separate third connected device (e.g., a phone)), using sensor data from the first sensor of the first sensor type, whether the capsule portion of the wrist-wearable device is coupled with the cradle portion of the wrist-wearable device (e.g.,
[0186] The method includes, in accordance with a determination that the data from the first sensor of the first sensor type indicates that the capsule portion of the wrist-wearable device is coupled with the cradle portion of the wrist-wearable device, (activating a second sensor from a powered-off state, and then) obtaining (1106) data from the second sensor of the second sensor type (e.g.,
[0187] The method includes, in accordance with a determination that the data from the second sensor of the second sensor type indicates that the wrist-wearable device is worn on a wrist of a user, (activating a third sensor from a powered-off state, and then) monitoring (1108) data from a third sensor of a third sensor type that is distinct from the first and second sensor types to determine an activity being performed by the user while the wrist-wearable device is worn on the user’s wrist (e.g.,
[0188] In some embodiments, the first sensor of the first type is integrated into the capsule portion of the wrist-wearable device. In some embodiments, the first sensor of the first type is integrated into the cradle portion (including a watchband of the cradle portion) of the wrist-wearable device. In some embodiments, the second sensor of the second type is integrated into the capsule portion of the wrist-wearable device. In some embodiments, the second sensor of the second type is integrated into the cradle portion (including a watchband of the cradle portion) of the wrist-wearable device. In some embodiments, the third sensor of the third type is integrated into the capsule portion of the wrist-wearable device. In some embodiments, the third sensor of the third type is integrated into the cradle portion (including a watchband of the cradle portion) of the wrist-wearable device.
[0189] Operations 1110 and the other operations shown on
[0190] In some embodiments, the first sensor is a hall effect sensor (HES) that provides data that is used to check if the capsule portion of the wrist-wearable device is coupled or decoupled from the cradle portion of the wrist-wearable device, (e.g., the HES is used to sense the presence of the magnetic field of the cradle portion of the wearable device) and the second sensor is (i) photoplethysmography (PPG) sensor that provides data that is used to detect changes in blood flow in a user, (ii) electromyography (EMG) sensor that provides data that is used to determine muscular activities of a user, or (iii) an infrared (IR) sensor that provides data used to detect proximity to objects (1110).
[0191] In some embodiments, the method includes, after determining whether the capsule portion of the wrist-wearable device is on the wrist of the user, determining (1112) using a component of a third sensor whether the wrist-wearable device is being used during sleep or during exercise (e.g.,
[0192] In some embodiments, the third sensor is an inertial measurement unit (IMU) sensor that is configured to provide data used to approximate or measure inertial parameters associated with motion of the wrist-wearable devices (1114) (e.g.,
[0193] In some embodiments, in accordance with a determination that the third sensor indicates that the wrist-wearable device is being used during exercise, operating (1116) the second sensor at a first frequency (e.g.,
[0194] In some embodiments, the first frequency is between 15 Hz and 50 Hz (e.g.,
[0195] In some embodiments, the method includes, in accordance with a determination that the third sensor indicates that the wrist-wearable device is being used during sleep, activating (1120) a first subset of components of the second sensor (e.g., as shown in step 318 of
[0196] In some embodiments, the first subset of components of the second sensor includes an IR sensor, a red light emitting diode (LED), and a green LED (e.g., as shown in step 318 of
[0197] In some embodiments, the wrist-wearable device includes at least two cameras (e.g., see
[0198] In some embodiments, the at least two cameras include a front facing camera and a rear facing camera (e.g., the front facing camera is on the same face as the display, and the rear-facing camera is on the opposite surface (1126) (e.g., the surface that mates to the cradle)).
[0199] In some embodiments, the method includes, in accordance with a determination that the data from the first sensor of the first sensor type indicates that the capsule portion of the wrist-wearable device is not coupled with the cradle portion of the wrist-wearable device, enabling (1128) the rear facing camera (e.g., step 304 of
[0200] Various examples discussed above related to use of data from a hall-effect sensor to make determinations as to whether or not a capsule is attached to a cradle portion on a band of a wearable device. As an alternative, or in addition, capacitively-sensed information about a user’s digits relative to the capsule can also be used to assist with the cradle on/off determinations. In one example, a metallic data-transmission component (e.g., an NFC coil) can be repurposed to be used for this capacitive-sensing application. A skilled artisan would also appreciate that repurposing of the metallic data-transmission component can be used as a capacitive sensor independent of whether or not it is for the cradle on/off example application. More details are provided below. In some embodiments, the camera is on the opposite surface (1126) (e.g., the surface that mates to the cradle)).
[0201]
[0202] The method 1200 includes, receiving (1204), via the metallic data-transmission component, operational data indicative of a current operational state of the metallic data-transmission component. The method 1200 includes determining (1206), based on the operational data, whether data-transmission criteria are present.
[0203] The method 1200 also includes, in accordance with a determination that the data-transmission criteria are present, the electronic device operates (1208) the metallic data-transmission component in a first mode, wherein, while in the first mode, the metallic data-transmission component is configured to be used as antenna in conjunction with transferring data between the electronic device and at least one other electronic device using the metallic data-transmission component. In some embodiments, this first mode is a mode in which the metallic data-transmission component performs a first function and, as discussed below, when the metallic data-transmission component is operated in a second mode, it then performs a second function that is distinct from the first function.
[0204] The method 1200 includes that in accordance with a determination that the data-transmission criteria are not present, operating (1210) the metallic data-transmission component in a second mode, in which the metallic data-transmission component functions as a capacitive sensor.
[0205] Operations 1212 and the other operations shown on
[0206] In some embodiments, the electronic device includes one or more sensors. In some embodiments, the method includes receiving (1212), via the one or more sensors, sensor data indicating a position of the metallic data-transmission component with respect to the electronic device; and determining whether data-transmission criteria are present based in part the sensor data.
[0207] In some embodiments, the one or more sensors include another capacitive sensor. In some embodiments, the method includes, while the metallic data-transmission component is operating in the second mode, receiving (1214), via the capacitive sensor, a first capacitance value. In some embodiments, the method includes, while the metallic data-transmission component is operating in the second mode, receiving, via the other capacitive sensor a second capacitance value. In some embodiments, the method includes, while the metallic data-transmission component is operating in the second mode, determining a true capacitance value based on a comparison of the first capacitance value and the second capacitance value. In some embodiments, the method includes, while the metallic data-transmission component is operating in the second mode, providing the true capacitance value to the electronic device to perform an action based on the true capacitance value (e.g., a touch screen input).
[0208] In some embodiments, the one or more sensors include another capacitive sensor that is disabled when the metallic data-transmission component is operating in a second mode (1216).
[0209] In some embodiments, the metallic data-transmission component is coupled to a switch. In some embodiments, the method includes, providing (1218) a switching signal to the switch to selectively couple the metallic data-transmission component with (i) data-communication circuitry while the metallic data-transmission component is operating in the first mode and (ii) sensor-processing circuitry while the metallic data-transmission component is operating in the second mode based on a determination whether the data-transmission criteria are present.
[0210] In some embodiments, capacitance detected via the metallic data-transmission component while it is operating in the second mode is used in conjunction with the methods of A1-A23 or G1-G11 (1220).
[0211] In some embodiments, the electronic device is a wrist-wearable device including a capsule and an accessory configured to couple with the capsule, and determining (1222) whether data-transmission criteria are present includes determining that the capsule is coupled to the accessory.
[0212] In some embodiments, the electronic device includes a communications component configured to communicatively coupled with at least one other device. In some embodiments, the communications components provides a beacon signal. In some embodiments, determining (1224) whether data-transmission criteria are present is further based on one or more communication signals transferred between the electronic device and the at least one other device. In some embodiments, a first signal transmitted by the metallic data-transmission component has a first wavelength and a second signal transmitted by the communications component has a second wavelength distinct from the first wavelength.
[0213] In some embodiments, the data-transmission criteria includes one or more of a predefined idle time, predefined separation distance, predefined capacitance value, and predefined number of coupling attempts (1226).
[0214] In some embodiments, the predefined separation distance is equal to or less than 5 millimeters (mm) (1230). In some embodiments, the metallic data-transmission component can detect capacitance up to a distance of at least 1 centimeter (cm), 2 cm, 3 cm.
[0215] In some embodiments, predefined capacitance value is within a range of values between 100-200 pF (1232).
[0216] In some embodiments, the predefined idle time is a nonzero value that is equal to or less than 5 ms seconds (1234).
[0217] In some embodiments, the predefined number of coupling attempts is a nonzero value that is equal to or less than three coupling attempts (1236).
[0218] In some embodiments, a wireless communications component of the wrist-wearable device (e.g., a BLUETOOTH radio) operates at a first frequency and the metallic data-transmission component operates at a second frequency lower than the first frequency (1238). In some embodiments, the metallic data-transmission component has an operations frequency of 13.5 MHz. In some embodiments, due to the frequency differences there is no interferences.
[0219] In some embodiments, the method includes, while operating the metallic data-transmission component in the second mode, receiving capacitance values at a first point and time and a second point in time, where a time between the first point in time and the second point in time is less than 300 ms (1240). In some embodiments, the method includes, receiving capacitance values in less than 200 ms, and in some other embodiments less than 150 ms. In some embodiments, pre-heating of a camera with SMA actuators takes about 300 to 500 ms, the metallic data-transmission component provides capacitance values quickly reducing the lag time between decoupling and activating the camera.
[0220]
[0221]
[0222]
[0223]
[0224] It will be understood that, although the terms “first,” “second,” etc. may be used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another.
[0225] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the claims. As used in the description of the embodiments and the appended claims, the singular forms “a,” “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that the term “and/or” as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items. It will be further understood that the terms “comprises” and/or “comprising,” when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
[0226] As used herein, the term “if” can be construed to mean “when” or “upon” or “in response to determining” or “in accordance with a determination” or “in response to detecting,” that a stated condition precedent is true, depending on the context. Similarly, the phrase “if it is determined [that a stated condition precedent is true]” or “if [a stated condition precedent is true]” or “when [a stated condition precedent is true]” can be construed to mean “upon determining” or “in response to determining” or “in accordance with a determination” or “upon detecting” or “in response to detecting” that the stated condition precedent is true, depending on the context.
[0227] The foregoing description, for purpose of explanation, has been described with reference to specific embodiments. However, the illustrative discussions above are not intended to be exhaustive or to limit the claims to the precise forms disclosed. Many modifications and variations are possible in view of the above teachings. The embodiments were chosen and described to best explain principles of operation and practical applications, to thereby enable others skilled in the art.