A SYSTEM COMPRISING A WEARABLE OR HAND-HELD DEVICE, AND A METHOD OF SPATIALLY STABILIZING DISPLAY INFORMATION UPON OPERATION OF SUCH DEVICE
20230396745 · 2023-12-07
Inventors
- Amir RMAILE (Eindhoven, NL)
- Lutz Christian Gerhardt (Eindhoven, NL)
- Bart GOTTENBOS (Budel, NL)
- Mark Thomas Johnson (Arendonk, BE)
Cpc classification
A61C17/228
HUMAN NECESSITIES
H04N9/3173
ELECTRICITY
International classification
Abstract
A system is provided. The system comprises an image projection device (6) configured to project display information; a wearable or hand-held device (100) which when worn is subjected to spatial movement; and a compensation element (7) configured to adjust the image projection device (6) based on the spatial movement such that the movement of the wearable or hand-held device (100) is compensated for during projection of the display information by the image projection device (6). Further, a method of spatially stabilizing display information projected by an image projection device (6) upon operation of a wearable or hand-held device is provided.
Claims
1. A system comprising: an image projection device configured to project display information; a wearable or hand-held device which when worn is subjected to spatial movement; and a compensation element configured to adjust the image projection device based on the spatial movement such that the movement of the wearable or hand-held device is compensated for during projection of the display information by the image projection device, wherein the spatial movement comprises a first movement caused by a drivetrain forming part of the wearable or hand-held device, and a second movement comprising manual movement of the wearable or hand-held device by a user of the wearable or hand-held device during operation thereof.
2. The system according to claim 1 further comprising a sensor configured to determine spatial movement of the wearable or hand-held device.
3. The system according to claim 2, wherein the sensor is a motion sensor comprising one or more sensor chosen from the group consisting of an accelerometer, a gyroscope and a rotation sensor.
4. The system according to claim 1, wherein the compensation element is configured to modulate the image projection device based on a vibration frequency of the wearable or hand-held device corresponding to said first movement to thereby spatially stabilize the projected display information.
5. The system according to claim 1, wherein the compensation element is configured to modulate the image projection device by mechanically moving at least a part of the image projection device by digitally adapting pixels of the image projection device.
6. The system according to claim 1, wherein the compensation element is configured to modulate the image projection device by synchronizing a frame rate of the image projection device with a vibration frequency of the wearable or hand-held device corresponding to said first movement or with a harmonic of said vibration frequency of the wearable or hand-held device.
7. The system according to claim 1, wherein the compensating element is configured to modulate the image projection device with a duty cycle of less than 50% of a total operation time.
8. The system according to claim 1, wherein the image projection device comprises one or more device chosen from the group consisting of an image projector, a laser scanning device, a MEMS based projection device, a laser diode and a LED.
9. The system according to claim 1, wherein the system further comprises a holographic 3D display and wherein the image projection device is configured to project the display information onto the holographic 3D display.
10. A method of spatially stabilizing display information projected by an image projection device upon operation of a wearable or hand-held device, comprising: determining spatial movement of the wearable or hand-held device; and providing an adjustment signal for adjusting the image projection device based on the determined spatial movement such that the display information projected by the image projection device is spatially stabilized, and compensating for a first movement caused by a drivetrain forming part of the wearable or hand-held device, and second movement comprising manual movement of the wearable or hand-held device by a user of the wearable or hand-held device during operation thereof.
11. The method of claim 10, wherein the determining of spatial movement of the wearable or hand-held device comprises determining a vibration frequency of the wearable or hand-held device corresponding to said first movement, and wherein the adjusting of the image projection device comprises synchronizing a frame rate of the image projection device with the vibration frequency of the wearable or hand-held device or with a harmonic of the vibration frequency of the wearable or hand-held device.
12. The method of claim 10, wherein the adjustment signal is for adjusting the image projection device by a compensation element and wherein the compensating element is configured to modulate the image projection device with a duty cycle of less than 50% of a total operation time.
13. The method of claim 10, wherein the determining of spatial movement of the wearable or hand-held device comprises determining a vibration frequency of the wearable or hand-held device corresponding to said first movement, and wherein the image projection device is adjusted by modulating the image projection device based on the determined vibration frequency.
14. The method of claim 13, wherein the adjustment signal is for adjusting the image projection device by a compensation element configured to mechanically move at least a part of the image projection device.
15. A non-transitory computer readable recording medium having computer readable program code recorded thereon which when executed on a device having processing capability is configured to perform a method including the steps of: determining spatial movement of a wearable or hand-held device; providing an adjustment signal for adjusting an image projection device based on the determined spatial movement such that the display information projected by the image projection device is spatially stabilized, and compensating for a first movement caused by a drivetrain forming part of the wearable or hand-held device, and a second movement comprising manual movement of the wearable or hand-held device by a user of the wearable or hand-held device during operation thereof.
16. The system according to claim 2, wherein the sensor is a motion sensor comprising one or more sensor chosen from the group consisting of an accelerometer, a gyroscope and a rotation sensor.
17. The method of claim 13, wherein the adjustment signal is for adjusting the image projection device by a compensation element configured to digitally adapt pixels of the image projection device.
18. The method of claim 13, wherein the adjustment signal is for adjusting the image projection device by a compensation element configured to synchronize a frame rate of the image projection device with the vibration frequency of the wearable or hand-held device.
19. The method of claim 13, wherein the adjustment signal is for adjusting the image projection device by a compensation element configured to synchronize a frame rate of the image projection device with a harmonic of the vibration frequency of the wearable or hand-held device.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
[0040] These and other aspects of the present invention will now be described in more detail, with reference to the appended drawings showing embodiment(s) of the invention.
[0041]
[0042]
[0043]
[0044]
[0045]
[0046]
[0047]
[0048]
[0049]
DETAILED DESCRIPTION
[0050]
[0051] Devices which are wearable or hand held will be described generally as wearable devices in the following description. The invention is especially suitable for wearable devices used for personal care. Non-limiting examples of personal care devices are oral care devices in the form of brushing mouthpieces, tooth brushes and shaving and grooming devices. The skilled person will also understand that the invention is equally applicable to other wearable devices and especially devices to be worn close to the body, such as gaming equipment.
[0052] The mouthpiece 1 which is disclosed highly schematically comprises a first section 2 which during use of the mouthpiece 1 is configured to be inserted into the mouth of a user and a second section 3 which during use is configured to project away from the mouth. In the disclosed embodiment, the first section 2 is embodied as a bite splint having a curvature of the row of teeth. The bite splint comprises bristles 13 or rubber-based cleaning elements which are configured to be moved by a drivetrain 15 to be discussed below. The first and second sections 2, 3 may be arranged as a unitary body or be arranged as detachable bodies.
[0053] The mouthpiece 1 is provided with an information system 4 which comprises an optional sensor 5 which is configured to determine spatial movement of the wearable device 100; an image projection device 6 which is configured to project display information; and a compensation element 7 which is configured to adjust the image projection device 6 based on the spatial movement determined by the sensor 5 such that the movement of the wearable device 100 is compensated for during projection of the display information by the image projection device 6. The information system 4 further comprises a control unit 8.
[0054] The skilled person realizes that the sensor 5 may be omitted in the event the wearable device 100 has a known frequency and hence a known spatial movement.
[0055] The spatial movement of the mouthpiece 1 may be caused by a drivetrain 15 comprising one or more moving parts like a motor and a transmission. The drivetrain 15 causes a vibrational movement having a frequency and an amplitude. Such vibrational movement will in the following be referred to as a “periodical movement”. However, this is one example of a general first movement which is caused by a drivetrain of the device
[0056] Alternatively, the spatial movement may be caused by manual movement of the wearable device 100 during its operation. A manual movement may be caused e.g. by an angular or twisting movement, e.g. by the user twisting her wrist during teeth brushing, or by the user shaking/moving her head while treating the teeth/gums with the mouthpiece 1. The manual movement causes what will be referred to below as a “non periodical movement”.
[0057] The sensor 5 may be a motion sensor comprising one or more of a sensor from the group consisting of an accelerometer, a gyroscope and a rotation sensor. The type of sensor 5 to be used may be determined based on what type of movement to sense, thus if it is a periodical or a non-periodical movement. The skilled person will understand that one and the same sensor may be used to determine both a periodical and a non-periodical movement.
[0058] It is to be understood that one and the same wearable device 100 may comprise one or more sensors 5 of the same type or a combination of sensors 5 of different types. The sensors 5 may be arranged at different positions of the wearable device 100.
[0059] If the movement is a periodical movement, the sensor 5 may by way of example be an accelerometer or a rotation sensor. As given above, one typical example of a periodical movement is a vibration which has a frequency and an amplitude. Thus, in the event of periodical movement, the sensor used should be configured to determine a vibration frequency of the wearable device 100.
[0060] If the movement is a non-periodical movement, the sensor 5 may by way of example be a gyroscope or a rotation sensor. As given above, one typical example of a non-periodical movement is the movement of the user's body, such as a head movement or a twisting of the hand holding the wearable device 100.
[0061] In yet another embodiment, the sensor 5 may be an optical sensor. The optical sensor may be used to, based on a camera image of a displayed information, provide a real-time feedback to keep the projected display information stationary. This may by way of example be made by monitoring markers at for example a corner or at an edge of the displayed information.
[0062] As given above, in some embodiments, where the wearable device 100 has a known frequency during operation, the skilled person realizes that a sensor may be omitted. Instead the device may be configured so that the compensation element 7 is operated at said known frequency.
[0063] Now turning to the image projection device 6. The image projection device 6 may be configured to project display information on any surface, no matter if it is a surface external from the wearable device 100, such as a wall, a ceiling or a holographic 3D display, or a surface forming part of the wearable device 100, such as a display or a holographic 3D display supported by the wearable device. In the event the surface forms part of the wearable device, the display information from the image projection device may be reflected to the surface on the wearable device via a reflecting surface such as a bathroom mirror.
[0064] The projected display information may be strictly informative, e.g. a pointer indicating areas with certain needs or a timer information. The projected display information may also be of entertaining character, e.g. a movie or a game. It goes without saying that the display information may be a combination of informative and entertaining character. The projected display information may be a real-time feedback to the user during operation of the wearable device 100, a streamed information or information stored in a memory.
[0065] The image projection device 6 may by way of example be an image projector, such as an LCD, a laser scanning device, a MEMS based projection device, a laser diode or a LED. The laser diode may be provided with a scanner, i.e. a mirror. The system may comprise a single image projection device 6 or a combination of several image projection devices 6 of the same type or of different types.
[0066] The image projection device 6 may as is disclosed in
[0067] As given above, the system comprises a compensation element 7 which is configured to adjust the image projection device 6 based on the spatial movement determined by the sensor 5 such that the movement of the wearable device 100 is compensated for during projection of the display information. The adjustment may be made in a number of ways depending on the type of spatial movement and also the type of image projection device 6 used.
[0068] The system further comprises a control unit 8. The control unit 8 is operatively connected to at least the sensor 5, the image projection device 6 and the compensation element 7. The control unit 8 is configured to receive information from the sensor 5 and evaluate if there, based on the information received from the sensor 5, is a need for an adjustment of the image projection device 6 or not. The evaluation is made in view of determined spatial movement as compared to a set acceptable spatial movement. If it is determined that there is a need for compensation, the control unit 8 is configured to instruct the compensation element 7 accordingly to adjust the image projection device 6 based on the spatial movement determined by the sensor 5 such that the spatial movement of the wearable device 100 is compensated for during projection of the display information by the image projection device 6. The skilled person understands that in the event the sensor 5 is omitted and the spatial movement is known beforehand, the control unit 8 is instead configured to instruct the compensation element 7 accordingly to adjust the image projection device 6 based on the known spatial movement.
[0069] The communication between the control unit 8, the sensor 5, the image projection device 6 and the compensation element 7 may be wired or wireless or even a combination thereof. In the event of a wireless communication, the communication may be made via Wi-Fi or Bluetooth.
[0070] The control unit 8 is accordingly configured to carry out overall control of functions and operations of the system to thereby allow the adjustment of the image projection device 6. The control unit 8 may include a processor, such as a central processing unit (CPU), a microcontroller, or a microprocessor. The processor is configured to execute program code stored in an internal storage device. The control unit 8 may form part of a non-disclosed circuitry supported by a non-disclosed printed circuitry board, which circuitry further comprises an internal data storage device and a power source. The internal data storage device may be one or more of a buffer, a flash memory, a hard drive, a removable media, a volatile memory, a non-volatile memory, a random access memory (RAM), or another suitable device. In a typical arrangement, the internal data storage device may include a non-volatile memory for long term data storage and a volatile memory that functions as system memory for the control unit 8.
[0071] In the following, a number of different embodiments will be described of how the invention may be put to practice.
[0072] Now turning to
[0073] Now turning to
[0074] In a first embodiment, see
[0075] In another embodiment, see
[0076] In yet another non-illustrated embodiment, the compensation may be made by synchronising a frame rate of the image projection device 6 with the frequency associated with the determined spatial motion. This is made by the control unit 8 being configured to modulate the image projection device 6 by synchronizing the frame rate by the determined frequency. The information will be displayed with the same phase as the determined frequency. Thereby the blurring of the image displayed by the image projection device 6 may be substantially eliminated. The modulation is hence made digitally. This may be useful in the event the image projection device 6 is a projector. Provided the vibration frequency is pre-known, there is no need for any separate sensor.
[0077] For image projection devices 6 having a faster projection frequency, such as a laser diode and scanner, the control unit 8, acting as a compensation element 7, may be configured to set the laser diode and scanner to project an image with exactly the same frame rate, i.e. the frequency of the brush head 12, e.g. 200 Hz. The modulation is made digitally also in this embodiment. Provided the vibration frequency is pre-known, there is no need for any separate sensor.
[0078] In still another non-illustrated embodiment, suitable for image projection devices 6 having a less rapid projection frequency, such as cheap LCDs, the display information may be displayed at a fraction of the frequency of the brush head 12. By way of example, the information may be displayed every second brushing strokes corresponding to a frequency of 100 Hz in the event of the brush head 12 having a frequency of 200 Hz. Like in the previous embodiments, the control unit 8 acts as the compensating element 7 being configured to adjust the image projection device 6 by modulating the projection frequency. Provided the vibration frequency is pre-known, there is no need for any separate sensor.
[0079] In yet another embodiment, see
[0080] In still another non-illustrated embodiment, the sensor 5 may be an optical sensor. The optical sensor may be used to, based on a camera image of a displayed information, provide a real-time feedback to the control unit 8 to thereby allow the same to modulate the image projection device 6 and hence keep the projected display information stationary. The optical sensor 5 may be set to monitor movement relative to markers at a corner or at an edge of the displayed information. The control unit 8 will hence act as a compensating element 7 and is configured to receive and prosecute information relating to spatial movement of the wearable device.
[0081] Now turning to
[0082] The holographic 3D display 14 may by way of example be used to provide real-time feedback to the user. By way of example, areas where the quality, such as the brushing quality, could be improved may be visualized to the user.
[0083] In a non-disclosed alternative to the embodiment of
[0084] Now turning to
[0085] Now turning to
[0086] Now turning to
[0087] Now turning to
[0088] Determining 1000 spatial movement of the wearable device 100.
[0089] The spatial movement of the wearable device may in one embodiment be determined by determining 1100 a vibration frequency of the wearable device 100. The vibration frequency may either be pre-known and thus retracted from an internal storage device or be determined by one or more sensors 5.
[0090] Adjusting 2000 the image projection device 6 based on the determined spatial movement such that the display information projected by the image projection device 6 is spatially stabilized. As given above in the context of different exemplified embodiments of the device, the adjustment many be accomplished in a number of ways.
[0091] Based on this information, the image projection device 6 may be adjusted by synchronizing 2100A a frame rate of the image projection device 6 with the vibration frequency of the wearable device 100 or with a harmonic of the vibration frequency of the wearable device 100. The synchronization may be made by digitally causing a phase shift.
[0092] The image projection device may be adjusted by a compensation element 7 which is configured to modulate the image projection device 6 with a duty cycle of less than 50%, preferably less than 20% and more preferably less than 10% of a total operation time.
[0093] In another embodiment, the determining 1000 of spatial movement of the wearable device 100 may comprise determining a vibration frequency of the wearable device 100, and wherein the image projection device 6 is adjusted by modulating 2100B the image projection device 6 based on the determined vibration frequency of the wearable device 100.
[0094] In yet another embodiment, the image projection device 6 may be adjusted by a compensation element 7 which is configured to mechanically 2100C modulate the image projection device or part thereof by moving the same; or to digitally adapt pixels 2100D of the image projection device.
[0095] According to yet another aspect, a non-transitory computer readable recording medium is provided. The non-transitory computer readable recording medium has a computer readable program code recorded thereon which when executed on a device having processing capability is configured to perform the method as presented above.
[0096] The person skilled in the art realizes that the present invention by no means is limited to the preferred embodiments described above. On the contrary, many modifications and variations are possible within the scope of the appended claims.
[0097] The image projection device may be any type of image projection device configured to project information. One and the same wearable device may be provided with image projection devices of different types. In one non-limiting example, the device may be provided with one projector and one or more laser diodes with scanners. Different types of information may be displayed separately or simultaneously. The two or more image projection devices may be arranged in different positions on the device and under different angles.
[0098] In the event the image projection device is provided as one or more laser diodes, these diodes may be combined with movable mirrors—earlier referred to as scanners—thereby allowing displaying variable patterns constituting informative content or entertaining content. The movable mirrors may be part of a MEMS such as a digital micromirror device (DMD). One such DMD provided by Texas Instruments comprises an array of highly reflective micromirrors. The DMD may be seen as a spatial light modulator.
[0099] The wearable device may be provided with a supplementary non-disclosed passive vibration damper, such as one or more springs. Thus, while the compensation element provides for an active compensation, the spring(s) may provide a passive dampening.
[0100] The obvious target for the displayed information may be a wall or ceiling. However, many people often brush their teeth in front to a mirror. Projecting on a mirror is not feasible, but in the mirror people can look at their face, which may enable highly intuitive feedback by projecting for example brushing information on the cheeks, e.g. a red laser pointing to areas which did not receive sufficient cleaning and a green laser showing well cleaned areas. The pointing, i.e. the projection may be directly from the handle and may be combined with a face recognition sensor to point to the correct positions.