System for capturing electrooculography signals
11992324 ยท 2024-05-28
Assignee
Inventors
- Thomas Lunner (Sm?rum, DK)
- Martin Skoglund (Sm?rum, DK)
- Alejandro Lopez Valdes (Sm?rum, DK)
- Tanveer Bhuiyan (Sm?rum, DK)
Cpc classification
H04R25/40
ELECTRICITY
A61B5/6803
HUMAN NECESSITIES
A61B5/398
HUMAN NECESSITIES
International classification
A61B5/398
HUMAN NECESSITIES
A61B5/00
HUMAN NECESSITIES
Abstract
An electrooculography (EOG) signal capture system comprises a) a sensor array adapted for being located at one of a left or right ears of a user and/or for fully or partially being implanted in the head at left or right ears of a user, the sensor array comprising a number Ns of electric potential sensors, respectively, for sensing respective electric potentials from the user's head, where Ns is larger than or equal to two, b) electronic circuitry coupled to the sensor array and configured to provide at least two different beamformed signals (S.sub.BF1, S.sub.BF2), each being representative of a weighted combination of said electric potentials, or of signals derived therefrom, and wherein a difference between said at least two different beamformed signals (?P.sub.BF=S.sub.BF1?S.sub.BF2)at least in a specific electrooculography mode of operationrepresents an electrooculography signal (?P.sub.EOG) from one or both eyes of said user. A single-sensor EOG-system is further proposed. The invention may e.g. be used in hearing aids or other electronic wearables.
Claims
1. A portable electrooculography (EOG) signal capture system comprising: a sensor array adapted for being located at one of a left or right ear of a user, the sensor array comprising a number Ns of electric potential sensors, respectively, for sensing respective electric potentials from the user's head, where Ns is larger than or equal to two; and electronic circuitry coupled to the sensor array and configured to provide at least two different combined signals (S.sub.BF1, S.sub.BF2), each being representative of a weighted combination of said electric potentials, or of signals derived therefrom, and wherein a difference between said at least two different combined signals (?P.sub.BF=S.sub.BF1?S.sub.BF2) represents an electrooculography signal (?P.sub.EOG) from one or both eyes of said user, said electrooculography signal (?P.sub.EOG) representing a current eye gaze used to estimate a current eye gaze angle ?? of the user, wherein said combined signals (S.sub.BF1, S.sub.BF2) are determined from said electric potentials from the user's head by applying beamformer weights representative of first and second beam patterns (BP1, BP2), respectively, which implement beam patterns that do not attenuate signals from respective first and second target directions of an eye ball of the user, wherein functionality of the portable EOG signal capture system is controlled via the current eye gaze angle ??.
2. A portable electrooculography signal capture system according to claim 1 wherein said electronic circuitry receives a reference potential from a reference sensor.
3. A portable electrooculography signal capture system according to claim 2 comprising respective differential buffers to provide respective voltage difference signals from said electric potentials and said reference potential.
4. A portable electrooculography signal capture system according to claim 3 wherein said electronic circuitry is coupled to the sensor array via said differential buffers to thereby provide that said at least two different combined signals (S.sub.BF1, S.sub.BF2) are formed as a weighted combination of said voltage difference signals.
5. A portable electrooculography signal capture system according to claim 1 wherein said beamformed signals (S.sub.BF1, S.sub.BF2) are formed by respective fixed beamformers.
6. A portable electrooculography signal capture system according to claim 1 comprising a processor for executing an algorithm, or accessing stored data, providing corresponding values of eye gaze angle and said electrooculography signal (?P.sub.EOG).
7. A portable electrooculography signal capture system according to claim 1 wherein the electric circuitry is configured to provide that two or more beamformers are adaptive and configured to maximize a difference between the combined signals.
8. A portable electrooculography signal capture system according to claim 1 wherein the electric circuitry is configured to scan a certain part of a skull of the user by adjusting delays and weights used to provide the combined signals.
9. A hearing device adapted for being located at or in an ear of a user, and comprising a portable electrooculography signal capture system according to claim 1.
10. A hearing device according to claim 9 comprising a forward audio signal path comprising an input unit for providing an electric input signal representative of sound, a configurable signal processor for executing a number of processing algorithms to provide a processed electric signal based on said electric input signal, and an output unit for providing stimuli perceivable as sound based on said processed electric signal, wherein said configurable signal processor is adapted to control one or more of said processing algorithms in dependence of said electrooculography signal.
11. A hearing device according to claim 10 wherein said configurable signal processor is adapted to control one or more of said processing algorithms in dependence of said electrooculography signal as well as a brain wave or EEG signals.
12. A hearing device according to claim 11 comprising a number of detectors configured to provide status signals relating to a current physical environment of the hearing device, and/or to a current state of the user wearing the hearing device, and/or to a current state or mode of operation of the hearing device, and wherein said configurable signal processor is adapted to control one or more of said processing algorithms in dependence of said electrooculography signal as well as said status signals.
13. A hearing device according to claim 9 comprising: an in-the-ear (ITE) part adapted for being located at or in an ear canal of the user wherein said ITE part comprises at least a part of said portable electrooculography signal capture system, and/or a behind-the-ear (BTE) part adapted for being located at or behind an ear of the user wherein said BTE part comprises at least a part of said electrooculography signal capture system.
14. A hearing device according to claim 9 additionally comprising one or more sensors for picking up potentials from a brain of the user.
15. A hearing device according to claim 9 comprising a hearing aid, a headset, an earphone, an ear protection device or a combination thereof.
16. A hearing system comprising left and right hearing devices according to claim 9.
17. A non-transitory computer readable medium storing an application comprising executable instructions configured to be executed on an auxiliary device to implement a user interface for a hearing device according to claim 9 and wherein the user interface is configured to allow a user to select functions of the hearing device for eye gaze control.
18. A non-transitory computer readable medium according to claim 17 wherein said functions include one or more of selecting a Beamforming mode of operation, controlling Volume and selecting an Active wireless receiver.
19. A non-transitory computer readable medium according to claim 18 configured to provide that Beamforming is controlled by horizontal eye gaze, and Volume is controlled by vertical eye gaze.
20. A portable electrooculography (EOG) signal capture system comprising: a sensor array adapted for being located at one of a left or right ear of a user, the sensor array comprising a number Ns of electric potential sensors, respectively, for sensing respective electric potentials from the user's head, where Ns is larger than or equal to two; and electronic circuitry coupled to the sensor array and configured to provide at least two different combined signals (S.sub.BF1, S.sub.BF2), each being representative of a weighted combination of said electric potentials, or of voltage difference signals, and wherein a difference between said at least two different combined signals (?PBF=S.sub.BF1?S.sub.BF2) represents an electrooculography signal (?P.sub.EOG) from one or both eyes of said user, said electrooculography signal (?P.sub.EOG) representing a current eye gaze used to estimate a current eye gaze angle ?? of the user, wherein said electronic circuitry receives a reference potential from a reference sensor, and said portable electrooculography (EOG) signal capture system comprises respective differential buffers to provide respective voltage difference signals from said electric potentials and said reference potential, wherein functionality of the portable EOG signal capture system is controlled via the current eye gaze angle ??.
21. A non-transitory computer readable medium storing an application comprising executable instructions configured to be executed on an auxiliary device to implement a user interface for a hearing device adapted for being located at or in an ear of a user, said hearing device comprising a portable electrooculography signal capture system comprising: a sensor array adapted for being located at one of a left or right ear, the sensor array comprising a number Ns of electric potential sensors, respectively, for sensing respective electric potentials from the user's head, where Ns is larger than or equal to two; and electronic circuitry coupled to the sensor array and configured to provide at least two different combined signals (S.sub.BF1, S.sub.BF2), each being representative of a weighted combination of said electric potentials, or of signals derived therefrom, and wherein a difference between said at least two different combined signals (?PBF=S.sub.BF1?S.sub.BF2) represents an electrooculography signal (?P.sub.EOG) from one or both eyes of said user, said electrooculography signal (?P.sub.EOG) representing a current eye gaze used to estimate a current eye gaze angle ?? of the user, wherein said combined signals (S.sub.BF1, S.sub.BF2) are determined from said electric potentials from the user's head by applying beamformer weights representative of first and second beam patterns (BP1, BP2), respectively, which implement beam patterns that do not attenuate signals from respective first and second target directions of an eye ball of the user, wherein the user interface is configured to allow a user to select functions of the hearing device for eye gaze control, said functions include one or more of selecting a Beamforming mode of operation, controlling Volume and selecting an Active wireless receiver, and Beamforming is controlled by horizontal eye gaze, and Volume is controlled by vertical eye gaze.
Description
BRIEF DESCRIPTION OF DRAWINGS
(1) The aspects of the disclosure may be best understood from the following detailed description taken in conjunction with the accompanying figures. The figures are schematic and simplified for clarity, and they just show details to improve the understanding of the claims, while other details are left out. Throughout, the same reference numerals are used for identical or corresponding parts. The individual features of each aspect may each be combined with any or all features of the other aspects. These and other aspects, features and/or technical effect will be apparent from and elucidated with reference to the illustrations described hereinafter in which:
(2)
(3)
(4)
(5)
(6)
(7)
(8)
(9)
(10)
(11)
(12)
(13)
(14)
(15) The figures are schematic and simplified for clarity, and they just show details which are essential to the understanding of the disclosure, while other details are left out. Throughout, the same reference signs are used for identical or corresponding parts.
(16) Further scope of applicability of the present disclosure will become apparent from the detailed description given hereinafter. However, it should be understood that the detailed description and specific examples, while indicating preferred embodiments of the disclosure, are given by way of illustration only. Other embodiments may become apparent to those skilled in the art from the following detailed description.
DETAILED DESCRIPTION OF EMBODIMENTS
(17) The detailed description set forth below in connection with the appended drawings is intended as a description of various configurations. The detailed description includes specific details for the purpose of providing a thorough understanding of various concepts. However, it will be apparent to those skilled in the art that these concepts may be practised without these specific details. Several aspects of the apparatus and methods are described by various blocks, functional units, modules, components, circuits, steps, processes, algorithms, etc. (collectively referred to as elements). Depending upon particular application, design constraints or other reasons, these elements may be implemented using electronic hardware, computer program, or any combination thereof.
(18) The electronic hardware may include microprocessors, microcontrollers, digital signal processors (DSPs), field programmable gate arrays (FPGAs), programmable logic devices (PLDs), gated logic, discrete hardware circuits, and other suitable hardware configured to perform the various functionality described throughout this disclosure. Computer program shall be construed broadly to mean instructions, instruction sets, code, code segments, program code, programs, subprograms, software modules, applications, software applications, software packages, routines, subroutines, objects, executables, threads of execution, procedures, functions, etc., whether referred to as software, firmware, middleware, microcode, hardware description language, or otherwise.
(19) In an aspect of the present application, a electrooculography signal capturing system, e.g. an eye gaze estimation system, is provided. The system comprises a sensor part adapted for being located at or in a left or a right ear and/or for fully or partially being implanted in the head at a left or a right ear of a user. The sensor part comprises at least two electrical potential sensors for sensing respective electrical potentials (P.sub.1, P.sub.2) from the user's head. The e system further comprises electronic circuitry coupled to the sensor array and configured to provide at least two different beamformed signals (S.sub.BF1, S.sub.BF2), each being representative of a weighted combination of said electric potentials, or of signals derived therefrom, and wherein a difference between said at least two different beamformed signals (?P.sub.BF=S.sub.BF1=S.sub.BF2)at least in a specific electrooculography mode of operationrepresents an electrooculography signal (?P.sub.EOG) from one or both eyes of said user. The electronic circuitry may further comprise a processor for providing an estimate of a current eye gaze angle in dependence of said EOG signal (?P.sub.EOG) or a processed version thereof.
(20) An aspect of the present application also relates to the field of hearing devices, e.g. hearing aids.
(21) In the following, some signal names may alternatingly comprise subscripts X.sub.y or be written in normal form (Xy, without subscripts) without any intended difference in meaning, e.g. an electrooculography signal may intermittently be denoted ?P.sub.EOG or ?PEOG, etc.
(22)
(23)
(24)
(25) The electronic circuitry (COMPU) comprises respective differential buffers (here analogue to digital converters, A/D) to provide respective voltage difference signals (?P1, ?P2) from the electric potentials (P1, P2) and the reference potential Pref. The electronic circuitry (COMPU) further comprises an electric potential beamformer (BF) providing two different beamformed signals (SBF1, SBF2), each being formed as a weighted combination of the electric potentials (P1, P2), or of signals derived therefrom (e.g. voltage difference signals (?P1, ?P2)). The beamformed signals are e.g. formed by respective fixed beamformers (e.g. as illustrated in
(26) The electronic circuitry (COMPU) or parts thereof (e.g. the beamformer (BF) and/or the processor (PR)) may be located in the ear piece, or be located in another device, e.g. form part of a hearing device (e.g. located behind an ear of the user) or another electronic device adapted for processing the electrooculography signals. Preferably, at least the differential buffers (A/D) are located in the ear piece. Thereby the electric connections to the sensor array (e.g. the electrodes) can be wired. The voltage difference signals (?P1, ?P2) may e.g. be wirelessly transmitted to another device, e.g. another part (e.g. a behind the ear (BTE) part) of a hearing device, and further processed there.
(27) Functionality (e.g. beamforming of a microphone system) of an electronic device, e.g. a hearing device, comprising the earpiece or in communication with the ear piece, may be controlled via an estimate of a current eye gaze.
(28)
(29)
(30)
(31) Each of the left and right hearing devices (HD.sub.left, HD.sub.right) comprises a forward path for propagating a signal representing sound from an input unit (IU) to an output unit (OU). The hearing devices each comprises an input unit (IU) for providing an electric input signal (IN.sub.left, IN.sub.right) based on an input sound signal. The input unit (IU) may comprise an input transducer, e.g. a microphone or an accelerometer or other vibration sensor. The hearing device (e.g. the forward path) comprises a signal processor (SPU), e.g. adapted to provide a frequency dependent gain to compensate for a hearing loss of a user, and/or to otherwise enhance the electric input signal (IN.sub.left, IN.sub.right) and to provide a processed electric signal (OUT.sub.left, OUT.sub.right). The hearing devices further comprise an output unit (OU) for providing a stimulus (OUS.sub.left, OUS.sub.right) perceived by the user as an acoustic signal based on the processed electric signal. In an embodiment, an output unit comprises a number of electrodes of a cochlear implant or a vibrator of a bone conducting hearing device. In an embodiment, an output unit comprises a receiver (speaker) for providing the stimulus as an acoustic signal to the user.
(32) The (configurable) signal processor (SPU) is configured to process the electric input signal (IN.sub.left, IN.sub.right) depending on a number of processing algorithms and to provide a resulting processed signal (OUT.sub.left, OUT.sub.right). One or more of the signal processing algorithms are parameterized and parameters controllable in dependence on detectors or analysis of properties of the present environment of the user and/or of the present condition of the user. For example, the processing performed by the signal processor (SPU) may be influenced by the resulting EOG, and/or EEG measurements (e.g. based on voltage difference signal ?V.sub.EOG or ?V.sub.EEG or data derived therefrom, possibly combined with other parameters). This is illustrated in
(33) The hearing system (HS) is configured to establish a wireless link between the left and right hearing devices (at least for the transfer of potential(s) picked up by the electric potential sensor(s) EPS.sub.right of the right hearing device (HD.sub.right) to the left hearing device (HD.sub.left) and for transferring a resulting estimate of eye gaze angle ?? from the amplifier and processing unit (COMPU) of the left hearing device (HD.sub.left) to the right hearing device (HD.sub.right). The wireless link comprises antenna and transceiver circuitry in the left and right hearing devices (HD.sub.left, HD.sub.right) allowing the transmission of a signal representing a voltage difference between the hearing devices. To that end, each of the left and right hearing devices (HD.sub.left, HD.sub.right) are configured to provide respective ground potentials P(HD.sub.lGND), and P(HD.sub.rGND), which are used as reference for voltages P.sub.left, P.sub.right provided by electric potential sensors to allow transmission to other devices.
(34) In the right hearing device, the right sensor potential P.sub.right and right ground potential HD.sub.rGND is fed to wireless transmitter unit Tx, e.g. comprising an analogue to digital (AD) converter providing sensor signal ?V.sub.right representative of the right sensor potential P.sub.right and/or appropriate modulation/coding circuitry. In an embodiment, the antenna and transceiver circuitry is adapted to establish a digital link, e.g. according to Bluetooth (e.g. Bluetooth Low Energy) or other low power wireless transmission technology (e.g. ZigBee). In an embodiment, the wireless link is based on near-field communication, e.g. based on capacitive or inductive coupling between corresponding antenna elements in the first and second parts of the hearing assistance system. The left hearing device (HD.sub.left) comprises antenna and transceiver circuitry allowing the reception of the voltage difference ?V.sub.right from the right hearing device. The transceiver of the left hearing device comprises wireless receiver Rx, e.g. comprising a low-noise amplifier and/or demodulation/decoding circuitry to extract right reference potential P.sub.right by addition of left hearing device ground P(HD.sub.lGND). In the embodiment of
(35) Preferably, the left and right hearing device ground potentials P(HD.sub.lGND) and P(HD.sub.rGND) are essentially equal. In an embodiment, the left and right hearing devices are configured to provide that the left and right hearing device ground potentials P(HD.sub.lGND) and P(HD.sub.rGND) are virtual ground potentials defined by the body of the user via capacitive coupling to the ground potential of the earth, as e.g. used in communication via body networks, cf. e.g. EP2997893A1.
(36)
(37) The voltage difference signals (?P1 and ?P2) are fed to electric potential beamformer (BF) which provides two different beamformed signals (S.sub.BF1, S.sub.BF2), which are formed as a weighted combination of the voltage difference signals ?P.sub.i, i=1, 2. In an embodiment, the beamformed signals S.sub.BFj are linear combinations of the voltage difference signals ?P.sub.i, i.e. S.sub.BF1=w.sub.11*?P.sub.1+w.sub.12*?P.sub.2, and S.sub.BF2=w.sub.21*?P.sub.1+w.sub.22*?P.sub.2, where the beamformer weights w.sub.ji are generally complex, frequency dependent values (e.g. fixed values for a fixed beamformer and time dependent values for an adaptive beamformer).
(38) The beamformer filtering coefficients w.sub.ij may e.g. be determined as is customary in the field of antenna array circuits, cf. e.g. Wikipedia entries on Antenna arrays (https://en.wikipedia.org/wiki/Antenna_array) or Dipole antenna (https://en.wikipedia.org/wiki/Dipole_antenna), or a standard textbook on antenna arrays (e.g. Antenna Arrays: A Computational Approach by Randy L. Haupt, Wiley-IEEE Press, May 2010). The electric potential beamformers may e.g. have an equivalent structure to a generalized sidelobe canceller (GSC). In an embodiment, the electric potential beamformers may comprise a minimum variance distortionless response (MVDR) beamformer, which (ideally) keeps signals from a target direction unchanged, while attenuating signals from other directions maximally Reference is further made to [Van Veen & Buckley; 1988] providing an overview of antenna array beamforming, in particular to FIGS. 2.1(a) and (b), where the basic terminology and concepts are defined and to equations (2.1), (2.2) and (2.3) as the general beamforming algorithm. The same beamforming methods can be used regardless of the contents or meaning of the data (be they electromagnetic fields, acoustic vibrations or electric potentials) picked by a configuration (e.g. an array) of sensors (be they antennas or microphones or electric potential sensors). Beamformer weights are designed in the same way that FIR-filter weights are determined, i.e. by defining an ideal spatial filter (in contrast to magnitude response in FIR-filters) and approximate with a FIR optimization algorithm, e.g. the Remez algorithm or any of the suggestions of [Elliott; 1987], cf. chapter 2. Examples from [Van Veen & Buckley; 1988] of different beamforming techniques that may be used in the present context: III. DATA INDEPENDENT BEAMFORMING (page 11 ff.): A. Classical Beamforming, cf. page 11-12 and the example in FIG. 3.1 with 16?16 array of sensors. B. General Data Independent response design, cf. page 12-13. May be used in connection with a scenario as illustrated in
(39) In an embodiment, the beamforming technology is based on phased arrays, and is used to scan a certain part of the skull (e.g. one eye at the time) by appropriately adjusting delays and weights (as e.g. described in sections III, IV and V of [Van Veen & Buckley; 1988]).
(40) The beamformer weights (w.sub.1, w.sub.12) and (w.sub.21, w.sub.22) may e.g. be representative of first and second beam patterns BP1 and BP2, respectively, as schematically illustrated in
(41) The electrooculography signal (?P.sub.EOG) provided in
(42) The estimation of eye gaze angle may be activated (e.g. automatically, by the user) in a hearing device in a specific electrooculography mode of operation, e.g. from a remote control, such as a smartphone, see e.g.
(43) In the exemplary system of
(44) In an embodiment, the electric circuitry is configured to provide that the two or more beamformers are adaptive and configured to maximize a difference between the beamformed signals.
(45) In practice, measurement artifacts, such as drift over time of the EOG signal and/or muscular activity may be significant. In that case, measures to compensate for that may preferably be taken, e.g. using Kalman filtering, cf. e.g. [Manabe & Fukamoto; 2010].
(46)
(47)
(48) The hearing aid (HD) exemplified in
(49) In an embodiment, the hearing device (HD) of
(50) The hearing aid (HD) may e.g. comprise a directional microphone system (including a beamformer filtering unit) adapted to spatially filter out a target acoustic source among a multitude of acoustic sources in the local environment of the user wearing the hearing aid, and to suppress noise from other sources in the environment. The beamformer filtering unit may receive as inputs the respective electric signals from input transducers IT.sub.1, IT.sub.2, IT.sub.3 (and possibly IT.sub.4) (or any combination thereof) and generate a beamformed signal based thereon. In an embodiment, the directional system is adapted to detect (such as adaptively detect) from which direction a particular part of the microphone signal (e.g. a target part and/or a noise part) originates. In an embodiment, the beam former filtering unit is adapted to receive inputs from a user interface (e.g. a remote control or a smartphone) regarding the present target direction. In an embodiment, the beamformer filtering unit is controlled or influenced by signals from the sensing electrodes (or processed versions thereof, e.g. EOG-signals representative of eye gaze of the user). In an embodiment, the direction of a beam (or a zero point) of the beamformer filtering unit is thereby controlled or influenced. In another embodiment, the input from one of the wireless receivers is selected based on signals from the sensing electrodes (or processed versions thereof, e.g. EOG-signals representative of eye gaze of the user). The memory unit (MEM) may e.g. comprise predefined (or adaptively determined) complex, frequency dependent constants (W.sub.ij) defining predefined (or adaptively determined) or fixed beam patterns (e.g. omni-directional, target cancelling, pointing in a number of specific directions relative to the user), together defining the beamformed signal Y.sub.BF.
(51) The hearing aid of
(52) The hearing aid (HD) according to the present disclosure may comprise a user interface UI, e.g. as shown in
(53) The auxiliary device (AD) and the hearing aid (HD) are adapted to allow communication of data, including data representative of the currently selected function to be controlled via eye gaze to the hearing aid via a, e.g. wireless, communication link (cf. dashed arrow WL2 in
(54) The hearing aid may comprise a number of wireless receivers (e.g. symbolized by WLR, in
(55) The aspects of eye gaze control described in connection with
(56)
(57) The hearing device (HD) comprises a system for capturing electrooculography signals, e.g. an eye gaze estimation system, according to the present disclosure, for picking up bio signals from the user's body. The system comprises an electric potential sensor part adapted for being located at or in an ear and/or for fully or partially for being implanted in the head of a user. The sensor part comprises a number of electrical potential sensors (EPSi) for sensing an electrical potential (Pi) from the body of the user (here i=1, 2), in particular from the head, e.g. due to brain activity or eye movement. The sensor part may further comprise a reference sensor electrode (EPSref) for providing a reference voltage (Pref). The electrical potential sensor (EPSx) may e.g. comprise an electrode configured to contact skin or tissue of the user's head, when the electric potential sensor part (e.g. the hearing device) is operationally mounted on the user (e.g. in an ear canal) or implanted in the head of the user. The eye gaze estimation system further comprises an amplifier and processing unit (COMPU), in the form of electronic circuitry coupled to the electrical potential sensor part to provide an amplified EOG signal in the form of an estimated eye gaze angle ?? (see e.g.
(58) The hearing device comprises at least two electrical potential sensor (EPS) (EPS1, EPS2), each providing a potential (P1, P2) representing an electro-oculography potential. The two potentials are used as inputs to an electric potential beamformer as discussed in connection with
(59) In an embodiment, at least one (such as all) of the input units comprises an input transducer, e.g. a microphone. In an embodiment, at least one (such as all) of the input units comprises a wireless transceiver, e.g. a wireless receiver, e.g. configured to receive a signal representative of sound picked up by a remote (wireless) microphone.
(60) The hearing device may further comprise or be coupled to a location sensor unit (LOCD) providing location data (LC) representative of a current location of the user, e.g. representative of the user's head, e.g. in a fixed coordinate system (e.g. relative to a specific location, e.g. a room). In an embodiment, the location sensor comprises a head tracker. In an embodiment, the location sensor comprises an accelerometer and a gyroscope. In an embodiment, the location sensor comprises a 9 degree of freedom sensor, comprising a 3D accelerometer, a 3D gyroscope, and a 3D magnetometer.
(61) In an embodiment, the hearing device further comprises a wireless transceiver and appropriate antenna circuitry (ANT, Rx/Tx) allowing the mentioned reception of electric potentials (Pix) (or voltage differences (?Pix) referring the potentials to a reference) from and transmission of such signals (Pi, ?Pi) to a contra-lateral hearing device, e.g. representative of eye movement, via a wireless link (X-WL) or wired connection (X-W), cf. waved and dotted straight, arrowed lines denoted To/From other HD in
(62) The hearing device further comprises a control unit (CONT) for providing a control signal for controlling a function of the hearing device based on the EarEOG signal(s), e.g. selecting wireless reception from a particular person, or as exemplified in
(63) In a specific mode of operation (a learning mode), the calculation unit may be configured to determine locations representing preferred eye gaze directions of the user. The locations (e.g. represented in a fixed coordinate system) may be stored in a memory of the hearing device (or in an auxiliary device, e.g. a smartphone or the like). The locations may e.g. be displayed via a user interface (e.g. via an app of a smartphone).
(64)
(65) Monaural, Single Electrode EOG:
(66) Up till now, it has been assumed that the measurement of Electroocculography (EOG) signals signals (e.g. to monitor eye gaze of a user) requires a differential measurement, e.g. from two independent electrodes located at left and right eyes of a user (or in left and right ears of a user, cf. e.g. EP3185590A1) or a directional system utilizing at least two electrodes located at the same eye or ear of a user (as proposed in the present application).
(67) The present inventor has uncovered that it is possible to capture an EOG signal (e.g. representative of eye gaze) from a single monitoring electrode e.g. located at one of the eyes (or ears) of the user. In particular, capacitive sensors (so-called electric potential sensors, EPS) lend themselves to such application, e.g. sensors produced by Plessey Semiconductors (cf. e.g. http://www.plesseysemiconductors.com/).
(68) Measurement at one location is advantageous, in particular in wearable devices, such as ear plugs or hearing aids, because it removes the need to transmit measurements from the pick-up locations (e.g. left right eyes or ears) to other devices for determining a resulting EOG-value.
(69)
(70) In the electrical potential sensor (EPS) of
(71) Body networks using virtual ground is e.g. discussed in EP2997893A1.
(72) In a hearing device, a capacitive electrode may be formed by coating techniques, e.g. on a generic housing, or on a dome or instant fit device (or on a customized housing).
(73) Beamforming. Inverse Model.
(74)
(75) Given a number of sensors (e.g. electric potential sensors) spatially distributed in an array and a number of sources contributing to the signal received in all sensors, the contribution of each source to the captured signal can be reconstructed via calculating an Inverse Model (i.e. from observed data to source location). Three signal sources (denoted Source 1, 2, 3 in
(76) The observed data at the sensors (here e.g. electric potentials, or electric potential differences) can be considered as
X=s*h
(77) Where X is the data matrix of all sensors across time, s is a single source in question (e.g. one of Source 1, 2, 3) and h is the transformation model vector of the source at each sensor. The beamformer procedure aims to create a number of spatial filters w such that
w.sup.T*X=?
(78) Where ? is the estimate of the source. The formula can be rewritten as
w.sup.T*h*s=?
(79) Given the constraints of unity gain to the desired source while attenuating all interfering sources,
w.sup.T*h=1 while w.sup.T*h.sub.non desired source=0
(80) The beamformer algorithm seeks to minimise the variance of the filter output in the form of
w.sup.T=[h.sup.T*Cov(X).sup.?1*h].sup.?1*h.sup.T*Cov(X).sup.?1
(81) It is intended that the structural features of the devices described above, either in the detailed description and/or in the claims, may be combined with steps of the method, when appropriately substituted by a corresponding process.
(82) As used, the singular forms a, an, and the are intended to include the plural forms as well (i.e. to have the meaning at least one), unless expressly stated otherwise. It will be further understood that the terms includes, comprises, including, and/or comprising, when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. It will also be understood that when an element is referred to as being connected or coupled to another element, it can be directly connected or coupled to the other element but an intervening elements may also be present, unless expressly stated otherwise. Furthermore, connected or coupled as used herein may include wirelessly connected or coupled. As used herein, the term and/or includes any and all combinations of one or more of the associated listed items. The steps of any disclosed method is not limited to the exact order stated herein, unless expressly stated otherwise.
(83) It should be appreciated that reference throughout this specification to one embodiment or an embodiment or an aspect or features included as may means that a particular feature, structure or characteristic described in connection with the embodiment is included in at least one embodiment of the disclosure. Furthermore, the particular features, structures or characteristics may be combined as suitable in one or more embodiments of the disclosure. The previous description is provided to enable any person skilled in the art to practice the various aspects described herein. Various modifications to these aspects will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other aspects.
(84) The claims are not intended to be limited to the aspects shown herein, but is to be accorded the full scope consistent with the language of the claims, wherein reference to an element in the singular is not intended to mean one and only one unless specifically so stated, but rather one or more. Unless specifically stated otherwise, the term some refers to one or more.
(85) Accordingly, the scope should be judged in terms of the claims that follow.
REFERENCES
(86) [Brandstein & Ward; 2001] EP2997893A1 (OTICON) 23.03.2016 [Manabe & Fukamoto; 2010] H. Manabe, M. Fukamoto, Using earphones to Perform Gaze Detection for Wearable Interfaces, NTT Docomo Technical Journal, Vol. 12, No. 3, pp. 12-17 (2010). EP3185590A1 (OTICON) 28.06.2017, co-pending European patent application 16205776.4 (published on 28 Jun. 2017 as EP3185590A1) with the title A hearing device comprising a sensor for picking up electromagnetic signals from the body, filed at the EPO on 21 Dec. 2016. [Van Veen & Buckley; 1988] Barry Van Veen & Kevin M. Buckley, Beamforming: A Versatile Approach to Spatial Filtering, IEEE ASSP Magazine, April 1988, pp. 4-24. [Elliott; 1987]. Douglas F. Elliott, ed., Handbook of Digital Signal Processing: Engineering applications. Academic Press: San Diego, chapter 2, Vaidyanathan, Design and Implementation of Digital FIR Filters]