A METHOD FOR MONITORING AND DETECTING IF HEARING INSTRUMENTS ARE CORRECTLY MOUNTED

Abstract

A binaural hearing aid system comprises left and right hearing instruments each comprising a BTE-part comprising a housing configured to be located at or behind an outer ear of the user and an acceleration sensor configured to measure acceleration in at least two directions relative to the housing of the BTE-part and to provide acceleration data indicative thereof. A first one of the left and right hearing instruments comprises a transmitter configured to transmit said acceleration data to the other hearing instrument, and the other hearing instrument comprises a receiver for receiving said acceleration data from the first one of the hearing instruments. The other hearing instrument further comprises a controller for detecting whether or not the left and right hearing instruments are correctly mounted in dependence of a similarity measure between said acceleration data provided by the left and right hearing instruments. A method of operating a binaural hearing aid system is further disclosed.

Claims

1. A binaural hearing aid system comprising left and right hearing instruments adapted to be located at or in left and right ears, respectively, of a user, each of the left and right hearing instruments comprising a BTE-part configured to be located at or behind an outer ear of the user, the BTE-part comprising a housing; an acceleration sensor configured to measure acceleration in at least two directions relative to the housing of the BTE-part and to provide acceleration data indicative thereof; at least a first one of the left and right hearing instruments comprising a transmitter being configured to allow transmission of acceleration data from the acceleration sensor to the second one of the left and right hearing instruments, and at least a second one of the left and right hearing instruments comprising a receiver being configured to allow reception of the acceleration data from the acceleration sensor of the first one of the left and right hearing instruments, and, wherein at least the second one of the left and right hearing instruments comprising a controller configured to detect whether or not the left and right hearing instruments are correctly mounted in dependence of a similarity measure between said acceleration data provided by the left and right hearing instruments.

2. A binaural hearing aid system according to claim 1 wherein the similarity measure is constituted by or comprises a correlation measure.

3. A binaural hearing aid system according to claim 1 wherein the acceleration data of each of the left and right hearing instruments comprises acceleration data representing the at least two directions.

4. A binaural hearing aid system according to claim 1 wherein the controller is configured to base the decision of whether or not the left and right hearing instruments are correctly mounted on an estimated cross-covariance matrix obtained from the outer product of the acceleration signals from the left and right hearing instruments, respectively.

5. A binaural hearing aid system according to claim 4 wherein the controller is configured to decide that the hearing instruments are correctly mounted when the magnitude of the diagonal elements of the cross-covariance matrix are larger than a threshold value.

6. A binaural hearing aid system according to claim 4 wherein the controller is configured to decide that the hearing instruments are in-correctly mounted when at least one, or at least two, of the off-diagonal elements is(are) relatively high, and the diagonal elements at the same time have relatively low values.

7. A binaural hearing aid system according to claim 4 wherein the controller is configured to decide that the hearing instruments are in-correctly mounted when at least one, or at least two, of the off-diagonal elements is(are) larger than a threshold value, and the diagonal elements at the same time are smaller than a threshold value.

8. A binaural hearing aid system according to claim 1 wherein the controller comprises a neural network configured to detect an incorrect mounting of the hearing instruments.

9. A binaural hearing aid system according to claim 8 wherein the neural network is configured to receive acceleration data from the left and right hearing instruments as input features, or elements of a cross-covariance matrix of the acceleration data or otherwise processed versions of the acceleration data from the left and right hearing instruments.

10. A binaural hearing aid system according to claim 1 wherein the controller comprises a neural network wherein at least some of the layers are implemented as a recurrent neural network.

11. A binaural hearing aid system according to claim 1 configured to trigger a warning of the user in case it is detected that the left and/or right hearing instruments is/are not correctly mounted.

12. A binaural hearing aid system according to claim 1 configured to disable a directional noise reduction algorithm of the left and right hearing instruments in case it is detected that the left and/or right hearing instruments is/are not correctly mounted.

13. A binaural hearing aid system according to claim 1 configured to provide that the detection of whether the left and right hearing instruments are correctly mounted is dependent on other input features than said acceleration data.

14. A binaural hearing aid system according to claim 13 configured to provide that the detection of whether the left and right hearing instruments are correctly mounted is dependent on the detection of a changed feedback path estimate

15. A binaural hearing aid system according to claim 1 configured to provide that correlation between the acceleration data provided by the left and the right instruments is maximum when at least one direction relative to the housing of the BTE-parts of the respective left and right hearing instruments are parallel.

16. A binaural hearing aid system according to claim 2 configured to average the covariance between the acceleration data provided by the left and the right instruments across time based only on samples in which acceleration above a certain threshold has been detected.

17. A binaural hearing aid system according to claim 1 wherein at least one of the left and right hearing instruments comprises a predefined reference position representing a correctly mounted hearing instrument, and wherein the binaural hearing aid system is configured to provide that the detection of whether the left and right hearing instruments are correctly mounted is dependent on said reference position.

18. A binaural hearing aid system according to claim 1 wherein at last one of the left and right hearing instruments is constituted by or comprises an air-conduction type hearing aid, a bone-conduction type hearing aid, a cochlear implant type hearing aid, or a combination thereof.

19. A method of operating a binaural hearing aid system comprising left and right hearing instruments adapted to be located at or in left and right ears, respectively, of a user, each of the left and right hearing instruments comprising a BTE-part configured to be located at or behind an outer ear of the user, the BTE-part comprising a housing; and an acceleration sensor configured to measure acceleration in at least two directions relative to the housing of the BTE-part; the method comprising provide by said acceleration sensor acceleration data indicative of a current acceleration in said at least two directions; transmitting by at least a first one of the left and right hearing instruments acceleration data from the acceleration sensor to the second one of the left and right hearing instruments, and receiving by at least a second one of the left and right hearing instruments the acceleration data from the acceleration sensor transmitted by the first one of the left and right hearing instruments, and, determining a similarity measure between said acceleration data provided by the left and right hearing instruments; and detecting by the at least the second one of the left and right hearing instruments whether or not the left and right hearing instruments are correctly mounted in dependence of said correlation measure.

20. A non-transitory computer readable medium storing a computer program comprising instructions which, when the program is executed by a computer, cause the computer to carry out the method of claim 19.

Description

BRIEF DESCRIPTION OF DRAWINGS

[0127] The aspects of the disclosure may be best understood from the following detailed description taken in conjunction with the accompanying figures. The figures are schematic and simplified for clarity, and they just show details to improve the understanding of the claims, while other details are left out. Throughout, the same reference numerals are used for identical or corresponding parts. The individual features of each aspect may each be combined with any or all features of the other aspects. These and other aspects, features and/or technical effect will be apparent from and elucidated with reference to the illustrations described hereinafter in which:

[0128] FIG. 1 is a schematic illustration of two hearing instruments mounted at the ear of a toddler, where the instrument is correctly positioned on the toddler's left ear but incorrectly positioned on the toddler's right ear,

[0129] FIG. 2A illustrates in side (top) and top (bottom) views of two hearing instruments mounted in parallel that they will have highly correlated acceleration patterns, such that the left and right x acceleration pattern will be highly correlated, and similarly for accelerations measured in the y and z-direction; and

[0130] FIG. 2B illustrates again in side (top) and top (bottom) views a case where the hearing instruments are not mounted in parallel (or as intended), where it can be detected that accelerations caused by movements will be correlated in a different way (the x and y accelerations are not parallel in the two hearing instruments), and

[0131] FIG. 3A shows measured left and right acceleration patterns in left and right instrument, respectively, with two correctly mounted instruments; and

[0132] FIG. 3B shows measured left and right acceleration patterns in left and right instrument, respectively, in the case, where the right instrument is incorrectly mounted.

[0133] FIG. 4 shows a first embodiment of a binaural hearing aid system according to the present disclosure,

[0134] FIG. 5 shows a second embodiment of a binaural hearing aid system according to the present disclosure,

[0135] FIG. 6A shows an example of cross-covariances of acceleration data between a left and a right correctly mounted hearing instrument of a binaural hearing aid system according to the present disclosure; and

[0136] FIG. 6B shows an example of cross-covariances of acceleration data between a left and a right in-correctly mounted hearing instrument of a binaural hearing aid system according to the present disclosure, and

[0137] FIG. 7A shows a controller comprising a neural network according to a first embodiment of the present disclosure;

[0138] FIG. 7B shows a controller comprising a neural network according to a second embodiment of the present disclosure;

[0139] FIG. 7C shows a controller comprising a neural network according to a third embodiment of the present disclosure;

[0140] FIG. 7D shows a controller comprising a neural network according to a fourth embodiment of the present disclosure;

[0141] FIG. 7E shows x a controller comprising a neural network according to a fifth embodiment of the present disclosure; and

[0142] FIG. 7F shows a controller comprising a neural network according to a sixth embodiment of the present disclosure.

[0143] The figures are schematic and simplified for clarity, and they just show details which are essential to the understanding of the disclosure, while other details are left out. Throughout, the same reference signs are used for identical or corresponding parts.

[0144] Further scope of applicability of the present disclosure will become apparent from the detailed description given hereinafter. However, it should be understood that the detailed description and specific examples, while indicating preferred embodiments of the disclosure, are given by way of illustration only. Other embodiments may become apparent to those skilled in the art from the following detailed description.

DETAILED DESCRIPTION OF EMBODIMENTS

[0145] The detailed description set forth below in connection with the appended drawings is intended as a description of various configurations. The detailed description includes specific details for the purpose of providing a thorough understanding of various concepts. However, it will be apparent to those skilled in the art that these concepts may be practiced without these specific details. Several aspects of the apparatus and methods are described by various blocks, functional units, modules, components, circuits, steps, processes, algorithms, etc. (collectively referred to as “elements”). Depending upon particular application, design constraints or other reasons, these elements may be implemented using electronic hardware, computer program, or any combination thereof.

[0146] The electronic hardware may include micro-electronic-mechanical systems (MEMS), integrated circuits (e.g. application specific), microprocessors, microcontrollers, digital signal processors (DSPs), field programmable gate arrays (FPGAs), programmable logic devices (PLDs), gated logic, discrete hardware circuits, printed circuit boards (PCB) (e.g. flexible PCBs), and other suitable hardware configured to perform the various functionality described throughout this disclosure, e.g. sensors, e.g. for sensing and/or registering physical properties of the environment, the device, the user, etc. Computer program shall be construed broadly to mean instructions, instruction sets, code, code segments, program code, programs, subprograms, software modules, applications, software applications, software packages, routines, subroutines, objects, executables, threads of execution, procedures, functions, etc., whether referred to as software, firmware, middleware, microcode, hardware description language, or otherwise.

[0147] The present disclosure relates to the field of hearing devices, e.g. hearing aids, configured to be worn at one or more ears of a user. The present relates in particular to a scheme for detecting whether or not the hearing device (or hearing devise) are correctly mounted at the ear (or ears) of the user. The disclosure may also be relevant for wireless headset, e.g. in the form of earbuds or hearables, e.g. to control if a specific feature shall be in monaural or binaural mode.

[0148] In a binaural setup, the fact that the acceleration patterns between the two hearing instruments will be highly correlated while the hearing instruments are worn at the two ears can be advantageously utilized. If the hearing instruments both are pointing towards the same direction (or calibrated such that the axis of the left and the right accelerometers are aligned), it is expected that the acceleration along the x-axis of an instrument worn at one ear mainly will be correlated with the acceleration along the x-axis measured along the instrument worn at the contralateral ear. And similarly, the accelerations measured along the y axis of the instruments will be correlated as well as the accelerations along the z axis.

[0149] By comparing the acceleration patterns of the two instruments, it is thus possible to detect whether the two instruments are pointing towards the same direction or pointing in different directions, as illustrated in FIG. 1.

[0150] FIG. 1 is a schematic illustration of two hearing instruments mounted at the ear of a toddler, where the instrument is correctly positioned on the toddler's left ear but incorrectly positioned on the toddler's right ear. Even though both moulds are correctly mounted in the ear canals, the BTE part of the hearing instrument is not necessarily correctly positioned behind the ear.

[0151] If a hearing instrument of a binaural hearing aid system is not correctly positioned, it may affect the quality of the signal presented to the listener. E.g. directional noise reduction may assume that the target is in a particular direction compared to the hearing instrument, e.g. in front of the instrument. If the instrument is not correctly mounted, noise reduction may degrade the audio signal rather than improving it. Furthermore, if the placement is incorrect, the applied gain is incorrect, such that the audibility of sounds or the speech intelligibility is degraded. It is thus important to detect if the hearing instrument is incorrectly mounted. This is in particular important for people who are not aware that the hearing instrument is not working as intended (e.g. toddlers).

[0152] In case it is detected that the directions of the two instruments are not aligned, a warning could be triggered. This could e.g. be a warning message sent from the hearing instrument to the caretaker's phone (or similar device).

[0153] Another consequence of detecting an unintended hearing instrument position could be to disable directional noise reduction, hereby preventing a directional noise reduction from attenuating audio from a target direction. The hearing instrument should thus be set into an omnidirectional mode, until it is detected that the issue has been fixed. This may as well be detected automatically or manually, e.g. via a phone connected to the instrument.

[0154] The detection could be combined with other input features e.g. the detection of a changed feedback path estimate.

[0155] One way to detect if the left and right instrument is not aligned is to monitor the covariance between the acceleration measured at the left and the right instruments (e.g. determining the cross-covariance matrix).

[0156] If the left and right instruments are mounted as intended, it is assumed that the x, y, and z acceleration patterns measured on the left ear, are correlated in a certain way with the corresponding acceleration patterns measured on the right ear. This is illustrated in FIG. 2A, 2B. FIG. 2A. shows the intended mounting, where the acceleration directions on the left and the right instruments are parallel (for simplicity, we show the intended directions as parallel, as we can always rotate the acceleration patterns such that they are parallel when the instrument is used as intended).

[0157] FIG. 2A shows in side (top) and top (bottom) views of two hearing instruments mounted in parallel that they will have highly correlated acceleration patterns, such that the left and right x acceleration pattern will be highly correlated, and similarly for accelerations measured in the y and z-direction.

[0158] FIG. 2B illustrates again in side (top) and top (bottom) views a case where the hearing instruments are not mounted in parallel (or as intended), where it can be detected that accelerations caused by movements will be correlated in a different way (the x and y accelerations are not parallel in the two hearing instruments). Or phrased more generally: It can be detected that the acceleration coordinate systems in the two hearing instruments (each coordinate system is spanned by the x, y and z accelerations) are mis-aligned.

[0159] When the left and right acceleration patterns are parallel, it is likely that the highest correlation is observed between the left and right x-axes, the left and right y-axes, and the left and the right z-axes. FIG. 2B shows the left and right accelerometer axes, in the case where the instruments are not mounted as intended (in this case, the right instrument is pointing slightly towards the right).

[0160] FIG. 3A shows measured left and right acceleration patterns in left and right instrument, respectively, with two correctly mounted instruments.

[0161] FIG. 3B shows measured left and right acceleration patterns in left and right instrument, respectively, in the case, where the right instrument is incorrectly mounted.

[0162] FIGS. 3A and 3B each show measurements of acceleration patterns from the x, y, and z directions. FIG. 3A shows the acceleration patterns when the instruments are mounted in parallel (e.g. instruments correctly mounted). The term ‘parallel’ is in the present context taken to mean that the coordinate systems spanned by the three acceleration directions of the respective hearing instruments have parallel axes pointing towards the same directions.

[0163] FIG. 3B shows the measured acceleration patterns when the instruments are not correctly mounted. The mean has been removed in order to focus on the movement-related patterns rather than the gravity-related accelerations. We notice that the left and right acceleration axes are highly correlated in the correctly mounted instruments, whereas the acceleration patterns between the left and the right instruments in the incorrectly mounted instruments are less aligned. Notice, even though the hearing instruments are mounted correctly, the acceleration along the y-axes (pointing towards the side) may be negatively correlated. This is due to the fact that the y-accelerations from the left and right instruments point in the opposite direction, when the head is turned from side to side. Likewise, as illustrated in FIG. 3B, strong, unexpected correlations between different axes may as well indicate that one of the instruments is incorrectly mounted.

[0164] In an embodiment, the alignment is based on the correlation between the left and the right acceleration patterns. The correlation measure may e.g. be based on the covariance between the left and right acceleration pattern, i.e.

R= custom-character a.sub.La.sub.R.sup.T,

[0165] where R (elsewhere in the application also denoted R.sub.LR) is the cross-covariance matrix, a.sub.L=[x.sub.L, y.sub.L, z.sub.L].sup.T is the left instrument's acceleration vector, a.sub.R=[x.sub.R, y.sub.R, z.sub.R].sup.T is the right instrument's acceleration vector at a given time, .sup.T denotes transposition, and custom-character ⋅ denotes the expectation operator. The expectation operator indicates an averaging (e.g. low-pass filtering) across time. The averaging across time may be based on a moving average or on an IIR-filter-based averaging, such as a first order IIR filter. In an embodiment, the averaging is only based on samples in which acceleration above a certain threshold has been detected. If the instruments are well-aligned, it is expected that the diagonal elements will contain the highest values. A high off-diagonal value will indicate that the instruments are not aligned.

[0166] In an embodiment, the cross-covariance matrix is normalized, where the normalization is based on at least one diagonal element.

[0167] In an embodiment, the DC value has been removed from the acceleration data in order to base the alignment assessment on head movements (as implied by the cross-covariance).

[0168] In an embodiment, the DC value has not been removed from the acceleration data in order to base the alignment assessment on the direction of gravity.

[0169] It may be advantageous to make assessments based on the gravity direction (to check if the two instruments agree on the direction of gravity), as well as assessments based on the case where gravity has been removed (to check if the instruments agree on the movement patterns).

[0170] In an embodiment the alignment assessment between the instruments is based on the angle between the left and the right acceleration vector, e.g. given by

[00003] $θ = acos \frac{a_{L}^{T} a_{R}}{\sqrt{a_{L}^{T} a_{L} a_{R}^{T} a_{R}}} . ()$

[0171] Or a similar monotonic function such as

[00004] $\dot{θ} = \frac{{(a_{L}^{T} a_{R})}^{2}}{a_{L}^{T} a_{L} a_{R}^{T} a_{R}} sign (a_{L}^{T} a_{R}) .$

[0172] In this case, we need the direction of gravity (i.e. the mean acceleration value should not be removed from the acceleration data).

[0173] In another embodiment, the alignment assessment procedure is used to detect how well a set of instruments is mounted compared to a reference position. This assessment may be divided into two different parts: Each instrument's tilt may be measured based on the angle between each instrument's angle and a reference gravity direction. As such a measurement does not reveal if the instrument is pointing towards a reference direction in horizontal plane (which is orthogonal to the direction of gravity), an additional assessment on whether the instruments are mounted in parallel may reveal if one or both instruments are pointing towards the sides rather than towards the front. Hereby anatomical differences between individuals can be taken into account during fitting.

[0174] FIG. 4 shows a first embodiment of a binaural hearing aid system according to the present disclosure. FIG. 4 shows a binaural hearing aid system comprising left and right hearing instruments (HA.sub.L, HA.sub.R) adapted to be located at or in left and right ears, respectively, of a user. Each of the left and right hearing instruments comprises a BTE-part configured to be located at or behind an outer ear (pinna) of the user. The BTE-part comprises a housing (e.g. a mechanical shell enclosing components, e.g. electronic components) of the BTE-part. Each of the left and right hearing instruments further comprises an acceleration sensor (ACS.sub.L, ACS.sub.R) configured to measure acceleration in at least two directions relative to the housing of the BTE-part and to provide acceleration data (ACD.sub.L, ACD.sub.R) indicative thereof. The acceleration sensor (ACS.sub.L, ACS.sub.R) of each or the left and right hearing instruments may comprise a two-dimensional (2D), or a three-dimensional (3D), acceleration sensor, configured to provide respective measures of acceleration in two (2D-sensor) or three (3D-sensor) directions relative to the housing of the BTE-part.

[0175] It should be noted that from accelerations from a single dimension (a simple movement detector), we may detect if instruments have similar movement patterns. This may indicate if both instruments are located on the person's ears (or not located on the person's ears), based on similar movement patterns. However, we need at least two acceleration directions in order to determine if the instrument is correctly mounted.

[0176] The binaural hearing aid system is configured to provide that at least a first one (e.g. both) of the left and right hearing instruments comprises a transmitter (Tx.sub.L, Tx.sub.R) configured to allow transmission of acceleration data from the acceleration sensor to the second one of the left and right hearing instruments (or to a separate external device). The binaural hearing aid system is further configured to provide that at least a second one (e.g. both) of the left and right hearing instruments comprises a receiver (Rx.sub.R, Rx.sub.L) configured to allow reception of the acceleration data (ACD.sub.L, ACD.sub.R) from the acceleration sensor of the first one of the left and right hearing instruments. In other words, each of the left and right hearing instruments may be configured to allow an exchange of acceleration data from the respective acceleration sensors between them, cf. e.g. wireless link (WL) and data (ACD.sub.L, ACD.sub.R) in FIG. 4. The acceleration data may be pre-processed before being exchanged between the left and right instruments. The pre-processing may e.g. include low-pass filtering and/or down-sampling (e.g. to limit bandwidth (and hence power) consumption of the wireless link).

[0177] The binaural hearing aid system is further configured to provide that at least the second one (or both) of the left and right hearing instruments comprises a controller (CTR.sub.L, CTR.sub.R) configured to detect whether or not the left and right hearing instruments (HA.sub.L, HA.sub.R) are correctly mounted in dependence of a correlation measure between said acceleration data (ACD.sub.L, ACD.sub.R) provided by the left and right hearing instruments.

[0178] In the embodiment of FIG. 4, each of the left and right hearing instruments (HA.sub.L, HA.sub.R) of the binaural hearing aid system comprises a forward audio signal path configured to process an electric input signal (IN.sub.L, IN.sub.R) representing sound (e.g. sound in an environment around the user) to thereby provide a processed signal (OU.sub.L, OU.sub.R), and to provide output stimuli at the left and right ears, respectively, of the user in dependence of the processed signal. The forward audio signal path comprises a signal processor (SP.sub.L, SP.sub.R) configured to apply a processing algorithm to the electric input signal (IN.sub.L, IN.sub.R) or to a signal originating therefrom and to provide the processed signal (OU.sub.L, OU.sub.R).

[0179] In the embodiment of FIG. 4, the forward audio signal path of each of the left and right hearing instruments (HA.sub.L, HA.sub.R) comprises an input unit (IU.sub.L, IU.sub.R) for providing at least one electric input signal (IN.sub.L, IN.sub.R) representing sound. The input unit (IU.sub.L, IU.sub.R) may comprise an input transducer for the providing at least one electric input signal (IN.sub.L, IN.sub.R). The input transducer may comprise a microphone, a vibration sensor, or an audio receiver. The input unit (IU.sub.L, IU.sub.R) may comprise an analogue to digital converter for converting an analogue input audio signal to a stream of digital audio samples. The input unit may comprise an analysis filter bank for converting a time domain input signal to a multitude of frequency sub-band signals representing the (time-domain) electric input signal in the (time-) frequency domain.

[0180] In the embodiment of FIG. 4, the forward audio signal path of each of the left and right hearing instruments (HA.sub.L, HA.sub.R) comprises an output unit (OU.sub.L, OU.sub.R) for converting an output signal ((OU.sub.L, OU.sub.R), e.g. a processed signal) to stimuli perceivable to the user as sound. The output unit may comprise (OU.sub.L, OU.sub.R) an output transducer ((OT.sub.L, OT.sub.R), cf. e.g. FIG. 5). The output transducer (OT.sub.L, OT.sub.R) may comprise a loudspeaker, a vibrator, a multi-electrode, or an audio transmitter.

[0181] The controller (CTR.sub.L, CTR.sub.R) may be configured to determine how or whether the different signals comprised in the acceleration data (ACD.sub.L, ACD.sub.R), e.g. acceleration data from each of the at least two directions (e.g. from three directions) are correlated. The acceleration data (of each of the left and right hearing instruments) may comprise acceleration data representing three (e.g. orthogonal) directions (e.g. x, y, z).

[0182] FIG. 5 shows a second embodiment of a binaural hearing aid system according to the present disclosure. The embodiment of a binaural hearing aid system illustrated in FIG. 5 is similar to the embodiment of FIG. 4, apart from the differences discussed in the following. In FIG. 5, each of the input units (IU.sub.L; IU.sub.R¤ in FIG. 4) of the left and right hearing instruments (HA.sub.L; HA.sub.R) comprises two microphones (IN1.sub.L, IN2.sub.L and IN1.sub.R, IN2.sub.R, respectively) each microphone providing an electric input signal representing sound in the environment at the location of the microphone in question. Each of the left and right hearing instruments (HA.sub.L; HA.sub.R) further comprises beamformer (BF.sub.L; BF.sub.R) configured to provide respective spatially filtered signals (YBF.sub.L; YBF.sub.R) in dependence of the respective input signals (IN1.sub.L, IN2.sub.L and IN1.sub.R, IN2.sub.R, respectively) and fixed or adaptively determined beamformer weights. The beamformer weights of the beamformers (BF.sub.L; BF.sub.R) of the left and right hearing instruments (HA.sub.L; HA.sub.R) may be controlled by control signals (CT.sub.BT,L; CT.sub.BF,R) provided by the respective control units (CTR.sub.L; CTR.sub.R) in dependence of acceleration data (ACD.sub.L, ACD′.sub.R; ACD′.sub.L, ACD.sub.R) from the acceleration sensors of left and right hearing instruments. In the embodiment of FIG. 5, both of the left and right hearing instruments (HA.sub.L; HA.sub.R) are incorrectly mounted at the left and right ears (cf. ‘Left ear’; Right ear′ in FIG. 5). This is indicated in FIG. 5 by the askew, dotted arrow through the microphones (FM.sub.L, RM.sub.L and FM.sub.R, RM.sub.R). In the embodiment of FIG. 5, the signal processor (SP.sub.L; SP.sub.R) of the forward audio signal path of the respective left and right hearing instruments (HA.sub.L; HA.sub.R) may comprise the beamformer (BF.sub.L; BF.sub.R) and a hearing aid processor (HA-Pro.sub.L; HA-Pro.sub.R) for applying a level and frequency dependent gain to the spatially filtered signal ((YBF.sub.L; YBF.sub.R) or to another signal of the respective forward audio paths) to compensate for the user's hearing impairment. The control units (CTR.sub.L; CTR.sub.R) may influence (control) processing of the respective hearing aid processors (HA-Pro.sub.L; HA-Pro.sub.R), cf. control signals (CT.sub.PRO,L; CT.sub.PRO,R) from the control units to the hearing aid processors. In the embodiment of FIG. 5, the output units (OU.sub.L; OU.sub.R¤ in FIG. 4) of left and right hearing instruments (HA.sub.L; HA.sub.R) comprise respective output transducers (OT.sub.L; OT.sub.R), e.g. in the form of loudspeakers. The output transducers may alternatively comprise respective vibrators of a bone conducting hearing aid.

[0183] The control units (CTR.sub.L; CTR.sub.R) of the left and right hearing instruments (HA.sub.L; HA.sub.R) may be configured to decide whether or not the hearing instruments are correctly mounted in dependence of (e.g. a comparison of) values of elements of the cross-covariance matrix R.sub.LR, e.g. as described in connection with FIG. 6.

[0184] FIG. 6A shows an example of cross-covariances of acceleration data between a left and a right correctly mounted hearing instrument of a binaural hearing aid system according to the present disclosure.

[0185] FIG. 6B shows an example of cross-covariances of acceleration data between a left and a right in-correctly mounted hearing instrument of a binaural hearing aid system according to the present disclosure.

[0186] In the two cases of FIG. 6A, 6B, the cross-covariance matrix R.sub.LR is estimated as the outer (vector) product between the left acceleration vector (a.sub.L=[x.sub.L, y.sub.L, z.sub.L]) and the right acceleration vector (a.sub.R=[x.sub.R, y.sub.R, z.sub.R]) (averaged over time), both vectors being real-valued:

[00005] $R_{L R} = < a_{L} .Math. a_{R} > = < [\begin{matrix} x_{L} x_{R} & y_{L} x_{R} & z_{L} x_{R} \\ x_{L} y_{R} & y_{L} y_{R} & z_{L} y_{R} \\ x_{L} z_{R} & y_{L} z_{R} & z_{L} z_{R} \end{matrix}] >$

[0187] where <⋅> denotes averaging over time (or the ‘expectation operator’). The acceleration vector(s) may be provided in the frequency domain (e.g. using a filter bank). Also, more general than just removing the mean, the acceleration data may be high-pass filtered (i.e. same as removing the mean) or band-pass filtered or low-pass filtered.

[0188] For simplicity, a.sub.L and a.sub.R may be processed versions of the raw acceleration vectors.

[0189] There may be two different mean values of the acceleration data: a) the DC-mean indicates a very slow varying mean value, whereas b) the mean value applied to movement (e.g. of the body) is based on fewer samples. The two different mean values may be implemented in terms of IIR lowpass filtering having two different time constants. Hereby the resulting signal becomes a band-pass filtered signal. The low-pass filter may be based on a moving average, i.e. a finite number of samples.

[0190] The figure shows the estimated cross covariance matrix (normalized over time by the number of time frames) in the case of correctly mounted hearing instruments (FIG. 6A) and the case of incorrectly mounted hearing instruments (FIG. 6B). The mean (caused by gravity) has been subtracted from the data (e.g. by providing R.sub.LR=E[(a.sub.L−E[a.sub.L]) (a.sub.R−E[a.sub.R]).sup.T]), but in addition, the difference between the direction of the left and right gravity vectors may also be used as an indication of both instruments having been correctly mounted. If the angle between the vectors deviates more than a threshold value, it may be assumed that the hearing instruments are incorrectly mounted. In the case where we would like to ‘measure’ the angle(s), the angle(s) between the mean acceleration vectors E[a.sub.L] and E[a.sub.R] may be determined (and used as an estimate).

[0191] In the case of the correctly mounted instruments, we notice that the magnitudes of the diagonal elements are higher compared to the off-diagonal elements. In the opposite case (incorrectly mounted instruments), we notice that the magnitudes of the off-diagonal elements are higher compared to the diagonal elements.

[0192] A criterion for detection of correctly mounted instruments may thus be estimated from the ratio between the diagonal elements and the off-diagonal elements (D-OD-R) of the cross-covariance matrix R,

[00006] $D - OD - R = \frac{.Math. .Math. diagonal elements of R .Math.}{.Math. .Math. off - diagonal elements of R .Math.}$

[0193] where |⋅| represents the magnitude (or squared magnitude) of individual elements of the cross-covariance matrix R.

[0194] For the 3D acceleration sensor, e.g., the ratio (D-OD-R) may be determined as:

[00007] $D - OD - R = < \frac{.Math. x_{L}^{*} x_{R} .Math. + .Math. y_{L}^{*} y_{R} .Math. + .Math. z_{L}^{*} z_{R} .Math.}{.Math. x_{L}^{*} y_{R} .Math. + .Math. x_{L}^{*} z_{R} .Math. + .Math. y_{L}^{*} z_{R} .Math. + .Math. y_{L}^{*} x_{R} .Math. + .Math. z_{L}^{*} x_{R} .Math. + .Math. z_{L}^{*} z_{R} .Math.} >$

[0195] where <⋅> denotes averaging over time. The ratio may as well be calculated as a difference (Log[SUM(diagonal)]−log[SUM(off-diagonal)]) in the logarithmic domain.

[0196] In the case of the two cross-covariance matrices shown in the FIG. 6A, 6B, the ratio in the correctly mounted case becomes 2.85, and the ratio in the off-diagonal case becomes 0.16.

[0197] A criterion for detection of correctly mounted instruments may thus be [0198] Hearing instruments are correctly mounted if D-OD-R≥TH1, and [0199] Hearing instruments are in-correctly mounted if D-OD-R<TH2.

[0200] TH1 may be larger than or equal to TH2. TH1 may e.g. be larger than or equal to 1, such as larger than or equal to 2, e.g. larger than or equal to 2.5. TH2 may e.g. be smaller than or equal to 1.5, such as smaller than or equal to 1, e.g. smaller than or equal to 0.5.

[0201] In an embodiment, the (cross-) covariance matrix is only updated, if a movement is detected (i.e. movement above a certain threshold, e.g. due to head movements, walking, etc.).

[0202] Another way to detect if the instruments are not correctly aligned could be to simply subtract the left and the right (possibly filtered) acceleration patterns from each other (as experienced by left and right hearing instruments of a binaural hearing system). This, however, would not detect if the misalignment is due to head movements. On the other hand. If the instruments are correctly mounted, a difference between the measured acceleration patterns may be used to detect head movements.

[0203] In the case of stereo audio playback (e.g. in a binaural headset), the detection of correctly mounted instruments may be used to determine if the playback should be either binaural stereo (when correctly mounted) or two mono streams (when incorrectly mounted, e.g. if the headphones are shared between two people).

[0204] An estimate of a current feedback path may provide an indication of whether or not, the two hearing instruments are correctly mounted. Alternatively, or additionally, an estimate of the direction of gravity relative to a housing of the hearing aid may provide such indication, assuming that the user wearing the hearing instruments is in an upright position. Although this is the common situation (when the hearing instruments are active), it may not always be the case.

[0205] In case own voice may is impinging on the microphones of the hearing instrument(s) from an unexpected direction, it may be an indication that the hearing instrument(s) is/are in-correctly mounted. If it is detected that the instrument(s) is/are not correctly mounted, we would select the correct instrument for own voice pickup (unless the user has taken the instrument off on purpose in order to talk right into the microphone?).

[0206] Prior knowledge on which instrument is left and right may be accessible to the hearing aid system (e.g. the controller), In that case deviations from normal acceleration patterns may be identified in the acceleration data in order to determine which instrument is incorrectly mounted.

[0207] E.g. head movements will cause opposite sign on the acceleration axes pointing away from the center of the head. If a head movement pattern is detected on one of the instruments, and not on the opposite instrument, it may indicate the incorrectly mounted instrument. The instrument which deviates from the expected acceleration pattern is then incorrectly mounted.

[0208] This may in particular be used to verify if a bone conduction instrument is not correctly mounted on its screw, e.g. if it is turned compared to its optimal placement.

[0209] Movement patterns (represented by acceleration data) above a certain magnitude (threshold) may be used to provide an indication of whether or not, the two hearing instruments are correctly mounted.

[0210] If we only have little movement on both ears (below a threshold), we may accidentally detect that an instrument is incorrectly mounted.

[0211] Only if both instruments have little movement (below a threshold), we do not update the detector. The system may e.g. be configured to only update the cross-covariance matrix in case of a sufficiently high amount of movement (above a threshold).

[0212] In the case where one instrument is mounted but the other is not, we may have little movement on one of the instruments, but movement on the other instrument. In that case all values of the cross-covariance matrix will be small. This is an indication that one instrument has fallen off (or the user is tapping the other instrument).

[0213] FIG. 7A shows a controller comprising a neural network according to a first embodiment of the present disclosure, wherein the input feature vector (IFV) of the neural network (NN) comprises stacked acceleration inputs (x, y, z) as function of the last N+1 samples.

[0214] FIG. 7B shows a controller comprising a neural network according to a second embodiment of the present disclosure, wherein the neural network (NN) is of the feed forward—(FF) recurrent—(RNN), or convolutive-type, or a combination thereof. Each of the inputs to the neural network comprises a band-pass filtered version of the acceleration inputs (x, y, z). The elements of the input feature vector may contain one or more bandpass-filtered signals (e.g. implemented by an analysis filter bank (FB)).

[0215] FIG. 7C shows a controller comprising a neural network according to a third embodiment of the present disclosure, wherein each of the inputs to the neural network (NN) comprises a low-pass filtered (LP) and (possibly down-sampled (DS)) version of the acceleration inputs (x, y, z).

[0216] FIG. 7D shows a controller comprising a neural network according to a fourth embodiment of the present disclosure with a binaural input, wherein the neural network is a recurrent neural network (RNN), e.g. a gated recurrent unit (GRU), cf. e.g. EPEP4033784A1.

[0217] FIG. 7E shows a controller comprising a neural network according to a fifth embodiment of the present disclosure as in FIG. 7A but in a binaural version. The neural network (NN) may e.g. be a feed-forward neural network or a convolutive neural network.

[0218] FIG. 7F shows a controller comprising a neural network according to a sixth embodiment of the present disclosure, with a binaural input, wherein input feature vector (IFV) to the neural network (NN) is a similarity measure.

[0219] FIG. 7A-7F schematically illustrates different input feature vectors (IFV) for the neural network (NN; RNN) implementing a controller (CTR.sub.L, CTR.sub.R) configured to detect whether or not the left and/or right hearing instruments are correctly mounted in dependence of a similarity measure (FIG. 7F), e.g. a correlation measure, between acceleration data provided by the left and right hearing instruments, or based on acceleration data directly (FIG. 7A, 7D, 7E), or based on otherwise processed versions of the acceleration data (FIG. 7B, FIG. 7C) provided by the left and/or right hearing instruments.

[0220] The input data (e.g. an input feature vector) of the neural network may be constituted by or comprise data for a given time instance (n, e.g. ‘now’), cf. e.g. FIG. 7B, 7D, 7F. The input data may e.g. be constituted by or comprise data for a the given time instance (n) and a number (N) of previous time instances, cf. e.g. FIG. 7A, 7E. The latter may be advantageous depending on the type of neural network used (in particular for feed forward-type or convolutional-type neural networks). The ‘history’ of the data represented by the (N previous time instances may be included in the neural network, e.g. in a recurrent-type neural network, e.g. comprising a GRU, cf. e.g. FIG. 7D (where the neural network is denoted ‘RNN’). Alternatively, the (time-) history of the data may be included by low-pass filtering the data (and/or down sampling) before entering the neural network (i.e. the input feature vector—at a given time instance (n)—may comprise lowpass-filtered (and/or down-sampled) acceleration data from x-, y- and z-directions of accelerometers at one or both sides of the ear of the user, cf. e.g. FIG. 7C (where blocks LP indicate low-pass filtering, and blocks DS indicate down-sampling). Thereby the number of computations performed by the neural network can be decreased. The input data may e.g. be time domain signals (e.g. time domain values of acceleration sensors, e.g. provided at successive discrete time instances ( . . . , n−1, n, n+1 . . . )) The input data my, however, be transformed to a transform domain before being input to the neural network. The transform domain may e.g. be the frequency domain, cf. e.g. FIG. 7B, where each of the data inputs (x, y, z) comprise an analysis filter bank (FB). The input data (x, y, z) may e.g. represent current accelerometer data for x-, y- and z-directions of a given hearing device (e.g. relative to a direction of the force of gravity of the earth).

[0221] The output of the neural network is denoted ‘Decision’ in the exemplary configurations of FIG. 7A-7F. The output may e.g. comprise a similarity measure based on input data from both sides of the head of the user (e.g. from accelerometer data from both hearing instruments of a binaural hearing aid system). The output of the neural network may e.g. comprise an indicator of whether or not the hearing instrument at a given ear is correctly mounted (e.g. from accelerometer data from one or both hearing instruments of a binaural hearing aid system). The output of the neural network may e.g. comprise an indicator of whether or not a given one of the hearing instruments is correctly mounted (e.g. from accelerometer data from both hearing instruments of a binaural hearing aid system).

[0222] Instead of labelling the output of the neural network ‘decision’, it might as well have been labeled “classification”, which eventually leads to a decision.

[0223] It is intended that the structural features of the devices described above, either in the detailed description and/or in the claims, may be combined with steps of the method, when appropriately substituted by a corresponding process.

[0224] As used, the singular forms “a,” “an,” and “the” are intended to include the plural forms as well (i.e. to have the meaning “at least one”), unless expressly stated otherwise. It will be further understood that the terms “includes,” “comprises,” “including,” and/or “comprising,” when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. It will also be understood that when an element is referred to as being “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, but an intervening element may also be present, unless expressly stated otherwise. Furthermore, “connected” or “coupled” as used herein may include wirelessly connected or coupled. As used herein, the term “and/or” includes any and all combinations of one or more of the associated listed items. The steps of any disclosed method are not limited to the exact order stated herein, unless expressly stated otherwise.

[0225] It should be appreciated that reference throughout this specification to “one embodiment” or “an embodiment” or “an aspect” or features included as “may” means that a particular feature, structure or characteristic described in connection with the embodiment is included in at least one embodiment of the disclosure. Furthermore, the particular features, structures or characteristics may be combined as suitable in one or more embodiments of the disclosure. The previous description is provided to enable any person skilled in the art to practice the various aspects described herein. Various modifications to these aspects will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other aspects.

[0226] The claims are not intended to be limited to the aspects shown herein but are to be accorded the full scope consistent with the language of the claims, wherein reference to an element in the singular is not intended to mean “one and only one” unless specifically so stated, but rather “one or more.” Unless specifically stated otherwise, the term “some” refers to one or more.

REFERENCES

[0227] US2013188796A1 (Oticon) 25 Jul. 2013 [0228] EP3370435A1 (Oticon) 5 Sep. 2018 [0229] EPEP4033784A1 (Oticon) 27 Jul. 2022

A METHOD FOR MONITORING AND DETECTING IF HEARING INSTRUMENTS ARE CORRECTLY MOUNTED

Assignee

Inventors

Cpc classification

Classification Explorer

H04R2225/61

ELECTRICITY

Classification Explorer

H04R25/30

ELECTRICITY

Classification Explorer

G01P15/18

PHYSICS

Classification Explorer

H04R1/1041

ELECTRICITY

Classification Explorer

H04R2225/021

ELECTRICITY

Classification Explorer

H04R25/552

ELECTRICITY

Classification Explorer

H04R1/1083

ELECTRICITY

Classification Explorer

H04R25/305

ELECTRICITY

Classification Explorer

H04R2460/07

ELECTRICITY

International classification

Classification Explorer

H04R25/00

ELECTRICITY

Classification Explorer

G01P15/18

PHYSICS

Abstract

Claims

Description