Method for the environment-dependent operation of a hearing system and hearing system

Abstract

In a method for the environment-dependent operation of a hearing system values for a first plurality of environmental data of a first user of the hearing system are determined each time in a training phase for survey times, and the values of the environmental data for each of the survey times are used to form respectively a feature vector in an at least four-dimensional feature space. Each of the feature vectors is mapped respectively onto a corresponding representative vector in a maximum three-dimensional representation space, and a spatial distribution of a subgroup of representative vectors is used to define a first region in the representation space for a first environmental situation of the hearing system. A value of a setting for signal processing of the hearing system is specified for the first environmental situation, and the hearing system is operated with the value set in this way.

Claims

1. A method for an environment-dependent operation of a hearing system (1), which comprises the steps of: performing a training phase, which comprises the substeps of: determining values for a first plurality of environmental data of a first user of the hearing system each time for a plurality of survey times; using the values of the environmental data for each of the survey times to form respectively a feature vector in an at least four-dimensional feature space; mapping each of the feature vectors respectively onto a corresponding representative vector in a maximum three-dimensional representation space; using a spatial distribution of a subgroup of representative vectors to define a first region in the maximum three-dimensional representation space for a first environmental situation of the hearing system; specifying at least one value of a setting for a signal processing of the hearing system for the first environmental situation; performing an application phase, which comprises the substeps of: determining at an application time values for the first plurality of environmental data of the first user or of a second user of the hearing system in the application phase; using the values of the environmental data to form a corresponding feature vector for the application time; using the first region of the maximum three-dimensional representation space and the feature vector for the application time to identify a presence of the first environmental situation, and the at least one value of the signal processing of the hearing system is set according to its specification for the first environmental situation; and operating the hearing system with the at least one value set in this way.

2. The method according to claim 1, wherein: in the training phase using a user input to save information on a current usage situation of the hearing system; and the information on the current usage situation is combined with the feature vectors and/or corresponding representative vectors which are formed with an aid of the values of the environmental data collected during a particular user situation.

3. The method according to claim 2, which further comprises: determining acoustical environmental data for the first plurality of environmental data with an aid of a signal of at least one electroacoustical input transducer and/or determining motion-related environmental data with an aid of at least one signal of an acceleration sensor and/or a gyroscope, and/or determining location-related environmental data with an aid of at least one signal of a global positioning system (GPS) sensor and/or a wireless local area network connection, and/or determining biometric environmental data with an aid of an electrocardiogram (ECG) sensor and/or an electroencephalograghy (EEG) sensor and/or a photoplethysmogram (PPG) sensor and/or an electromyography (EMG) sensor.

4. The method according to claim 3, wherein for the acoustic environmental data there is analyzed the signal of the at least one electroacoustic input transducer: in regard to speech activity of the first or second user of the hearing system; and/or in regard to an occurrence of wind at the electroacoustic input transducer; and/or in regard to a spectral centroid of a noise background; and/or in regard to a noise background in at least one frequency band; and/or in regard to a stationarity of a sound signal of the environment; and/or in regard to an autocorrelation function; and/or in regard to a modulation depth for a given modulation frequency, which is at most 10 Hz; and/or in regard to the commencement of the speech activity.

5. The method according to claim 3, wherein there are determined each time as the values of the environmental data for a survey time of the survey times and/or the application time a mean value and/or a variance and/or a mean crossing rate and/or a range of values and/or a median of the environmental data.

6. The method according to claim 3, wherein: the acoustic environmental data are used to form respectively individual vector projections of the feature vectors of the survey times in an acoustic feature space; the individual vector projections of the acoustic feature space are respectively mapped onto acoustic representative vectors in a maximum three-dimensional acoustic representative space; a second region is defined in the maximum three-dimensional acoustic representation space for the first environmental situation of the hearing system; and a presence of the first environmental situation is identified, in addition, with the aid of the second region of the maximum three-dimensional acoustic representation space.

7. The method according to claim 3, wherein: the first environmental situation is defined in addition with an aid of a first usage situation, and for the first environmental situation a first value of the setting for the signal processing of the hearing system is specified; a second environmental situation is defined with an aid of a second usage situation, and a corresponding second value of the setting is specified; and a presence of the first or the second environmental situation is identified with an aid of a presence of the first or second usage situation, and thereupon the first or second value of the signal processing of the hearing system is set, corresponding to its specification for the first or second environmental situation.

8. The method according to claim 3, wherein there are determined each time as the values of the environmental data for a survey time of the survey times and/or the application time a mean value and/or a variance and/or a mean crossing rate and/or a range of values and/or a median of the environmental data, namely in relation to a period of time between a respective survey time and an immediately preceding survey time or in relation to a period of time between the application time and an immediately preceding application time.

9. The method according to claim 2, wherein the step of using the user input to save the information on the current usage situation of the hearing system is performed in dependence on a defined situation of a daily routine of the first user of the hearing system.

10. The method according to claim 1, wherein: at least one partial area of the maximum three-dimensional representation space is visualized; at least one subset of corresponding representative vectors is displayed; and the first region in the maximum three-dimensional representation space is defined with an aid of a user input.

11. The method according to claim 10, wherein: the at least one partial area of the maximum three-dimensional representation space is visualized by means of a monitor screen; and the first region in the maximum three-dimensional representation space is defined with the aid of the user input in regard to a grouping of visualized representative vectors.

12. The method according to claim 1, wherein at least in the training phase the mapping of feature vectors onto corresponding representative vectors is done in such a way that distance relations of at least three the feature vectors in the at least four-dimensional feature space remain at least approximately preserved as a result of the mapping for distance relations of the three corresponding representative vectors in the maximum three-dimensional representation space.

13. The method according to claim 1, wherein in the application phase a presence of the first environmental situation is identified by mapping the feature vector for the application time in the maximum three-dimensional representation space, and a position of a resulting formed representative vector relative to the first region is evaluated.

14. The method according to claim 13, wherein the representative vector is identified as lying within the first region.

15. The method according to claim 1, wherein in the application phase a presence of the first environmental situation is identified with an aid of the feature vector for the application time and with an aid of at least some feature vectors in the at least four dimensional feature space that are mapped in the maximum three-dimensional representation space onto the representative vectors of the first region.

16. The method according to claim 1, wherein: in the application phase the values for the first plurality of environmental data in each case are determined for a plurality of successive application times and the values of the environmental data are used to form corresponding feature vectors for the successive application times; and the presence of the first environmental situation is identified with an aid of the first region and with an aid of the corresponding feature vectors for the successive application times.

17. The method according to claim 16, wherein a presence of the first environmental situation is identified with the aid of the first region, with the aid of the corresponding feature vectors for the successive application times, an aid of a polygon course of the feature vectors or a polygon course of representative vectors corresponding to the feature vectors in the maximum three-dimensional representation space.

18. The method according to claim 1, wherein: a definition of the first region for the first environmental situation is done in the training phase by the first user of the hearing system with a hearing device and is saved in a cloud server; and for the application phase the definition is downloaded by the second user of the hearing system comparable for the application from the cloud server to the hearing system.

19. The method according to claim 1, wherein: the at least four-dimensional feature space is an at least six-dimensional feature space; the maximum three-dimensional representation space is a two-dimensional representation space; and the at least one value of the signal processing of the hearing system is set according to its specification for the first environmental situation in an automatic manner.

20. A hearing system, comprising: a device selected from the group consisting of a hearing device, a hearing aid, a hearing assist device and a headphone; an auxiliary device with a processor; and the hearing system is adapted to perform a method according to claim 1.

Description

BRIEF DESCRIPTION OF THE DRAWING

(1) The single FIGURE of the drawing is a block diagram showing a method for an environment-dependent operation of a hearing system

DETAILED DESCRIPTION OF THE INVENTION

(2) Referring now to the sole FIGURE of the drawing, there is shown schematically in a block diagram a method for the environment-dependent operation of a hearing system 1, where the hearing system in the present instance is formed by a hearing device 3, configured as a hearing aid 2, as well as an auxiliary device 5, configured as a smartphone 4. The hearing device 3 contains at least one electro-acoustic input transducer 6, which in the present instance is configured as a microphone and which produces an audio signal 7 from an environmental sound. Furthermore, the hearing device 3 contains other sensors 8, generating additional sensor signals 9. The sensors 8 may comprise, e.g., an acceleration sensor or also a temperature sensor.

(3) In a training phase 10 of the method, the audio signal 7 and the sensor signal 9 are used to determine environmental data each time for a plurality of survey times T1, T2, T3. This is done in the present case by first generating the acoustic environmental data 12 in ongoing manner from the audio signal 7. The acoustic environmental data 12 contains here: a 4 Hz modulation; an onset mean; an autocorrelation function; a level for low and medium frequencies of a noise background, as well as a centroid of the noise background; a stationarity; a wind activity; a broadband maximum level; one's own voice activity. Likewise, motion-related environmental data 14 is generated in ongoing manner from the sensor signal 9, which contains the measured instantaneous accelerations in the three directions of space.

(4) Further kinds of acoustic environmental data 12 and/or motion-related environmental data 14 or other, especially location-related and/or biometric environmental data can generally be included as environmental data 15, such as magnetic field sensors, other cell phone and/or smartwatch sensors, a gyroscope, a pulse metering, a PPG measurement (photoplethysmogram), an electrocardiogram (ECG), a detection of stress through the measurement of the heart rate and its variation, a photosensor, a barometer, a listening effort or a listening activity (such as one through “auditory attention” by means of an EEG measurement), a measurement of eye or head motions through muscle activity (EMG), location information via GPS, WLAN information, geo-fencing or Bluetooth beacons for the current location or area.

(5) For the acoustic environmental data 12 (in the present case, ten different kinds of data) and the three (in the present case) motion-related environmental data 14, each time a buffering 16 is performed for the period between two survey times T1, T2, T3 (the mentioned signals are buffered from a start time T0 for a recording at the survey time T1). Then, for each individual kind of the acoustic environmental data 12 and the motion-related environmental data 14 there is formed a mean value Mn, a variance Var and a mean crossing rate MCR. The mentioned statistical quantities Mn, Var, MCR of the individual acoustic environmental data 12 and the motion-related environmental data 14 during the buffered time between two survey times T1, T2, T3 form respective environmental features 16 for the survey time T1, T2, T3 at the end of the buffering period, and are mapped each time onto a high-dimensional feature vector M1, M2, M3 in a high-dimensional feature space 18. The high dimensionality, such as 39D for respectively three statistical features from ten acoustic and three motion-related environmental data points, is only indicated here by the number of axes on the diagrams of the feature space 18 for the individual feature vectors M1, M2, M3.

(6) Each of the feature vectors M1, M2, M3 is now mapped from the feature space 18 onto a corresponding representative vector R1, R2, R3 in a two-dimensional representation space 20. The mapping is done here for example by means of a t-SNE optimization method (t-distributed stochastic neighbor embedding).

(7) In the following, the optimization method will be briefly described (see, e.g., “Visualizing Data using t-SNE”, 2008, Laurens van der Maaten and Geoffrey Hinton).

(8) A so-called perplexity parameter defines a number of effective neighbors of the feature vectors, i.e., the perplexity parameter determines how many neighbors have influence on the final position of the corresponding representative vector in the two-dimensional representation space 20 (this parameter in the present instance can be set, e.g., at a value of 50 or on the order of 1/100 of the number of feature vectors). Thereafter, for all pairs of high-dimensional feature vectors the degrees of probability are calculated one time, that two particular feature vectors are to be identified as closest neighbors in the high-dimensional feature space. This mirrors a starting situation.

(9) For the two-dimensional representation space, randomly Gauß-distributed random values Y are assumed as the start value. Thereafter, the current similarity relations in Y are calculated in individual iterations. For the optimization of the mapping of the similarity relations, a similarity is now determined between the feature space and the representation space with the aid of a Kullback-Leibler divergence. Using the gradient of the divergence, the representative vectors (or their end points) are shifted along in the representation space for T iterations.

(10) One possible representation of the algorithm is: feature space of the high-dimensional feature vectors X={x.sub.1; x.sub.2; . . . ; x.sub.n} with n being the number of all feature vectors present (in the present case, e.g., n=4016); cost function parameter: “perplexity” Perp: determines the number of effective neighbors, by choice of the variance σ.sub.i for each point by a binary search (strong influence on Y); optimization parameter: determination of a number of iterations t of T (e.g., 500), a learning rate h (e.g., 1000), and a momentum a(t) (e.g., 0.5 for t<250, otherwise a(t)=0.8); and result: two-dimensional representation space Y={y.sub.1; y.sub.2; . . . ; y.sub.n}

(11) start of method: calculate the degree of probability for all feature vector pairs μ.sub.ij in the high-dimensional space:

(12) $p_{j|i}=\frac{{\tilde{p}}_{j|i}}{{\Sigma }_{k\ne i}{\tilde{p}}_{k|i}}\mathrm{with}{\tilde{p}}_{j|i}=\exp \left(-{.Math.x_i-x_j.Math.}^2/2{\sigma }_i\right)$$\mathrm{set}{p}_{\mathrm{ij}}=\frac{p_{j|i}+p_{i|j}}{2n}$ “random drawing” of n two-dimensional Gauß-distributed random numbers for the initialization of Y; optimizing of the r mapping in the representation space: counting loop of the optimization for t=1 to T: Calculate the current degree of probability in the two-dimensional space:

(13) $q_{\mathrm{ij}}=\frac{{\left(1+{.Math.y_i-y_j.Math.}^2\right)}^{-1}}{{{\Sigma }_{k\ne l}\left(1+{.Math.y_k-y_l.Math.}^2\right)}^{-1}}$ measure the similarity between X and Y (Kullback-Leibler divergence)

(14) $C=\underset{j}{.Math.}\underset{i}{.Math.}{p}_{\mathrm{ij}}\log \left(\frac{p_{\mathrm{ij}}}{q_{\mathrm{ij}}}\right)$ calculate the gradient:

(15) $\frac{\partial c}{\partial y_i}=4{.Math.}_j\left(p_{\mathrm{ij}}-q_{\mathrm{ij}}\right)\left(y_i-y_j\right){\left(1+{.Math.y_i-y_j.Math.}^2\right)}^{-1}$ shift the two-dimensional representative vectors:

(16) $y_i^{\left(t\right)}=y_i^{\left(t-1\right)}+h\frac{\partial c}{\partial y_i}+a\left(y_i^{\left(t-1\right)}-y_i^{\left(t-2\right)}\right)$ end of optimization end of method

(17) In terms of the present method, the representative vectors R1, R2, R3 in the two-dimensional representation space 20 are thus generated by the above described mapping procedure from the feature vectors M1, M2, M3 of the feature space 18.

(18) A user of the hearing system 1 can now have the representation space 20 displayed on his auxiliary device 5 (on the monitor screen 21 of the smartphone 4), and define a cohesive area 22 as a first region 24 corresponding to a specific first environmental situation 25 in his use of the hearing system 1. The user can now match up the first region 24 with a specific setting 26 of a signal processing of the audio signal 7 in the hearing device 3, for example, frequency band-related amplification and/or compression values and parameters, or control parameters of a noise suppression and the like. With the matching up of the setting 26 of the signal processing and the first region 24 (and thus the present first environmental situation 25, as characterized by the values of the environmental data 15 in the individual feature vectors M1, M2, M3), the training phase 10 for a particular environmental situation may be considered as being finished. Preferably, multiple training phases 10 will be done for different environmental situations.

(19) In an application phase 30, now, the same environmental data 15 is gathered as in the training phase from the audio signal 7 of the hearing device 3 and from the sensor signal 9 for an application time T4, and a feature vector M4 in the high-dimensional feature space 18 is formed from it in corresponding manner, using the values determined for the application time T4 in the same way. The values here may be formed for example from the mean value Mn, the variance Var and the mean crossing rate MCR of the acoustic and motion-related data 12, 14 gathered during a short time (such as 60 seconds or the like) prior to the application time T4.

(20) The feature vector M4 for the application time T4 is now mapped onto a representative vector R4 in the representation space 20.

(21) Since the t-SNE method used in the training phase 10 of the present example for the mapping of the feature vectors M1, M2, M3 of the feature space 18 onto the representative vectors R1, R2, R3 in the representation space 20 is an optimization method requiring knowledge of all feature vectors used, a corresponding mapping in the application phase 30 is done by means of an approximation mapping (e.g., a so-called “out-of-sample extension”, 00S kernel). This may be done by a regression, by means of which a mapping with the aid of a plurality of feature vectors of the feature space 18 (such as 80% of the feature vectors) onto corresponding representative vectors of the representation space 20 is “learned”, and remaining feature vectors (i.e., in this case, 20%) are used to “test” the quality of the resulting mapping. With the mapping of the “learning vectors”, i.e., the feature vectors used to learn the mapping, onto corresponding representative vectors, a kernel function can then be determined, which preserves local distance relations between said feature and representative vectors in their respective spaces (feature and representation space). In this way, a new, unknown feature vector can be mapped from the feature space 18 onto a corresponding representative vector in the representation space 20, by preserving the local distance relations between the known “learning vectors”.

(22) A more detailed explanation will be found, e.g., in “Out-of-Sample Kernel and Extensions for Nonparametric Dimensionality Reduction”, Andrej Gisbrecht, Wouter Lueks, Bassam Mokbel and Barbara Hammer, ESANN 2012 proceedings, European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning, Bruges (Belgium), 25-27 Apr. 2012, as well as “Parametric nonlinear dimensionality reduction using kernel t-SNE”, Andrej Gisbrecht, Alexander Schulz and Barbara Hammer, Neurocomputing, Vol. 147, 71-82, January 2015.

(23) Now, if the representative vector R4 determined as described for the application time T4 lies in the first region 24, it will be recognized that the first environmental situation 25 is present for the hearing system 1, and, accordingly, the hearing device 3 will be operated with the settings 26 for the signal processing of the audio signal 26, and the previously defined amplification and/or compression values and parameters, or control parameters of a noise suppression, will be applied to the audio signal 7.

(24) Although the invention has been described and illustrated in detail by the preferred exemplary embodiment, the invention is not limited by this exemplary embodiment. Other variations may be deduced from it by the person skilled in the art, without leaving the scope of protection of the invention.

(25) The following is a summary list of reference numerals and the corresponding structure used in the above description of the invention: 1 Hearing system 2 Hearing aid 3 Hearing device 4 Smartphone 5 Auxiliary device 6 Input transducer 7 Audio signal 8 Sensor 9 Sensor signal 10 Training phase 12 Acoustic environmental data 14 Motion-related environmental data 16 Buffering 18 Feature space 20 Representation space 21 Monitor screen 22 Area 24 First region 25 First environmental situation 26 Setting (of a signal processing) 30 Application phase M1, M2, M3 Feature vector (in the training phase) M4 Feature vector (in the application phase) MCR Mean crossing rate Mn Mean value R1, R2, R3 Representative vector (in the training phase) R4 Representative vector (in the application phase) T0 Start time T1, T2, T3 Survey time T4 Application time Var Variance