HEARING DEVICE COMPRISING A FEEDBACK CONTROL SYSTEM
20230044509 · 2023-02-09
Assignee
Inventors
Cpc classification
International classification
Abstract
A hearing aid comprises a) at least one input transducer for providing at least one electric input signal representing said sound; b) an output transducer for providing stimuli perceivable to the user as sound; c) a feedback control system configured to minimize feedback from said output transducer to said at least one input transducer, and to at least provide a feedback corrected version of said at least one electric input signal; and d) an audio signal processor configured to apply one or more processing algorithms to said feedback corrected version of said at least one electric input signal, and to provide a processed signal in dependence thereof. The feedback control system is based on a machine learning model receiving input data at least representing said at least one electric input signal; and said processed signal; and providing said feedback corrected version of the at least one electric input signal as an output. A method of training a machine learning model is further disclosed.
Claims
1. A hearing aid adapted for being worn by a user at or in an ear of the user, the hearing aid comprising at least one input transducer for converting sound in an environment around the user to at least one electric input signal representing said sound; an output transducer for converting an output signal provided in dependence of said least one electric input signal to stimuli perceivable to the user as sound; a feedback control system configured to minimize feedback from said output transducer to said at least one input transducer, and to at least provide a feedback corrected version of said at least one electric input signal; and an audio signal processor configured to apply one or more processing algorithms to said feedback corrected version of said at least one electric input signal, and to provide a processed signal in dependence thereof>; wherein the feedback control system is based on a machine learning model receiving input data at least representing said at least one electric input signal; and said processed signal; and wherein the feedback control system is configured to provide said feedback corrected version of the at least one electric input signal as an output.
2. A hearing aid according to claim 1 wherein said feedback control system is configured to provide said output signal as a further output.
3. A hearing aid according to claim 1 wherein said machine learning model is configured to receive further input data representing information about said one or more processing algorithms.
4. A hearing aid according to claim 1 wherein said feedback control system is configured to provide a control input signal to the audio signal processor as a further output, said control input signal comprising parameters providing inputs to said one or more processing algorithms.
5. A hearing aid according to claim 1 wherein said machine learning model is trained with input data at least representing said at least one electric input signal; and said processed signal.
6. A hearing aid according to claim 5 wherein said machine learning model is trained with further input data representing information about said one or more processing algorithms.
7. A hearing aid according to claim 1 wherein the machine learning model is trained with synthetic input data at least representing said external part of said at least one electric input signal; said feedback part of said at least one electric input signal; and said processed signal; and with synthetic output data at least representing said feedback corrected version of the at least one electric input signal.
8. A hearing aid according to claim 1 wherein said processed signal from the processor provides said output signal.
9. A hearing aid according to claim 1 being constituted by or comprising an air-conduction type hearing aid, a bone-conduction type hearing aid, or a combination thereof.
10. A hearing aid according to claim 1 comprising at least one analysis filter bank for providing said at least one electric input signal in a time-frequency domain representation.
11. A hearing aid according to claim 10 wherein the input data to the machine learning model are said at least one electric input signal; and said processed signal, which for each time index l each are arranged as a vector with K elements, K being the number of frequency bands in the time-frequency domain representation (k,l).
12. A hearing aid according to claim 1 wherein the output transducer comprises a) a loudspeaker for providing said stimuli as an acoustic signal to the user, or b) a vibrator for providing said stimuli as mechanical vibration of a skull bone to the user.
13. A method of training a machine learning model for use in a feedback control system of a hearing aid, the hearing aid comprising at least one input transducer for converting input sound in an environment around the user to at least one electric input signal representing said input sound; an output transducer for converting an output signal provided in dependence of said at least one electric input signal to stimuli perceivable to the user as sound; wherein said input sound comprises an external sound and a feedback sound generated by said output transducer and leaked to said input transducer via feedback path, and wherein said at least one electric input signal likewise comprises an external part originating from said external sound and a feedback part originating from said feedback sound; a feedback control system for minimizing said feedback part of said at least one electric input signal and at least providing a feedback corrected version of said at least one electric input signal, the feedback control system comprising said machine learning model; and an audio signal processor configured to apply one or more processing algorithms to said feedback corrected version of said at least one electric input signal and to provide a processed signal in dependence thereof; wherein the machine learning model is trained with synthetic input data at least representing said external part of said at least one electric input signal; said feedback part of said at least one electric input signal; and said processed signal; and with synthetic output data at least representing said feedback corrected version of the at least one electric input signal.
14. A method according to claim 13 wherein said synthetic output data further represents said output signal.
15. A method according to claim 13 wherein said synthetic input data further represents information about said one or more processing algorithms.
16. A method according to claim 13 wherein said synthetic output data further represents parameters providing inputs to said one or more processing algorithms.
17. A method according to claim 13 wherein at least said synthetic output data are generated by computer simulation.
18. A method according to claim 13 wherein at least said synthetic output data are generated by computer simulation to reflect an imaginary and perfect feedback control system reacting instantly and accurately to feedback changes.
19. A method according to claim 13 wherein an imaginary and perfect feedback control system is used to generate data for the training of the machine learning model, both in static feedback situations and with dynamic feedback path changes.
20. A method according to claim 13 wherein the input signals for the training of the machine learning model comprise white noise, or speech, or music signals, or a mixture thereof.
21. A hearing aid comprising at least one input transducer for converting input sound in an environment around the user to at least one electric input signal representing said input sound; an output transducer for converting an output signal provided in dependence of said at least one electric input signal to stimuli perceivable to the user as sound; wherein said input sound comprises an external sound and a feedback sound generated by said output transducer and leaked to said input transducer via feedback path, and wherein said at least one electric input signal likewise comprises an external part originating from said external sound and a feedback part originating from said feedback sound; a feedback control system for minimizing said feedback part of said at least one electric input signal and at least providing a feedback corrected version of said at least one electric input signal, the feedback control system comprising said machine learning model; and an audio signal processor configured to apply one or more processing algorithms to said feedback corrected version of said at least one electric input signal and to provide a processed signal in dependence thereof; wherein the machine learning model is trained according to the method of claim 13.
Description
BRIEF DESCRIPTION OF DRAWINGS
[0114] The aspects of the disclosure may be best understood from the following detailed description taken in conjunction with the accompanying figures. The figures are schematic and simplified for clarity, and they just show details to improve the understanding of the claims, while other details are left out. Throughout, the same reference numerals are used for identical or corresponding parts. The individual features of each aspect may each be combined with any or all features of the other aspects. These and other aspects, features and/or technical effect will be apparent from and elucidated with reference to the illustrations described hereinafter in which:
[0115]
[0116]
[0117]
[0118]
[0119]
[0120]
[0121]
[0122]
[0123]
[0124]
[0125]
[0126]
[0127] The figures are schematic and simplified for clarity, and they just show details which are essential to the understanding of the disclosure, while other details are left out. Throughout, the same reference signs are used for identical or corresponding parts.
[0128] Further scope of applicability of the present disclosure will become apparent from the detailed description given hereinafter. However, it should be understood that the detailed description and specific examples, while indicating preferred embodiments of the disclosure, are given by way of illustration only. Other embodiments may become apparent to those skilled in the art from the following detailed description.
DETAILED DESCRIPTION OF EMBODIMENTS
[0129] The detailed description set forth below in connection with the appended drawings is intended as a description of various configurations. The detailed description includes specific details for the purpose of providing a thorough understanding of various concepts. However, it will be apparent to those skilled in the art that these concepts may be practiced without these specific details. Several aspects of the apparatus and methods are described by various blocks, functional units, modules, components, circuits, steps, processes, algorithms, etc. (collectively referred to as “elements”). Depending upon particular application, design constraints or other reasons, these elements may be implemented using electronic hardware, computer program, or any combination thereof.
[0130] The electronic hardware may include micro-electronic-mechanical systems (MEMS), integrated circuits (e.g. application specific), microprocessors, microcontrollers, digital signal processors (DSPs), field programmable gate arrays (FPGAs), programmable logic devices (PLDs), gated logic, discrete hardware circuits, printed circuit boards (PCB) (e.g. flexible PCBs), and other suitable hardware configured to perform the various functionality described throughout this disclosure, e.g. sensors, e.g. for sensing and/or registering physical properties of the environment, the device, the user, etc. Computer program shall be construed broadly to mean instructions, instruction sets, code, code segments, program code, programs, subprograms, software modules, applications, software applications, software packages, routines, subroutines, objects, executables, threads of execution, procedures, functions, etc., whether referred to as software, firmware, middleware, microcode, hardware description language, or otherwise.
[0131] The present application relates to the field of hearing devices, e.g. hearing aids, in particular to feedback control in such devices.
[0132] Modern machine learning techniques provide a new tool, which may deliver a completely new generation of feedback control systems, which can solve the biased estimation problem without compromising sound quality, and it can better handle critical feedback situations and hence better ensure that the optimal gain can be provided to the users.
[0133]
[0134] For the feedback control purpose, the processing unit in the forward path typically consists of a decorrelation block, a gain control block, and optionally a fast feedback reduction block. This is illustrated in
[0135]
[0136] In our envisioned future machine learning based feedback control system, we can in principle replace all these blocks with a machine learning block, as shown in
[0137]
[0138]
[0139] As an alternative, it is proposed to train the machine learning based feedback control system with synthetic data v(n), x(n), y(n), e(n), p(n), and u(n), that would only be presented if there were a perfect feedback control system, i.e., there is no need for decorrelation, the feedback system would react to feedback path changes instantly without the need of a convergence period known from the adaptive filters.
[0140] Furthermore, it is proposed to provide more information from the hearing aid processing unit to the machine learning based feedback control system, to gain better performance, as illustrated in
[0141]
[0142] A simpler model is shown in
[0143]
[0144] In this model, although it is possible (indirectly by modifying e(n)), it is not the intention that the model can modify the output signal u(n), as was the case in the previous model.
[0145] More details on how to train the machine learning model (as shown in
[0146] We consider some standard algorithms for the machine learning training, such as the supervised learning method, one specific way of training such a model is backpropagation.
[0147] In this case, we would provide training signals generated from realistic computer simulations. In simulations, we have access to all signals, including the feedback signal v(n) and the incoming signal x(n), which are not observable in practice. We can then create the microphone signals y(n)=x(n)+v(n), and the desired processing signal p(n) depends on the chosen (and known) hearing aid processing, both y(n) and p(n) would be available to use (for us or our machine learning model) during the normal hearing aid operation.
[0148] Furthermore, we can generate the desired feedback compensated signal e(n) (ideally e(n)=x(n)), and the desired output signal u(n) (ideally u(n)=p(n)), both e(n) and u(n) are used as the reference signals (labeled data) for the training.
[0149] We would need to create many sets of the signals v(n), x(n), y(n) and p(n), e(n) and u(n) to train the network, under different conditions (signal type of x(n), dynamics of v(n), the processing of hearing aid to mention the most important ones).
[0150] We expect that the input signals to the machine learning network are the time-frequency units Y(k,l) and P(k,l) after the transformation (e.g., STFT) of the time domain signals y(n), p(n), where l and k are the time-frequency domain time and frequency indices.
[0151] Furthermore, we would have the time-domain signals e(n) and u(n), or their time-frequency transformed signals E(k,l) and U(k,l) as labeled data for our training.
[0152] In one setup, we would arrange Y(k,l′), P(k,l′) as a K-element (column) vector Y.sub.j′ and P.sub.j′ for each time index m, where k=0, . . . , K−1. This is illustrated in
[0153] In another setup, we would concatenate more K-element (column) vectors of Y.sub.j′, P.sub.j′ over several time indices l′ into matrices Y.sub.j′=[Y.sub.l′Y.sub.l′-1 . . . ] and P.sub.l′=[P.sub.l′P.sub.l′-1 . . . ]. This is illustrated in
[0154]
[0155]
[0156]
[0157] The (synthetic) training data preferably comprises a larger number of data sets leading to feedback howl (for a given hearing device, e.g. a specific hearing aid style), wherein the input and output and intermediate signals are known as described above.
[0158]
[0159] Similarly, we can arrange the information from the hearing aid processing (dotted line in
[0160] An output (dotted line in
[0161] Different types of networks may be used to train the machine learning model, such as a dense neural network, convolutional neural network, and recurrent neural network, e.g. a gated recurrent unit (GRU), or combinations thereof.
[0162] Another way of training the network may be to use the reinforcement learning method.
[0163] A method of training for machine learning model (e.g. implemented by a neural network) for use in a feedback control system of a hearing device, e.g. hearing aid, is proposed. This is illustrated in
[0164] The synthetic input data may represent [0165] an external part of an electric input signal provided by an input transducer; [0166] a feedback part of the electric input signal propagated from an output transducer to the input transducer; and [0167] a processed signal provided by processing of a feedback corrected version of the electric input signal; and
[0168] The synthetic output data may represent [0169] the feedback corrected version of the at least one electric input signal.
[0170] The synthetic output data may further represent an output signal provided to the output transducer for being presented to a user.
[0171] The synthetic input data may further represent information about one or more processing algorithms applied to feedback corrected version of the at least one electric input signal.
[0172] The synthetic output data may further represent parameters providing inputs to the one or more processing algorithms.
[0173] The training procedure may involve the use of one or more “loss functions” (or “cost functions”), i.e. functions to be optimized (e g minimized or maximized) during the training of the network. Many such functions could be envisioned, including: [0174] Mean-Squared Error (MSE) between complex STFTs of network compensated signal and ideal signal. [0175] MSEs of transformed STFTs, e.g., log-magnitude-STFTs. [0176] More perceptually oriented loss functions (e.g. speech intelligibility measures, e.g. Short-Time Objective Intelligibility (STOI), Speech Intelligibility Index (SII), Hearing Aid Speech Perception Index (HASPI), etc.).
[0177]
[0178] The hearing aid comprises [0179] at least one input transducer for converting input sound in an environment around the user to at least one electric input signal representing said input sound; [0180] an output transducer for converting an output signal provided in dependence of said at least one electric input signal to stimuli perceivable to the user as sound;
wherein said input sound comprises an external sound and a feedback sound generated by said output transducer and leaked to said input transducer via feedback path, and wherein said at least one electric input signal likewise comprises an external part originating from said external sound and a feedback part originating from said feedback sound; [0181] a feedback control system for minimizing said feedback part of said at least one electric input signal and at least providing a feedback corrected version of said at least one electric input signal; and [0182] an audio signal processor configured to apply one or more processing algorithms to said feedback corrected version of said at least one electric input signal and to provide a processed signal in dependence thereof.
[0183] The method comprises that the machine learning model is trained with synthetic input data at least representing [0184] said external part of said at least one electric input signal; [0185] said feedback part of said at least one electric input signal; and [0186] said processed signal; and
and with synthetic output data at least representing [0187] said feedback corrected version of the at least one electric input signal.
[0188] The synthetic output data may further represent an output signal provided to the output transducer for being presented to a user.
[0189] The synthetic input data may further represent information about one or more processing algorithms applied to feedback corrected version of the at least one electric input signal.
[0190] The synthetic output data may further represent parameters providing inputs to the one or more processing algorithms.
[0191] It is intended that the structural features of the devices described above, either in the detailed description and/or in the claims, may be combined with steps of the method, when appropriately substituted by a corresponding process.
[0192] Embodiments of the disclosure may e.g. be useful in electronic appliances, where acoustic feedback can be expected.
[0193] As used, the singular forms “a,” “an,” and “the” are intended to include the plural forms as well (i.e. to have the meaning “at least one”), unless expressly stated otherwise. It will be further understood that the terms “includes,” “comprises,” “including,” and/or “comprising,” when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. It will also be understood that when an element is referred to as being “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, but an intervening element may also be present, unless expressly stated otherwise. Furthermore, “connected” or “coupled” as used herein may include wirelessly connected or coupled. As used herein, the term “and/or” includes any and all combinations of one or more of the associated listed items. The steps of any disclosed method are not limited to the exact order stated herein, unless expressly stated otherwise.
[0194] It should be appreciated that reference throughout this specification to “one embodiment” or “an embodiment” or “an aspect” or features included as “may” means that a particular feature, structure or characteristic described in connection with the embodiment is included in at least one embodiment of the disclosure. Furthermore, the particular features, structures or characteristics may be combined as suitable in one or more embodiments of the disclosure. The previous description is provided to enable any person skilled in the art to practice the various aspects described herein. Various modifications to these aspects will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other aspects.
[0195] The claims are not intended to be limited to the aspects shown herein but are to be accorded the full scope consistent with the language of the claims, wherein reference to an element in the singular is not intended to mean “one and only one” unless specifically so stated, but rather “one or more.” Unless specifically stated otherwise, the term “some” refers to one or more.
REFERENCES
[0196] EP3139636A1 (Oticon, Bernafon) 8 Mar. 2017 [0197] EP3291581A2 (Oticon) 7 Mar. 2018 [0198] EP3236675A1 (Starkey) 25 Oct. 2017