HEARING AID DETERMINING TALKERS OF INTEREST
20220295191 · 2022-09-15
Assignee
Inventors
- Michael Syskind Pedersen (Smørum, DK)
- Jesper Jensen (Smørum, DK)
- Jan M. DE HAAN (Smørum, DK)
- Poul Hoang (Smørum, DK)
Cpc classification
H04R2225/41
ELECTRICITY
International classification
Abstract
A hearing aid includes an input providing an input signal representing sound in an environment, the input signal including no speech signal, or one or more speech signals from one or more speech sound sources and additional signal components, termed noise signal, from one or more other sound sources, an own voice detector, a voice activity detector, and a talker extraction unit to determine and/or receive one or more speech signals as separated one or more speech signals from speech sound sources other than the hearing aid user and to detect the speech signal originating from the voice of the user. The talker extraction unit provides separate signals, each including, or indicating presence of, one of the one or more speech signals. A noise reduction system determines speech overlap and/or gap between the speech signal originating from the user's voice and each of the separated one or more speech signals.
Claims
1. Hearing aid adapted for being located at or in an ear of a hearing aid user, or for being fully or partially implanted in the head of a hearing aid user, the hearing aid comprising: an input unit for providing at least one electric input signal representing sound in an environment of the hearing aid user, said electric input signal comprising no speech signal, or one or more speech signals from one or more speech sound sources and additional signal components, termed noise signal, from one or more other sound sources, an own voice detector (OVD) for repeatedly estimating whether or not, or with what probability, said at least one electric input signal, or a signal derived therefrom, comprises a speech signal originating from the voice of the hearing aid user, and providing an own voice control signal indicative thereof, a voice activity detector (VAD) for repeatedly estimating whether or not, or with what probability, said at least one electric input signal, or a signal derived therefrom, comprises the no speech signal, or the one or more speech signals from speech sound sources other than the hearing aid user, and providing a voice activity control signal indicative thereof, a talker extraction unit configured to determine and/or receive the one or more speech signals as separated one or more speech signals from speech sound sources other than the hearing aid user and to detect the speech signal originating from the voice of the hearing aid user, and where the talker extraction unit is further configured to provide separate signals, each comprising, or indicating the presence of, one of said one or more speech signals, and a noise reduction system configured to determine a speech overlap and/or gap between said speech signal originating from the voice of the hearing aid user and each of said separated one or more speech signals.
2. Hearing aid according to claim 1, wherein the noise reduction system is configured to determine the speech overlap and/or gap based at least on estimating whether or not, or with what probability, said at least one electric input signal, or signal derived therefrom, comprises speech signal originating from the voice of the hearing aid user and/or speech signals from each of said separated one or more speech signals.
3. Hearing aid according to claim 1, wherein the noise reduction system is further configured to determine said speech overlap and/or gap based on an XOR-gate estimator for estimating the speech overlap and/or gap between said speech signal originating from the own voice of the hearing aid user and each of said separated one or more speech signals.
4. Hearing aid according to claim 1, wherein the noise reduction system is further configured to determine said speech overlap and/or gap based on a maximum mean-square-error estimator for estimating the speech overlap and/or gap between said speech signal originating from the own voice of the hearing aid user and each of said separated one or more speech signals.
5. Hearing aid according to claim 1, wherein the hearing aid further comprises a timer configured to determine one or more time segments of said speech overlap between the speech signal originating from the own voice of the hearing aid user and each of said separated one or more speech signals.
6. Hearing aid according to claim 5, wherein the hearing aid is configured to rank said separated one or more speech signals depending on the time segments of each of the speech overlaps between the speech signal originating from the own voice of the hearing aid user and each of said separated one or more speech signals, where the speech signals are ranked with an increasing degree of interest as a function of a decreasing time segment of speech overlap.
7. Hearing aid according to claim 5, wherein the hearing aid is configured to determine whether said one or more of the time segments exceeds a time limit, and if so to label the respective speech signal as being part of the noise signal or to rank the respective speech signal with a lower degree of interest to the hearing aid user compared to speech signals that do not exceed said time limit.
8. Hearing aid according to claim 1, wherein the one or more speech signals are grouped into one or more conversation groups depending at least on the amount of speech overlap between the speech signal of the hearing aid user estimated by the OVD and the separated one or more speech signals, and where the one or more conversation groups are categorized with a varying degree of interest to the hearing aid user.
9. Hearing aid according to claim 8, wherein the one or more conversation groups are defined by comparing the speech overlaps between each of the one or more speech signals and all of the other one or more speech signals, including the speech signal from the hearing aid user.
10. Hearing aid according to claim 1, wherein the noise reduction system is configured to group the one or more separated speech signals into said one or more conversation groups depending at least on the determined direction and/or location of said one or more speech signals.
11. Hearing aid according to claim 1, wherein the hearing aid comprises one or more beamformers, and wherein the input unit is configured to provide at least two electric input signals connected to the one or more beamformers, and wherein the one or more beamformers are configured to provide at least one beamformed signal.
12. Hearing aid according to claim 11, wherein the one or more beamformers comprises one or more own voice cancelling beamformers configured to attenuate the speech signal originating from the own voice of the hearing aid user as determined by the OVD.
13. Hearing aid according to claim 1, wherein the noise reduction system is configured to additionally detect said noise signal during time segments wherein said VAD and OVD both indicate an absence of a speech signal in the at least one electric input signal, or a signal derived therefrom, or a presence of speech signal with a probability below a speech presence probability (SPP) threshold value.
14. Hearing aid according to claim 11, wherein, when the OVD estimates that the own voice of the hearing aid user is inactive, the one or more beamformers of the hearing aid is configured to estimate the direction to and/or location of one or more the sound sources providing speech signals, and to use the estimated direction and/or location to update the one or more beamformers of the hearing aid to not attenuate said one or more speech signals.
15. Hearing aid according to claim 8, wherein the hearing aid further comprises a movement sensor, and wherein the noise reduction system is configured to group one or more estimated speech signals in a group categorized with a high degree of interest to the hearing aid user, when movement is detected by the movement sensor.
16. A binaural hearing system comprising a hearing aid and a contralateral hearing aid according to claim 1, the binaural hearing system being configured to allow an exchange of data between the hearing aid and the contralateral hearing aid, e.g. via an intermediate auxiliary device.
17. A method of operating a hearing aid adapted for being located at or in an ear of a hearing aid user, or for being fully or partially implanted in the head of a hearing aid user, the method comprising: providing at least one electric input signal representing sound in an environment of the hearing aid user, by an input unit, said electric input signal comprising no speech signal, or one or more speech signals from one or more speech sound sources and additional signal components, termed noise signal, from one or more other sound sources, repeatedly estimating whether or not, or with what probability, said at least one electric input signal, or a signal derived therefrom, comprises a speech signal originating from the voice of the hearing aid user, and providing an own voice control signal indicative thereof, by an own voice detector (OVD), repeatedly estimating whether or not, or with what probability, said at least one electric input signal, or a signal derived therefrom, comprises the no speech signal, or the one or more speech signals from speech sound sources other than the hearing aid user, and providing a voice activity control signal indicative thereof, by a voice activity detector (VAD), determining and/or receiving the one or more speech signals as separated one or more speech signals from speech sound sources other than the hearing aid user and detecting the speech signal originating from the voice of the hearing aid user, by a talker extraction unit, providing separate signals, each comprising, or indicating the presence of, one of said one or more speech signals, by the talker extraction unit, and determining a speech overlap and/or gap between said speech signal originating from the voice of the hearing aid user and each of said separated one or more speech signals, by a noise reduction system.
18. Hearing aid according to claim 2, wherein the noise reduction system is further configured to determine said speech overlap and/or gap based on an XOR-gate estimator for estimating the speech overlap and/or gap between said speech signal originating from the own voice of the hearing aid user and each of said separated one or more speech signals.
19. Hearing aid according to claim 2, wherein the noise reduction system is further configured to determine said speech overlap and/or gap based on a maximum mean-square-error estimator for estimating the speech overlap and/or gap between said speech signal originating from the own voice of the hearing aid user and each of said separated one or more speech signals.
20. Hearing aid according to claim 3, wherein the noise reduction system is further configured to determine said speech overlap and/or gap based on a maximum mean-square-error estimator for estimating the speech overlap and/or gap between said speech signal originating from the own voice of the hearing aid user and each of said separated one or more speech signals.
Description
BRIEF DESCRIPTION OF DRAWINGS
[0228] The aspects of the disclosure may be best understood from the following detailed description taken in conjunction with the accompanying figures. The figures are schematic and simplified for clarity, and they just show details to improve the understanding of the claims, while other details are left out. Throughout, the same reference numerals are used for identical or corresponding parts. The individual features of each aspect may each be combined with any or all features of the other aspects. These and other aspects, features and/or technical effect will be apparent from and elucidated with reference to the illustrations described hereinafter in which:
[0229]
[0230]
[0231]
[0232]
[0233]
[0234] The figures are schematic and simplified for clarity, and they just show details which are essential to the understanding of the disclosure, while other details are left out. Throughout, the same reference signs are used for identical or corresponding parts.
[0235] Further scope of applicability of the present disclosure will become apparent from the detailed description given hereinafter. However, it should be understood that the detailed description and specific examples, while indicating preferred embodiments of the disclosure, are given by way of illustration only. Other embodiments may become apparent to those skilled in the art from the following detailed description.
DETAILED DESCRIPTION OF EMBODIMENTS
[0236] The detailed description set forth below in connection with the appended drawings is intended as a description of various configurations. The detailed description includes specific details for the purpose of providing a thorough understanding of various concepts. However, it will be apparent to those skilled in the art that these concepts may be practiced without these specific details. Several aspects of the apparatus and methods are described by various blocks, functional units, modules, components, circuits, steps, processes, algorithms, etc. (collectively referred to as “elements”). Depending upon particular application, design constraints or other reasons, these elements may be implemented using electronic hardware, computer program, or any combination thereof.
[0237]
[0238] In
[0239] Alternatively, each of the talkers B, C, and D may be equipped with a microphone (e.g. in form of a hearing aid) capable of transmitting audio or information about when each of the talkers B, C, and D voices are active. The voices may be detected by a VD and/or a VAD.
[0240]
[0241] In
[0242] As seen from the speech signals of the hearing aid user A, of the talker B, and of the combined signal A+B, the speech signal of the hearing aid user A does not overlap in time with the speech signal of talker B.
[0243] Similarly, as seen from the speech signals of the talkers C and D, and of the combined signal C+D, the speech signal of the talker C does not overlap in time with the speech signal of the talker D.
[0244] At the bottom of
[0245] Accordingly, as the hearing aid user A and talker B do not talk at the same time, it indicates that a conversation is going on between the hearing aid user A and talker B. Similarly, as the talkers C and D do not talk at the same time, it indicates that a conversation is going on between the talkers C and D.
[0246] As seen in the combined speech signal (A+B+C+D), the speech signals of talker C and talker D overlap in time with talker A and talker B. Therefore, it may be concluded that talkers C and D have a simultaneous conversation, independent of the hearing aid user A and the talker B. Thus, the conversation between talker C and talker D is of less interest to the hearing aid user, and may be regarded as part of the background noise signal.
[0247] Thereby, the talkers belonging to the same group of talkers do not overlap in time while talkers belonging to different dialogues (e.g. hearing aid user A and talker C) do overlap in time. It may be assumed that talker B is of main interest to the hearing aid user, while talkers C and D are of less interest as talker C and D overlap in time with the hearing aid user A and talker B. The hearing aid(s) may therefore group the speech signal of talker B into a conversation group categorized with a higher degree of interest than the conversation group comprising the speech signals of talkers C and D, based on the overlaps/no-overlaps of the speech signals.
[0248]
[0249] In
[0250] The hearing aid may further comprise an OVD (not shown) and a VAD (not shown).
[0251] The hearing aid 3 may further comprise a talker extraction unit 5 for receiving the electric input signals from the plurality of input transducers 4A;4n. The talker extraction unit 5 may be configured to separate the one or more speech signals, estimated by the VAD, and to detect the speech signal originating from the voice of the hearing aid user, by the OVD.
[0252] The talker extraction unit 5 may be further configured to provide separate signals, each comprising, or indicating the presence of, one of said one or more speech signals.
[0253] In the example of
[0254] The hearing aid 3, such as a speech ranking and noise reduction system 6 of the hearing aid 3, may further be configured to determine/estimate a speech overlap between said speech signal originating from the voice of the hearing aid user A and each of said separated one or more speech signals by a speech ranking algorithm, which is illustrated to originate from talkers B, C, and D.
[0255] Based on the determined speech overlap, the hearing aid 3 may be configured to determine the speech signal(s) of interest to the hearing aid user and to output the interesting speech signal(s) and the own voice via an output unit 7, thereby providing a stimulus perceived by the hearing aid user as an acoustic signal.
[0256]
[0257]
[0258] The total space 10 surrounding the hearing aid user 8 may be a cylinder volume, but may alternatively have any other form. The total space 10 can also for example be represented by a sphere (or semi-sphere, a dodecahedron, a cube, or similar geometric structures). A subspace 11 of the total space 10 may correspond to a cylinder sector. The subspaces 11 can also be spheres, cylinders, pyramids, dodecahedra or other geometrical structures that allow to divide the total space 10 into subspaces 11. The subspaces 11 add up to the total space 10, meaning that the subspaces 11 fill the total space 10 completely and do not overlap. Each beam.sub.p, p=1, 2, . . . , P, may constitute a subspace (cross-section) where P (here equal to 8) is the number of subspaces 11. There may also be empty spaces between the subspaces 11 and/or overlap of subspaces 11. The subspaces 11 in
[0259] A spatial filterbank may be configured to divide the one or more sound signals into subspaces corresponding to directions of a horizontal “pie”, which may be divided into, e.g., 18 slices of 20 degrees with a total space 10 of 360 degrees.
[0260] The location coordinates, extension, and number of subspaces 11 depends on subspace parameters. The subspace parameters may be adaptively adjusted, e.g., in dependence of an outcome of the VAD, etc. The adjustment of the extension of the subspaces 11 allows to adjust the form or size of the subspaces 11. The adjustment of the number of subspaces 11 allows to adjust the sensitivity, respective resolution and therefore also the computational demands of the hearing aids 9 (or hearing system). Adjusting the location coordinates of the subspaces 11 allows to increase the sensitivity at certain location coordinates or directions in exchange for a decreased sensitivity for other location coordinates or directions.
[0261]
[0262] In
[0263]
[0264]
[0265]
[0266] In
[0267] As shown, the voice activity of each of the speaking partners (‘SP1’, ‘SP2’, . . . ‘SPN’) may be compared with the voice activity of the hearing aid user (‘User’).
[0268] The comparisons of the voice activity (thereby determining speech overlap) may be carried out in one or more of several different ways. In
[0269] The XOR-gate estimator may compare the own voice (own voice control signal) with each of the separate speaking partner signals (speaking partner control signals) to thereby provide an overlap control signal for each of said separate signals. The resulting overlap control signals for the speech signals (‘User’, ‘SP1’, ‘SP2’, . . . ‘SPN’) identify time segments where speaking partner speech signals has no overlap with the voice of the hearing aid user by providing a ‘1’.
[0270] Time segments with speech overlap provides a ‘0’.
[0271] Thereby, the speech signal of the speaking partners (‘SP1’, ‘SP2’, . . . ‘SPN’) in the sound environment of the hearing aid user (‘User’) at a given time may be ranked according to a minimum speech overlap with the own voice speech signal of the hearing aid user (and/or the speaking partner with the smallest speech overlap may be identified).
[0272] Thereby, an indication of a probability of a conversation being conducted between the hearing aid user (‘User’) and one or more of the speaking partners (‘SP1’, ‘SP2’, . . . ‘SPN’) around the hearing aid user (‘User’) may be provided. Further, by comparing each of the separate signals with all the other separate signals and ranking the separate signals according to the smallest overlap with the own voice speech signal, the separate signals may be grouped into different conversation groups of varying interest to the hearing aid user.
[0273] The output of the comparison may be low-pass filtered (by a low-pass filter of the hearing aid). For example, a low-pass filter may have a time constant of 1 second, 10 seconds, 20 seconds, or 100 seconds.
[0274] Additionally, a NAND-gate estimator may compare the own voice (own voice control signal) with each of the separate speaking partner signals (speaking partner control signals). The NAND-gate estimator may be configured to indicate that speech overlaps are the main cue for disqualifying speaking partners.
[0275] For example, in
[0276] In
[0277] The duration of the conversations between the hearing aid user (‘User’) and each (more) of the speaking partners (‘SP1’, ‘SP2’, . . . ‘SPN’) may be logged in the hearing aid (e.g. in a memory of the hearing aid).
[0278] The duration of said conversations may be measured by a timer/counter, e.g. to count the amount of time where OV is detected and the amount of time where the voice(s) (of interest) of one or more of the speaking partners (‘SP1’, ‘SP2’, . . . ‘SPN’) are detected.
[0279] It is intended that the structural features of the devices described above, either in the detailed description and/or in the claims, may be combined with steps of the method, when appropriately substituted by a corresponding process.
[0280] As used, the singular forms “a,” “an,” and “the” are intended to include the plural forms as well (i.e. to have the meaning “at least one”), unless expressly stated otherwise. It will be further understood that the terms “includes,” “comprises,” “including,” and/or “comprising,” when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. It will also be understood that when an element is referred to as being “connected” or “coupled” to another element, it can be directly connected or coupled to the other element but an intervening element may also be present, unless expressly stated otherwise. Furthermore, “connected” or “coupled” as used herein may include wirelessly connected or coupled. As used herein, the term “and/or” includes any and all combinations of one or more of the associated listed items. The steps of any disclosed method are not limited to the exact order stated herein, unless expressly stated otherwise.
[0281] It should be appreciated that reference throughout this specification to “one embodiment” or “an embodiment” or “an aspect” or features included as “may” means that a particular feature, structure or characteristic described in connection with the embodiment is included in at least one embodiment of the disclosure. Furthermore, the particular features, structures or characteristics may be combined as suitable in one or more embodiments of the disclosure. The previous description is provided to enable any person skilled in the art to practice the various aspects described herein. Various modifications to these aspects will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other aspects.
[0282] The claims are not intended to be limited to the aspects shown herein but are to be accorded the full scope consistent with the language of the claims, wherein reference to an element in the singular is not intended to mean “one and only one” unless specifically so stated, but rather “one or more.” Unless specifically stated otherwise, the term “some” refers to one or more.