MONAURAL INTRUSIVE SPEECH INTELLIGIBILITY PREDICTOR UNIT, A HEARING AID AND A BINAURAL HEARING AID SYSTEM

Abstract

A monaural intrusive speech intelligibility predictor unit comprises: first and second input units for providing time-frequency representations s(k,m) and x(k,m) of noise-free and noisy and/or processed versions of a target signal, respectively, k being a frequency bin index, k=1, 2, . . . , K, and m being a time index; first and second envelope extraction units for providing time-frequency sub-band representations of the signals s.sub.j(m) and x.sub.j(m), j being a frequency sub-band index, j=1, 2, . . . , J; first and second time-frequency segment division units for dividing the time-frequency sub-band representations s.sub.j(m) and x.sub.j(m) into time-frequency segments S.sub.m and X.sub.m corresponding to a number N of successive samples of the sub-band signals; an intermediate speech intelligibility calculation unit adapted for providing intermediate speech intelligibility coefficients d.sub.m estimating an intelligibility of said time-frequency segment X.sub.m, based on said time-frequency segments S.sub.m and X.sub.m or normalized and/or transformed versions {tilde over (S)}.sub.m, and {tilde over (X)}.sub.m thereof; and a final monaural speech intelligibility calculation unit for calculating a final monaural speech intelligibility predictor d estimating an intelligibility of said noisy and/or processed version x of the target signal by combining said intermediate speech intelligibility coefficients d.sub.m, or a transformed version thereof, over time. A hearing aid comprises a monaural, intrusive intelligibility predictor unit, and a configurable signal processor adapted to control or influence the processing of one or more electric input signals representing environment sound to maximize the final speech intelligibility predictor d. A binaural hearing aid system comprises first and second hearing aids.

Claims

1. A monaural speech intelligibility predictor unit adapted for receiving a target signal comprising speech in an essentially noise-free version s and in a noisy and/or processed version x, the monaural speech intelligibility predictor unit being configured to provide as an output a final monaural speech intelligibility predictor value d indicative of a listener's perception of said noisy and/or processed version x of the target signal, the monaural speech intelligibility predictor unit comprising A first input unit for providing a time-frequency representation s(k,m) of said noise-free version s of the target signal, k being a frequency bin index, k=1, 2, . . . , K, and m being a time index; A second input unit for providing a time-frequency representation x(k,m) of said noisy and/or processed version x of the target signal, k being a frequency bin index, k=1, 2, . . . , K, and m being a time index; A first envelope extraction unit for providing a time-frequency sub-band representation s.sub.j(m) of the noise-free version s of the target signal representing temporal envelopes, or functions thereof, of frequency sub-band signals s.sub.j(m) of said noise-free target signal, j being a frequency sub-band index, j=1, 2, . . . , J, and m being the time index; A second envelope extraction unit for providing a time-frequency sub-band representation x.sub.j(m) of the noisy and/or processed version x of the target signal representing temporal envelopes, or functions thereof, of frequency sub-band signals x.sub.j(m) of said noisy and/or processed version of the target signal, j=1, 2, . . . , J, and m being the time index; A first time-frequency segment division unit for dividing said time-frequency sub-band representation s.sub.j(m) of the noise-free version s of the target signal into time-frequency segments S.sub.m corresponding to a number N of successive samples of said sub-band signals; A second time-frequency segment division unit for dividing said time-frequency sub-band representation x.sub.j(m) of the noisy and/or processed version x of the target signal into time-frequency segments X.sub.m corresponding to a number N of successive samples of said sub-band signals; A normalization and transformation unit configured to provide at least one normalization and/or transformation operation of rows and at least one normalization and/or transformation operation of columns of the time-frequency segments S.sub.m and X.sub.m; An intermediate speech intelligibility calculation unit adapted for providing intermediate speech intelligibility coefficients d.sub.m estimating an intelligibility of said time-frequency segment X.sub.m, said intermediate speech intelligibility coefficients d.sub.m being based on said essentially noise-free, normalized and/or transformed time frequency segments {tilde over (S)}.sub.m, and said noisy and/or processed, normalized and/or transformed time-frequency segments {tilde over (X)}.sub.m; A final monaural speech intelligibility calculation unit for calculating a final monaural speech intelligibility predictor d estimating an intelligibility of said noisy and/or processed version x of the target signal by combining, e.g. by averaging, or by applying a MIN or MAX-function to, said intermediate speech intelligibility coefficients d.sub.m, or a transformed version thereof, over time.

2. A monaural speech intelligibility predictor unit according to claim 1 comprising a voice activity detector unit for indicating whether or not or to what extent a given time-segment of the essentially noise-free version s and the noisy and/or processed version x, respectively, of the target signal comprises or is estimated to comprise speech, and providing a voice activity control signal indicative thereof.

3. A monaural speech intelligibility predictor unit according to claim 1 comprising a voice activity detector unit for identifying time-segments of the essentially noise-free version s and the noisy and/or processed version x, respectively, of the target signal comprising or estimated to comprise speech, and wherein the monaural speech intelligibility predictor unit is configured to provide modified versions of the essentially noise-free version s and of the noisy and/or processed version x, respectively, of the target signal, said modified versions comprising only such time segments comprising speech or being estimated to comprise speech.

4. A monaural speech intelligibility predictor unit according to claim 1 comprising a hearing loss model unit configured to apply a modification of the said noisy and/or processed version x of the target signal reflecting a deviation from normal hearing of a relevant ear of the listener to provide a modified noisy and/or processed version x of the target signal for use together with said essentially noise-free version s of the target signal as a basis for calculating the final monaural speech intelligibility predictor d.

5. A monaural speech intelligibility predictor unit according to claim 1 wherein said hearing loss model unit is configured to add a statistically independent noise signal, which is spectrally shaped according to an audiogram of the relevant ear of the listener, to said noisy and/or processed version x of the target signal.

6. A monaural speech intelligibility predictor unit according to claim 1 adapted to extract said temporal envelope signals x.sub.j(m) and s.sub.j(m), respectively, as $z_j\left(m\right)=f\left(\sqrt{\underset{k=k.Math..Math.1.Math.\left(j\right)}{\overset{k.Math..Math.2.Math.\left(j\right)}{.Math.}}.Math..Math.{.Math.z\left(k,m\right).Math.}^2}\right),$ where z represents x or s, j=1, . . . , J and m=1, . . . , M, k1(j) and k2(j) denote DFT bin indices corresponding to lower and higher cut-off frequencies of the j.sup.th sub-band, J is the number of sub-bands, and M is the number of signal frames in the signal in question, and ƒ(•) is a function.

7. A monaural speech intelligibility predictor unit according to claim 6 wherein the function f(•)-f(w), where w represents $\left(\sqrt{\underset{k=k.Math..Math.1.Math.\left(j\right)}{\overset{k.Math..Math.2.Math.\left(j\right)}{.Math.}}.Math..Math.{.Math.z\left(k,m\right).Math.}^2}\right),$ is selected among the following functions ƒ(w)=w representing the identity ƒ(w)=w.sup.2 providing power envelopes, ƒ(w)=2.Math.log w or ƒ(w)=w.sup.β, 0<β<2, allowing the modelling of the compressive non-linearity of the healthy cochlea, or combinations thereof.

8. A monaural speech intelligibility predictor unit according to claim 1 wherein said first and second time-frequency segment division units are configured to divide said time-frequency representations s.sub.j(m) and x.sub.j(m), respectively, into segments in the form of spectrograms corresponding to N successive samples of all sub-band signals, wherein the m.sup.th segment is defined by the J×N matrix $Z_m=\left[\begin{array}{ccc}{z}_1\left(m-N+1\right)& \mathrm{.Math.}& z_1\left(m\right)\\ \mathrm{.Math.}& & \mathrm{.Math.}\\ z_J\left(m-N+1\right)& \mathrm{.Math.}& z_J\left(m\right)\end{array}\right].$ where z, Z represents s, S or x, X, respectively.

9. A monaural speech intelligibility predictor unit according to claim 1 comprising A first normalization and/or transformation unit adapted for providing normalized and/or transformed versions {tilde over (S)}.sub.m of said time-frequency segments S.sub.m; A second normalization and/or transformation unit adapted for providing normalized and/or transformed versions {tilde over (X)}.sub.m of said time-frequency segments X.sub.m;

10. A monaural speech intelligibility predictor unit according to claim 9, wherein first and second normalization and/or transformation units are configured to apply one or more of the following algorithms to the time-frequency segments X.sub.m and S.sub.m, respectively, commonly denoted Z.sub.m, where sub-script, time index m is skipped for simplicity in the following expressions: Normalization of rows to zero mean:
g.sub.1(Z)=Z−μ.sub.z.sup.r1.sup.T, where μ.sub.z.sup.r is a J×1 vector whose j′th entry is the mean of the j′th row of Z, hence the superscript r in μ.sub.z.sup.r, where 1 denotes an N×1 vector of ones, and where superscript T denotes matrix transposition; Normalization of rows to unit-norm:
g.sub.2(Z)=D.sup.r(Z)Z, where D.sup.r(Z)=diag(└1/√{square root over (Z(1,:)Z(1,:).sup.H)} . . . 1/√{square root over (Z(J,:)Z(J,:).sup.H)}┘), where diag(•) is a diagonal matrix with the elements of the arguments on the main diagonal, and where Z(j,:) denotes the j′th row of Z, such that D′(Z) is a J×J diagonal matrix with the inverse norm of each row on the main diagonal, and zeros elsewhere, the superscript H denotes Hermitian transposition, and pre-multiplication with D′(Z) normalizes the rows of the resulting matrix to unit-norm; Fourier transformation applied to each row
g.sub.3(Z)=ZF, where F is an N×N Fourier matrix; Fourier transformation applied to each row followed by computing the magnitude of the resulting complex-valued elements
g.sub.4=|ZF| where |•| computes the element-wise magnitudes; The identity operator
g.sub.5(Z)=Z. Normalization of columns to zero mean:
h.sub.1(Z)=Z−1μ.sub.z.sup.c.sup.T, where μ.sub.z.sup.c is a N×1 vector whose i.sup.th entry is the mean of the i.sup.th row of Z, and where 1 denotes a J×1 vector of ones; Normalization of columns to unit-norm:
h.sub.2(Z)=ZD.sup.c(Z), where D.sup.c(Z)=diag(└1/√{square root over (Z(:,1).sup.HZ(:,1))} . . . 1/√{square root over (Z(:,N).sup.HZ(:,N))}┘, where Z(:,n) denotes the n′th row of Z, such that D.sup.c(z) is a diagonal N×N matrix with the inverse norm of each column on the main diagonal, and zeros elsewhere, and where a post-multiplication with D.sup.c(Z) normalizes the rows of the resulting matrix to unit-norm.

11. A monaural speech intelligibility predictor unit according to claim 1 wherein the intermediate speech intelligibility calculation unit is adapted to determine said intermediate speech intelligibility coefficients d.sub.m in dependence on a, e.g. linear, sample correlation coefficient d(a,b) of the elements in two K×1 vectors a and b, d(a,b) being defined by: $d\left(a,b\right)=\frac{\underset{k=1}{\overset{K}{.Math.}}.Math..Math.\left(a\left(k\right)-{\mu }_a\right).Math.\left(b\left(k\right)-{\mu }_b\right)}{\sqrt{\underset{k=1}{\overset{K}{.Math.}}.Math..Math.{\left(a\left(k\right)-{\mu }_a\right)}^2.Math.{\left(b\left(k\right)-{\mu }_b\right)}^2}},.Math.\mathrm{where}$ ${\mu }_a=\frac{1}{K}.Math.\underset{k=1}{\overset{K}{.Math.}}.Math..Math.a\left(k\right).Math..Math.\mathrm{and}.Math.$ $.Math.{\mu }_b=\frac{1}{K}.Math.\underset{k=1}{\overset{K}{.Math.}}.Math.b\left(k\right),$ where k is the index of the vector entry and K is the vector dimension.

12. A monaural speech intelligibility predictor unit according to claim 11 wherein the intermediate intelligibility index d.sub.m is defined as the average sample correlation coefficient of all columns in S.sub.m and X.sub.m, or {tilde over (S)}.sub.m and {tilde over (X)}.sub.m, respectively, i.e., $d_m=\frac{1}{N}.Math.\underset{n=1}{\overset{N}{.Math.}}.Math..Math.d\left({\tilde{S}}_m\left(:,n\right),{\tilde{X}}_m\left(:,n\right)\right),$ n being a column index, or as the average sample correlation coefficient of all rows in S.sub.m and X.sub.m, or {tilde over (S)}.sub.m and {tilde over (X)}.sub.m, i.e., $d_m=\frac{1}{J}.Math.\underset{j=1}{\overset{J}{.Math.}}.Math..Math.d\left({{\tilde{S}}_m\left(j,:\right)}^T,{{\tilde{X}}_m\left(j,:\right)}^T\right),$ j being a row index, or as the sample correlation coefficient of all elements in S.sub.m and X.sub.m, or {tilde over (S)}.sub.m and {tilde over (X)}.sub.m, i.e.,
d.sub.m=d({tilde over (S)}.sub.m(:),{tilde over (X)}.sub.m(:)), where the notation S.sub.m(:) and X.sub.m(:), or {tilde over (S)}.sub.m(:) and {tilde over (X)}.sub.m(:), represents NJ×1 vectors formed by stacking the columns of the respective matrices.

13. A monaural speech intelligibility predictor unit according to claim 1 wherein the final speech intelligibility calculation unit is adapted to calculate the final speech intelligibility predictor d from the intermediate speech intelligibility coefficients d.sub.m, optionally transformed by a function u(d.sub.m), as an average over time of said information signal x: $d=\frac{1}{M}.Math.\underset{m=1}{\overset{M}{.Math.}}.Math..Math.u\left(d_m\right).$ where M represents the duration in time units of the speech active parts of said noisy and/or processed version x of the target signal.

14. A monaural speech intelligibility predictor unit according to claim 13 wherein function u(d.sub.m) is defined as $u\left(d_m\right)=\log \left(\frac{1}{1-d_m^2}\right),$ or as
u(d.sub.m)=d.sub.m.

15. A hearing aid adapted for being located at or in left or right ears of a user, or for being fully or partially implanted in the head of the user, the hearing aid comprising a monaural speech intelligibility predictor unit according to claim 1.

16. A hearing aid according to claim 15 configured to adaptively modify the processing of an input signal to the hearing aid to maximize the monaural speech intelligibility predictor d, to enhance the user's intelligibility of an output signal of the hearing aid presented to the user.

17. A binaural hearing system comprising left and right hearing aids according to claim 15, wherein each of the left and right hearing aids comprises antenna and transceiver circuitry for allowing a communication link to be established and information to be exchanged between said left and right hearing aids, the binaural hearing system further comprising a binaural speech intelligibility prediction unit for providing a final binaural speech intelligibility measure d.sub.binaural of the predicted speech intelligibility of the user, when exposed to said sound input, based on the monaural speech intelligibility predictor values d.sub.left, d.sub.right of the respective left and right hearing aids.

18. A binaural hearing system according to claim 17, wherein the respective configurable signal processors of the left and right hearing aids are adapted to control or influence the processing of the respective electric input signals to maximize said final binaural speech intelligibility measure d.sub.binaural.

19. A method of providing a monaural speech intelligibility predictor for estimating a user's ability to understand an information signal x comprising a noisy and/or processed version of a target speech signal, the method comprising providing a time-frequency representation s(k,m) of said noise-free version s of the target signal, k being a frequency bin index, k=1, 2, . . . , K, and m being a time index; providing a time-frequency representation x(k,m) of said noisy and/or processed version x of the target signal, k being a frequency bin index, k=1, 2, . . . , K, and m being a time index; providing a time-frequency sub-band representation s.sub.j(m) of the noise-free version s of the target signal representing temporal envelopes, or functions thereof, of frequency sub-band signals s.sub.j(m) of said noise-free target signal, j being a frequency sub-band index, j=1, 2, . . . , J, and m being the time index; providing a time-frequency sub-band representation x.sub.j(m) of the noisy and/or processed version x of the target signal representing temporal envelopes, or functions thereof, of frequency sub-band signals x.sub.j(m) of said noisy and/or processed version of the target signal, j=1, 2, . . . , J, and m being the time index; dividing said time-frequency sub-band representation s.sub.j(m) of the noise-free version s of the target signal into time-frequency segments S.sub.m corresponding to a number N of successive samples of said sub-band signals; dividing said time-frequency sub-band representation x.sub.j(m) of the noisy and/or processed version x of the target signal into time-frequency segments X.sub.m corresponding to a number N of successive samples of said sub-band signals; providing at least one normalization and/or transformation operation of rows and at least one normalization and/or transformation operation of columns of the time-frequency segments S.sub.m and X.sub.m; providing intermediate speech intelligibility coefficients d.sub.m estimating an intelligibility of said time-frequency segment X.sub.m, said intermediate speech intelligibility coefficients d.sub.m being based on said essentially noise-free, normalized and/or transformed time frequency segments {tilde over (S)}.sub.m, and said noisy and/or processed, normalized and/or transformed time-frequency segments {tilde over (X)}.sub.m; calculating a final monaural speech intelligibility predictor d estimating an intelligibility of said noisy and/or processed version x of the target signal by combining, e.g. by averaging or applying a MIN or MAX-function, said intermediate speech intelligibility coefficients d.sub.m, or a transformed version thereof, over time.

20. A non-transitory computer-readable medium storing a computer program comprising instructions which, when executed by a computer, cause the computer to carry out the method of claim 19.

Description

BRIEF DESCRIPTION OF DRAWINGS

[0111] The aspects of the disclosure may be best understood from the following detailed description taken in conjunction with the accompanying figures. The figures are schematic and simplified for clarity, and they just show details to improve the understanding of the claims, while other details are left out. Throughout, the same reference numerals are used for identical or corresponding parts. The individual features of each aspect may each be combined with any or all features of the other aspects. These and other aspects, features and/or technical effect will be apparent from and elucidated with reference to the illustrations described hereinafter in which:

[0112] FIG. 1A schematically shows a time variant analogue signal (Amplitude vs time) and its digitization in samples, the samples being arranged in a number of time frames, each comprising a number N.sub.s of samples, and

[0113] FIG. 1B illustrates a time-frequency map representation of the time variant electric signal of FIG. 1A,

[0114] FIG. 2A symbolically shows an intrusive monaural speech intelligibility predictor unit providing a monaural speech intelligibility predictor d based on a time-frequency representation x.sub.j(m) of an information signal x,

[0115] FIG. 2B shows a first embodiment an intrusive monaural speech intelligibility predictor unit, and

[0116] FIG. 2C shows a second embodiment an intrusive monaural speech intelligibility predictor unit.

[0117] FIG. 3A shows an intrusive monaural speech intelligibility predictor unit in combination with a hearing loss model and an evaluation unit,

[0118] FIG. 3B shows an intrusive monaural speech intelligibility predictor unit in combination with a signal processor and an evaluation unit, and

[0119] FIG. 3C shows a first embodiment of a hearing aid comprising an intrusive monaural speech intelligibility predictor unit with a hearing loss model configured to optimize a user's intelligibility of an output signal of the hearing aid,

[0120] FIG. 4A shows a first scenario for using a hearing aid comprising an intrusive monaural speech intelligibility predictor according to the present disclosure to improve a hearing aid user's intelligibility of speech from a speaker wearing a wireless microphone (e.g. in a teaching or lecture situation), and

[0121] FIG. 4B shows a second scenario for using a hearing aid comprising an intrusive monaural speech intelligibility predictor according to the present disclosure to improve a hearing aid user's intelligibility of speech from a remote speaker of a telephone conversation using a handsfree telephone set in a car, where remote sound is wirelessly as well as acoustically transmitted to the hearing aid user,

[0122] FIG. 5A shows a third scenario for using a hearing aid comprising an intrusive monaural speech intelligibility predictor according to the present disclosure to improve a hearing aid user's intelligibility of speech from a remote speaker of a telephone conversation using a handsfree telephone set in a car, where remote sound is (only) wirelessly transmitted to the hearing aid user,

[0123] FIG. 5B shows an embodiment of a hearing aid comprising an intrusive monaural speech intelligibility predictor for use in the (third) scenario of FIG. 5A, and

[0124] FIG. 5C illustrates an exemplary hearing aid formed as a receiver in the ear (RITE) type of hearing aid comprising a part adapted for being located behind pinna and a part comprising an output transducer (e.g. a loudspeaker/receiver) adapted for being located in an ear canal of the user,

[0125] FIG. 6A shows a binaural hearing aid system according to the present disclosure comprising first and second hearing aids and an auxiliary device, and

[0126] FIG. 6B shows the auxiliary device comprising a user interface in the form of an APP for controlling and displaying data related to the speech intelligibility predictors, and

[0127] FIG. 7 shows an embodiment of a binaural hearing aid system according to the present disclosure comprising a left and right hearing devices, each comprising a monaural speech intelligibility predictor whose individual predictor values d.sub.left and d.sub.right are exchanged between the hearing devices and used to influence or control signal processing in the hearing devices to optimize binaural speech intelligibility of the user.

[0128] The figures are schematic and simplified for clarity, and they just show details which are essential to the understanding of the disclosure, while other details are left out. Throughout, the same reference signs are used for identical or corresponding parts.

[0129] Further scope of applicability of the present disclosure will become apparent from the detailed description given hereinafter. However, it should be understood that the detailed description and specific examples, while indicating preferred embodiments of the disclosure, are given by way of illustration only. Other embodiments may become apparent to those skilled in the art from the following detailed description.

DETAILED DESCRIPTION OF EMBODIMENTS

[0130] The detailed description set forth below in connection with the appended drawings is intended as a description of various configurations. The detailed description includes specific details for the purpose of providing a thorough understanding of various concepts. However, it will be apparent to those skilled in the art that these concepts may be practised without these specific details. Several aspects of the apparatus and methods are described by various blocks, functional units, modules, components, circuits, steps, processes, algorithms, etc. (collectively referred to as “elements”). Depending upon particular application, design constraints or other reasons, these elements may be implemented using electronic hardware, computer program, or any combination thereof.

[0131] The electronic hardware may include microprocessors, microcontrollers, digital signal processors (DSPs), field programmable gate arrays (FPGAs), programmable logic devices (PLDs), gated logic, discrete hardware circuits, and other suitable hardware configured to perform the various functionality described throughout this disclosure. Computer program shall be construed broadly to mean instructions, instruction sets, code, code segments, program code, programs, subprograms, software modules, applications, software applications, software packages, routines, subroutines, objects, executables, threads of execution, procedures, functions, etc., whether referred to as software, firmware, middleware, microcode, hardware description language, or otherwise.

[0132] The present application relates to the field of hearing aids or hearing aid systems.

[0133] The present disclosure relates to signal processing methods for predicting the intelligibility of speech, e.g., the output signal of a signal processing device such as a hearing aid. The intelligibility prediction is made in the form of an index that correlates highly with the fraction of words that an average listener would be able to understand from some speech material. For situations where an estimate of absolute intelligibility, i.e., the actual percentage of words understood, is desired, this index may be transformed to a number in the range 0-100 percent, see e.g. [3] for one method to do this.

[0134] The method proposed here belongs to the class of so-called intrusive methods. Methods in this class are characterized by the fact that they make their intelligibility prediction by comparing the noisy—and potentially signal processed—speech signal, with a noise-free, undistorted version of the underlying speech signal, see [1, 2, 3] for examples of existing methods. The assumption that a noise-free reference signal is available is reasonable in many practically relevant situations. For example, when evaluating the impact of various hearing aid signal processing algorithms on intelligibility, one normally conducts a listening test with human subjects. In preparing such a test, the stimuli are often created artificially by explicitly adding noise signal to noise-free speech signals—in other words, noise-free signals are readily available. Hence, the proposed intelligibility prediction algorithm allows one to replace a costly and time-consuming listening test involving human subjects, with machine predictions.

[0135] Much of the signal processing of the present disclosure is performed in the time-frequency domain, where a time domain signal is transformed into the (time-)frequency domain by a suitable mathematical algorithm (e.g. a Fourier transform algorithm) or filter (e.g. a filter bank).

[0136] FIG. 1A schematically shows a time variant analogue signal (Amplitude vs time) and its digitization in samples, the samples being arranged in a number of time frames, each comprising a number N.sub.s of digital samples. FIG. 1A shows an analogue electric signal (solid graph), e.g. representing an acoustic input signal, e.g. from a microphone, which is converted to a digital audio signal in an analogue-to-digital (AD) conversion process, where the analogue signal is sampled with a predefined sampling frequency or rate f.sub.s, f.sub.s being e.g. in the range from 8 kHz to 40 kHz (adapted to the particular needs of the application) to provide digital samples x(n) at discrete points in time n, as indicated by the vertical lines extending from the time axis with solid dots at its endpoint coinciding with the graph, and representing its digital sample value at the corresponding distinct point in time n. Each (audio) sample x(n) represents the value of the acoustic signal at n by a predefined number N.sub.b of bits, N.sub.b being e.g. in the range from 1 to 16 bits. A digital sample x(n) has a length in time of 1/f.sub.s, e.g. 50 μs, for f.sub.s=20 kHz. A number of (audio) samples N.sub.s are arranged in a time frame, as schematically illustrated in the lower part of FIG. 1A, where the individual (here uniformly spaced) samples are grouped in time frames (1, 2, . . . , N.sub.s)). As also illustrated in the lower part of FIG. 1A, the time frames may be arranged consecutively to be non-overlapping (time frames 1, 2, . . . , m, . . . , M) or overlapping (here 50%, time frames 1, 2, . . . , m, . . . , M′), where in is time frame index. In an embodiment, a time frame comprises 64 audio data samples. Other frame lengths may be used depending on the practical application.

[0137] FIG. 1B schematically illustrates a time-frequency representation of the (digitized) time variant electric signal x(n) of FIG. 1A. The time-frequency representation comprises an array or map of corresponding complex or real values of the signal in a particular time and frequency range. The time-frequency representation may e.g. be a result of a Fourier transformation converting the time variant input signal x(n) to a (time variant) signal x(k,m) in the time-frequency domain. In an embodiment, the Fourier transformation comprises a discrete Fourier transform algorithm (DFT). The frequency range considered by a typical hearing device (e.g. a hearing aid) from a minimum frequency f.sub.min to a maximum frequency f.sub.max comprises a part of the typical human audible frequency range from 20 Hz to 20 kHz, e.g. a part of the range from 20 Hz to 12 kHz. In FIG. 1B, the time-frequency representation x(k,m) of signal x(n) comprises complex values of magnitude and/or phase of the signal in a number of DFT-bins defined by indices (k,m), where k=1, . . . , K represents a number K of frequency values (cf. vertical k-axis in FIG. 1B) and m=1, . . . , M (M′) represents a number M (M′) of time frames (cf. horizontal m-axis in FIG. 1B). A time frame is defined by a specific time index m and the corresponding K DFT-bins (cf. indication of Time frame m in FIG. 1B). A time frame m represents a frequency spectrum of signal x at time m. A DFT-bin (k,m) comprising a (real) or complex value x(k,m) of the signal in question is illustrated in FIG. 1B by hatching of the corresponding field in the time-frequency map. Each value of the frequency index k corresponds to a frequency range Δƒ.sub.k, as indicated in FIG. 1B by the vertical frequency axis ƒ. Each value of the time index m represents a time frame. The time Δt.sub.m spanned by consecutive time indices depend on the length of a time frame (e.g. 25 ms) and the degree of overlap between neighbouring time frames (cf. horizontal t-axis in FIG. 1B).

[0138] In the present application, a number J of (non-uniform) frequency sub-bands with sub-band indices j=1, 2, . . . , J is defined, each sub-band comprising one or more DFT-bins (cf. vertical Sub-band j-axis in FIG. 1B). The j.sup.th sub-band (indicated by Sub-band j (x.sub.j(m)) in the right part of FIG. 1B) comprises DFT-bins with lower and upper indices k1(j) and k2(j), respectively, defining lower and upper cut-off frequencies of the j.sup.th sub-band, respectively. A specific time-frequency unit (j,m) is defined by a specific time index m and the DFT-bin indices k1(j)-k2(j), as indicated in FIG. 1B by the bold framing around the corresponding DFT-bins. A specific time-frequency unit (j,m) contains complex or real values of the j.sup.th sub-band signal x.sub.j(m) at time m.

[0139] FIG. 2A symbolically shows an intrusive monaural speech intelligibility predictor unit (MSIP) providing a monaural speech intelligibility predictor d based on either [0140] time domain versions s(n), x(n) (n being a time (sample) index), [0141] time-frequency band representations s(k,m), x(k,m) (k being a frequency index, m being a time (frame) index), or [0142] sub-band representations s.sub.j(m), x.sub.j(m) (j being a frequency sub-band index) of an essentially noise-free (clean) version of a target signal s and a noisy and/or processed version x of the target signal s comprising speech, respectively.

[0143] FIG. 2B shows a first embodiment an intrusive monaural speech intelligibility predictor unit (MSIP). The intrusive monaural speech intelligibility predictor unit (MSIP) is adapted for receiving a target signal comprising speech in an essentially noise-free version s(n) and in a noisy and/or processed version x(n), where n is a time index. The monaural speech intelligibility predictor unit is configured to provide as an output a final monaural speech intelligibility predictor value d indicative of a listener's (user's) perception of the noisy and/or processed version x of the target signal.

[0144] The monaural speech intelligibility predictor unit (MSIP) comprises a first input unit (IU) for providing a time-frequency representation s(k,m) of said noise-free version s of the target signal from the time variant signal s(n), and a second input unit (IU) for providing a time-frequency representation x(k,m) of the noisy and/or processed version x of the target signal from the the time variant signal x(n), k being a frequency bin index, k=1, 2, . . . , K, and m being a time index.

[0145] The monaural speech intelligibility predictor unit (MSIP) further comprises a first envelope extraction unit (AEU) for providing a time-frequency sub-band representation s.sub.j(m) of the noise-free version s of the target signal representing temporal envelopes, or functions thereof, of frequency sub-band signals s.sub.j(m) of said noise-free target signal from the time-frequency representation s(k,m), and a second envelope extraction unit (AEU) for providing a time-frequency sub-band representation x.sub.j(m) of the noisy and/or processed version x of the target signal representing temporal envelopes, or functions thereof, of frequency sub-band signals x.sub.j(m) of said noisy and/or processed version of the target signal from the time-frequency representation s(k,m), j=1, 2, . . . , J, and m being the time index.

[0146] The monaural speech intelligibility predictor unit (MSIP) further comprises a first time-frequency segment division unit (SDU) for dividing said time-frequency sub-band representation s.sub.j(m) of the noise-free version s of the target signal into time-frequency segments S.sub.m corresponding to a number N of successive samples of the sub-band signals s.sub.j(m), and a second time-frequency segment division unit (SDU) for dividing the time-frequency sub-band representation x.sub.j(n) of the noisy and/or processed version x of the target signal into time-frequency segments X.sub.m corresponding to a number N of successive samples of the sub-band signals x.sub.j(m).

[0147] The monaural speech intelligibility predictor unit (MSIP) further optionally comprises a first normalization and/or transformation unit (N/TU) adapted for providing normalized and/or transformed versions {tilde over (S)}.sub.m of the time-frequency segments S.sub.m, and optionally a second normalization and/or transformation unit (N/TU) adapted for providing normalized and/or transformed versions {tilde over (X)}.sub.m of the time-frequency segments X.sub.m.

[0148] The monaural speech intelligibility predictor unit (MSIP) further comprises an intermediate speech intelligibility calculation unit (ISIU) adapted for providing intermediate speech intelligibility coefficients d.sub.m estimating an intelligibility of the time-frequency segment X.sub.m, wherein the intermediate speech intelligibility coefficients d.sub.m are based on the essentially noise-free, optionally normalized and/or transformed, time frequency segments S.sub.m, {tilde over (S)}.sub.m, and the noisy and/or processed, optionally normalized and/or transformed, time-frequency segments X.sub.m, {tilde over (X)}.sub.m.

[0149] The monaural speech intelligibility predictor unit (MSIP) further comprises a final monaural speech intelligibility calculation unit (FSIU) for calculating a final monaural speech intelligibility predictor d estimating an intelligibility of the noisy and/or processed version x of the target signal by combining, e.g. by averaging or applying a MIN or MAX-function, the intermediate speech intelligibility coefficients d.sub.m, or a transformed version thereof, over time.

[0150] FIG. 2C shows a second embodiment an intrusive monaural speech intelligibility predictor unit (MSIP). The embodiment of FIG. 2C comprises the same functional units as described in connection with FIG. 2B. Additionally, it comprises a voice activity detector unit (VAD) for indicating whether or not or to what extent a given time-segment of the essentially noise-free version s (s′(n) in FIG. 2C) and the noisy and/or processed version x (x′(n) in FIG. 2C), respectively, of the target signal comprises or is estimated to comprise speech, and providing a voice activity control signal indicative thereof. In the embodiment of FIG. 2C, the voice activity detector unit (VAD) itself is configured to provide modified versions of the essentially noise-free version s and the noisy and/or processed version x, respectively, of the target signal comprising only time segments comprising speech or being estimated to comprise speech (in FIG. 2C denoted s(n) and x(n) respectively). Alternatively, the modified signals s(n) and x(n) may be created in respective separation units.

[0151] In order to simulate the potential decrease in intelligibility due to a hearing loss, an optional hearing loss model is included (cf. FIG. 3A, 3B, 3C). Several methods for simulating a hearing loss exist [4]. Typically, a hearing loss model builds (at least) on an audiogram containing frequency dependent hearing thresholds of a user (or representative of a type of hearing loss). The, perhaps, simplest hearing loss model consists of adding to the input signal x(n) a statistically independent noise signal, which is spectrally shaped according to the audiogram of the listener [5].

[0152] The proposed monaural, intrusive speech intelligibility predictor may be decomposed into a number of sub-stages as illustrated in FIGS. 2B and 2C and discussed above. Each sub-stage is described in more detail in the following.

Voice Activity Detection (VAD).

[0153] Speech intelligibility (SI) relates to regions of the input signal with speech activity—silence regions do no contribute to SI. Hence, the first step is to detect voice activity regions in the input signals. Since the noise-free speech signal s′(n) is available, voice activity is trivial. For example, in [3] the noise-free speech signal s′(n) was divided into successive frames. Speech-active frames were then identified as the ones with a frame-energy no less than e.g. 40 dB of the frame with maximum energy. The speech inactive frames, i.e., the ones with energy less than e.g., 40 dB of the maximum frame energy, are then discarded from both signals, x′(n) and s′(n). Let us denote the input signals with speech activity by x(n) and s(n), respectively, where n is a discrete-time index. A voice activity detector is shown in FIG. 2C as unit VAD.

Frequency Decomposition (IU) and Envelope Extraction (AEU)

[0154] The first step is to perform a frequency decomposition (cf. input unit IU in FIG. 2C) of the signals x(n) and s(n). This may be achieved in many ways, e.g., using a short-time Fourier transform (STFT), a band-pass filterbank (e.g., a Gamma-tone filter bank), etc. Subsequently, the temporal envelopes of each sub-band signal are extracted (cf. unit AEU in FIG. 2C). This may, e.g., be achieved using a Hilbert transform, or by low-pass filtering the magnitude of complex-valued STFT signals, etc.

[0155] As an example, we describe in the following how the frequency decomposition and envelope extraction can be achieved using an STFT; the described procedure is similar to the one in [3]. Let us assume, as an example, that signals are sampled with a frequency of f.sub.s=10000 Hz. First, a time-frequency representation is obtained by segmenting signals x(n) and s(n) into (e.g. 50%) overlapping, windowed frames (cf. e.g. FIG. 1A). Normally, a tapered window, e.g. a Hanning-window, is used. The window length may e.g. be 256 samples when the sample rate is 10000 Hz. Then, each frame is Fourier transformed using a fast Fourier transform (FFT) (potentially after appropriate zero-padding). The resulting DFT bins may be grouped in perceptually relevant sub-bands. For example, one could use one-third octave bands (e.g. as in [4]), but it should be clear that any other sub-band division can be used (for example, the grouping could be uniform, i.e., unrelated to perception in this respect, cf. FIG. 1B). In the case of one-third octave bands and a sampling rate of 10000 Hz, there are 15 bands which cover the frequency range 150-5000 Hz. Other numbers of bands and another frequency range can be used. We refer to the time-frequency tiles defined by these frames and sub-bands as time-frequency (TF) units (or STFT coefficients), cf. FIG. 1B. Applying this to the noisy/processed input signal x(n) leads to (generally complex-valued) STFT coefficients x(k,m), where k and m denote frequency and frame (time) indices, respectively. STFT coefficients s(k,m) for the noise-free reference speech signal are found in a similar manner.

[0156] Temporal envelope signals may then be extracted as

[00010]$x_j\left(m\right)=f\left(\sqrt{\underset{k=k.Math..Math.1.Math.\left(j\right)}{\overset{k.Math..Math.2.Math.\left(j\right)}{.Math.}}.Math..Math.{.Math.x\left(k,m\right).Math.}^2}\right),j=1,\mathrm{.Math.}.Math..Math.J,.Math.\mathrm{and}$$m=1,\mathrm{.Math.}.Math..Math.M,$

where k1(j) and k2(j) denote DFT bin indices corresponding to lower and higher cut-off frequencies of the j.sup.th sub-band, J is the number of sub-bands (e.g. 16), and M is the number of signal frames in the signal in question, and where the function ƒ(x) is included for generality. For example, for ƒ(x)=x, we get the temporal envelope used in [4], with ƒ(x)=x.sup.2, we extract power envelopes, and with ƒ(x)=2 log x, or ƒ(x)=x.sup.β, 0<β<2, we can model the compressive non-linearity of the healthy cochlea, respectively. It should be clear that other reasonable choices for ƒ(x) exist. Temporal envelope signals s.sub.j(m) for the noise-free speech signal are found in a similar manner. The same choice of ƒ(x) may be used in both cases.

[0157] As mentioned, other envelope representations may be implemented, e.g., using a Gammatone filterbank, followed by a Hilbert envelope extractor, etc., and functions ƒ(x) may be applied to these envelopes in a similar manner as described above for STFT based envelopes. In any case, the result of this procedure is a time-frequency representation in terms of sub-band temporal envelopes, x.sub.j(m) and s.sub.j(m), where j is a sub-band index, and m is a time index.

Time-Frequency Segments (SDU)

[0158] Next, we divide the time-frequency representations x.sub.j(m) and s.sub.j(m) into segments, i.e., spectrograms corresponding to N successive samples of all sub-band signals. For example, the m.sup.th segment for the noisy/processed signal is defined by the J×N matrix

[00011]$X_m=\left[\begin{array}{ccc}{x}_1\left(m-N+1\right)& \mathrm{.Math.}& x_1\left(m\right)\\ \mathrm{.Math.}& & \mathrm{.Math.}\\ x_J\left(m-N+1\right)& \mathrm{.Math.}& x_J\left(m\right)\end{array}\right].$

[0159] The corresponding segment S.sub.m for the noise-free reference signal is found in an identical manner.

[0160] It should be understood that other versions of the time-segments could be used, e.g., segments, which have been shifted in time to operate on frame indices m−N/2+1 through m+N/2.

Normalizations and Transformation of Time-Frequency Segments (N/TU)

[0161] The rows and columns of each segment X.sub.m and S.sub.m may be normalized/transformed in various ways (below, we show the normalizations/transformations as applied to X.sub.m; they are applied to S.sub.m in a completely analogously manner. The same normalization/transformation is applied to both X.sub.m and S.sub.m). In particular, we consider the following row (R) normalizations/transformations

R1) Normalization of rows to zero mean:

g.sub.1(X)=X−μ.sub.x.sup.r1.sup.T,

where μ.sub.x.sup.r is a J×1 vector whose j.sup.th entry is the mean of the j.sup.th row of X (hence the superscript r in μ.sub.x.sup.r), and where 1 denotes an N×1 vector of ones.
R2) Normalization of rows to unit-norm:

g.sub.2(X)=D.sup.r(X)X,

where

D.sup.r(X)=diag(└1/√{square root over (X(1,:)X(1,:).sup.H)} . . . 1/√{square root over (X(J,:)X(J,:).sup.H)}┘),

and where diag(•) is a diagonal matrix with the elements of the arguments on the main diagonal. Furthermore, X(j,:) denotes the j.sup.th row of X, such that D′(X) is a J×J diagonal matrix with the inverse norm of each row on the main diagonal, and zeroes elsewhere (the superscript H denotes Hermitian transposition). Pre-multiplication with D′(X) normalizes the rows of the resulting matrix to unit-norm.
R3) Fourier transformation applied to each row

g.sub.3(X)=XF,

where F is an N×N Fourier matrix.
R4) Fourier transformation applied to each row followed by computing the magnitude of the resulting complex-valued elements

g.sub.4(X)=|XF|,

where |•| computes the element-wise magnitudes.
R5) The identity operator

g.sub.5(X)=X.

[0162] We consider the following column (C) normalizations

C1) Normalization of columns to zero mean:

h.sub.1(X)=X−1μ.sub.x.sup.c.sup.T,

where μ.sub.x.sup.c is a N×1 vector whose i.sup.th entry is the mean of the i.sup.th row of X, and where 1 denotes an J×1 vector of ones.
C2) Normalization of columns to unit-norm:

h.sub.2(X)=XD.sup.c(X), where

D.sup.c(X)=diag(└1/√{square root over (X(:,1).sup.HX(:,1))} . . . 1/√{square root over (X(:,N).sup.HX(:,N))}┘).

[0163] Here X(:,n) denotes the n.sup.th row of X, such that D′(X) is a diagonal N×N matrix with the inverse norm of each column on the main diagonal, and zeros elsewhere. Post-multiplication with D.sup.c(X) normalizes the rows of the resulting matrix to unit-norm.

[0164] The row—(R#, #=1, 2, . . . , 5) and column (C#, #=1, 2) normalizations/transformations listed above may be combined in different ways. In a preferred embodiment, at least one of row normalizations/transformations g.sub.i(•) (i=1, 2, . . . , 5) and at least one of the column normalizations/transformations h.sub.j(•) (j=1, 2) is applied (in any order).

[0165] One combination of particular interest is where, first, the rows are normalized to zero-mean and unit-norm, followed by a similar mean and norm normalization of the columns. This particular combination may be written as

{tilde over (X)}.sub.m=h.sub.2(h.sub.1(g.sub.2(g.sub.1(X.sub.m))),

where X.sub.m is the resulting row- and column normalized matrix.

[0166] Another transformation of interest is to compute the magnitude Fourier spectrum of each row of matrix X.sub.m followed by mean- and norm-normalization of the resulting columns. With the introduced notation, this may be written simply as

{tilde over (X)}.sub.m=h.sub.2(h.sub.1(g.sub.3(X.sub.m))).

[0167] Other combinations of these normalizations/transformations may be of interest, e.g.,

{tilde over (X)}.sub.m=g.sub.2(g.sub.1(h.sub.2(h.sub.1(X.sub.m))))

(mean- and norm-standardization of the columns followed by mean- and norm-standardization of the rows), etc. As mentioned, a particular combination of row- and column-normalizations/transformations is chosen and applied to all segments X.sub.m and S.sub.m of the noisy/processed and noise-free signal, respectively.

Estimation of Intermediate Intelligibility Coefficients (ISIU)

[0168] The time-frequency segments S.sub.m or the normalized/transformed time-frequency segments {tilde over (S)}.sub.m of the noise-free reference signal may now be used together with the corresponding noisy/processed segments X.sub.m, {tilde over (X)}.sub.m to compute an intermediate intelligibility index d.sub.m, reflecting the intelligibility of the noisy/processed signal segment X.sub.m, {tilde over (X)}.sub.m. To do so, let us first define the sample correlation coefficient d(x,y) of the elements in two K×1 vectors x and y:

[00012]$d\left(x,y\right)=\frac{\underset{k=1}{\overset{K}{.Math.}}.Math..Math.\left(x\left(k\right)-{\mu }_x\right).Math.\left(y\left(k\right)-{\mu }_y\right)}{\sqrt{.Math.\underset{k=1}{\overset{K}{.Math.}}.Math.{\left(x\left(k\right)-{\mu }_x\right)}^2.Math.{\left(y\left(k\right)-{\mu }_y\right)}^2}},.Math.\mathrm{where}$${\mu }_x=\frac{1}{K}.Math.\underset{k=1}{\overset{K}{.Math.}}.Math..Math.x\left(k\right).Math..Math.\mathrm{and}.Math..Math.{\mu }_y=\frac{1}{K}.Math.\underset{k=1}{\overset{K}{.Math.}}.Math.y\left(k\right).$

[0169] Several options exist for computing the intermediate intelligibility index d.sub.m. In particular, d.sub.m may be defined as [0170] 1) the average sample correlation coefficient of the columns in {tilde over (S)}.sub.m and {tilde over (X)}.sub.m, i.e.,

[00013]$d_m=\frac{1}{N}.Math.\underset{n=1}{\overset{N}{.Math.}}.Math..Math.d\left({\tilde{S}}_m\left(:,n\right),{\tilde{X}}_m\left(:,n\right)\right),$

or [0171] 2) the average sample correlation coefficient of the rows in {tilde over (S)}.sub.m and {tilde over (X)}.sub.m, i.e.,

[00014]$d_m=\frac{1}{J}.Math.\underset{j=1}{\overset{J}{.Math.}}.Math..Math.d\left({{\tilde{S}}_m\left(j,:\right)}^T,{{\tilde{X}}_m\left(j,:\right)}^T\right),$

or [0172] 3) the sample correlation coefficient of all elements in {tilde over (S)}.sub.m and {tilde over (X)}.sub.m, i.e.,

d.sub.m=d({tilde over (S)}.sub.m(:),{tilde over (X)}.sub.m(:)),

where we adopted the notation {tilde over (S)}.sub.m (:) and {tilde over (X)}.sub.m(:) to represent NJ×1 vectors formed by stacking the columns of the respective matrices.

Estimation of Final Intelligibility Coefficient (FSIU)

[0173] The final intelligibility coefficient d, which reflects the intelligibility of the noisy/processed input signal x(n), is defined as the average of the intermediate intelligibility coefficients, potentially transformed via a function u(d.sub.m), across the duration of the speech-active parts of x(n), i.e.,

[00015]$d=\frac{1}{M}.Math.\underset{m=1}{\overset{M}{.Math.}}.Math..Math.u\left(d_m\right).$

[0174] The function u(d.sub.m) could for example be

[00016]$u\left(d_m\right)=\log \left(\frac{1}{1-d_m^2}\right),$

to link the intermediate intelligibility coefficients to information measures, but it should be clear that other choices exist.

[0175] The “do-nothing” function u(d.sub.m)=d.sub.m is also a possible choice (it has previously been used in the STOI algorithm [3]).

[0176] In the following, a noisy/reverberant speech signal x(n) which potentially has been passed through a signal processing device, e.g. in a hearing aid, is considered. An algorithm is proposed, which can predict the average intelligibility of x(n), as perceived by a group of listeners with similar hearing profiles, e.g. normal hearing or hearing impaired listeners. To achieve this, the proposed algorithm relies on the presence of the noise-free, undistorted underlying signal s(n), see FIG. 3A. FIG. 3A shows an intrusive monaural speech intelligibility predictor unit in combination with a hearing loss model (HLM) and an evaluation unit (MSIP) (together constituting a modified monaural speech intelligibility predictor unit (MSIP′)). In the embodiment of FIG. 3A, the signal x′(n) is passed through hearing loss model (HLM) configured to model the imperfections of an impaired auditory system (e.g. the impaired auditory system of a particular user). The hearing loss model unit (HLM) is e.g. based on an audiogram of an ear of a user (and possible other data related to a user's hearing ability). The hearing loss model unit (HLM) is e.g. configured to apply a frequency dependent modification of the noisy and/or processed version x′ of the target signal reflecting a deviation from normal hearing, e.g. a hearing impairment, of a relevant ear of the user to provide a modified noisy and/or processed version x of the target signal for use together with the essentially noise-free version s of the target signal as inputs to the monaural speech intelligibility predictor unit (MSIP) providing the final monaural speech intelligibility predictor d. In FIG. 3A an evaluation unit (EVAL) is shown to receive and evaluate the speech intelligibility predictor d and provide a processed predictor d′. The evaluation unit (EVAL) may e.g. further process the speech intelligibility predictor value d, to e.g. graphically and/or numerically display the current and/or recent historic values, derive trends, etc. Alternatively, or additionally the evaluation unit may propose actions to the user (or a communication partner or caring person), such as add directionality, move closer, speak louder, activate SI-enhancement mode, etc. The evaluation unit may e.g. be implemented in a separate device, e.g. acting as a user interface to the speech intelligibility predictor unit (MSIP) and/or to a hearing aid including such unit, e.g. implemented as a remote control devise, e.g. as an APP of a smartphone (cf. FIG. 6A, 6B).

[0177] FIG. 3B shows an intrusive monaural speech intelligibility predictor unit (MSIP) in combination with a signal processor (SPU) and an evaluation unit (EVAL), e.g. of a hearing device. Compared to FIG. 3A, the embodiment of FIG. 3B additionally comprises a number of input units (here 2: M1, M2), e.g. microphones, for providing a time-variant electric input signal representing a sound input received at the input unit in question. At least one (such as each) of the electric input signals comprises a target signal component (e.g. a speech component) and a noise signal component (termed Noisy target in FIG. 3B). The target signal component is assumed to originate from a target signal source in the environment of the device (e.g. a hearing device, see FIG. 3C). The embodiment of FIG. 3B further comprises a configurable signal processor (SPU) for processing the electric input signals (e.g. providing beamforming and/or noise reduction, frequency and level dependent amplification, level dependent compression, or the like) and providing a processed signal x′ based on one or more of the electric input signals, which are inputs to the configurable signal processor (SPU). The processed signal x′ from the configurable signal processor (SPU) is fed to the hearing loss model (HLM) unit of the monaural speech intelligibility predictor unit (MSIP). The hearing loss model unit (HLM) comprises a model of a hearing loss of a user (e.g. the user of the device) and is configured to shape an input signal to provide an output signal x representing a processed (possibly hearing loss compensated) and (again) deteriorated signal, which is fed to the monaural speech intelligibility predictor (MSIP). The embodiment of FIG. 3B further comprises an antenna and transceiver unit (Rx) for receiving a wireless signal (termed Clean target in FIG. 3B) comprising the target signal and for extracting an essentially noise-free version s of the target signal, which is connected to the monaural speech intelligibility predictor (MSIP). The final speech intelligibility predictor d from the monaural speech intelligibility predictor unit (MSIP) is fed to the evaluation unit (EVAL) whose modified predictor value d′ is fed to the configurable signal processor (SPU). The configurable signal processor (SPU) is adapted to control or influence the processing of the respective electric input signals based on the final speech intelligibility predictor d provided by the monaural speech intelligibility predictor unit and as modified by the evaluation unit (EVAL). The configurable signal processor (SPU) is adapted to control or influence the processing of the respective electric input signals to maximize the final speech intelligibility predictor d. (e.g. controlled by the evaluation unit (EVAL (max)).

[0178] The embodiment of FIG. 3B may e.g. further comprise an output unit for creating output stimuli configured to be perceivable by the user as sound based on an electric output either in the form of the processed signal x′ from the signal processor or a signal derived therefrom. The output unit (cf. e.g. OT in FIG. 3C) may e.g. comprise a loudspeaker for placement in an ear canal of a user, or a vibrator for being attached to the skull of a user, or electrodes for placement in cochlea of a user. Thereby a hearing aid according to the present disclosure is provided. The hearing aid may take the form or an air conducting hearing instrument, a bone-conducting hearing instrument, a cochlear implant prosthesis, an active ear-protection device, a headset, an earphone with active noise cancellation, etc.

[0179] FIG. 3C shows a first embodiment of a hearing device (HD, e.g. a hearing aid) comprising an intrusive monaural speech intelligibility predictor unit (MSIP′) comprising a hearing loss model part (HLM) and a predictor part (MSIP) configured to optimize a user's intelligibility (represented by index d) of an output signal u of the hearing device (HD). The embodiment of FIG. 3C is equivalent to the embodiment of FIG. 3B but further comprises an output unit comprising an output transducer (OT) in the form of a loudspeaker, which is directly connected to the output u of the signal processor (SPU). Further, the hearing aid of FIG. 3C only comprises one the input unit (IT) comprising a microphone for picking up a noisy representation y′ of the target signal hearing aid and converting it to an electric input signal y, which is fed to the configurable signal processor (SPU). As in FIG. 3B, the antenna and transceiver unit (Rx) is adapted for receiving a wireless signal (termed s′ in FIG. 3C) comprising the target signal and for extracting an essentially noise-free version s of the target signal, which is fed to the predictor part (MSIP) of the intrusive monaural speech intelligibility predictor unit (MSIP′). The monaural speech intelligibility predictor unit (MSIP) provides an estimate of the intelligibility of the output signal by the user in the form of the (final) speech intelligibility predictor d, which is fed to a control part of the configurable signal processor (SPU) to modify signal processing to optimize d. in a feedback loop.

[0180] FIG. 4A shows a first scenario for using a hearing aid (HD) comprising an intrusive monaural speech intelligibility predictor according to the present disclosure (as described in connection with FIG. 2A, 2B, 2C and FIG. 3A, 3B, 3C above) to improve a hearing aid user's (U) intelligibility of speech from a speaker (TLK) wearing a wireless microphone (M), e.g. in a teaching or lecture situation. The speaker's voice (the target signal) is picked up by the microphone (M) located close to the speaker's mouth. The microphone (M) comprises a transmitter (Tx) for wirelessly transmitting the essentially noise-free version of the target signal s to a corresponding receiver (Rx) of the hearing aid worn by the user (U) using wireless link WLS (e.g. using FM or Bluetooth or other standardized or proprietary technology). Simultaneously, an acoustically propagated version of the target signal coloured (modified) by the location (e.g. a room with reflecting surfaces, e.g. walls) and mixed with possible noise (noise) from the environment is picked up (noisy signal x) by one or more microphone of the hearing aid (HD).

[0181] FIG. 4B shows a second (similar) scenario for using a hearing aid (HD) comprising an intrusive monaural speech intelligibility predictor according to the present disclosure to improve a hearing aid user's (U) intelligibility of speech from a remote speaker of a telephone conversation using a handsfree telephone set in a car (CAR), where remote sound is wirelessly as well as acoustically transmitted to the hearing aid user. The ‘clean’ target signal of the remote speaker received by the telephone (CELL PHONE) is wirelessly transmitted to and received by a receiver (Rx) of the hearing aid (HD) as clean signal s (Clean target s in FIG. 4B), while the sound is simultaneously propagated to a microphone (IT) of the hearing aid by a loudspeaker (LOUDSPEAKER) of the handsfree telephone set providing ‘noisy signal’ x (Noisy target x in FIG. 4B). The user is driving a car (CAR, WHEEL) while talking in a telephone via a handsfree telephone set. The noise in the car cabin (indicated by noise sources N1, N2) is acoustically mixed with the ‘target signal’ played by a loudspeaker of the handsfree telephone set (or of the telephone itself) and picked up as noisy target signal x, by the hearing aid microphone(s) (IT).

[0182] The hearing aid (HD) used in the two scenarios of FIG. 4A, 4B may be a hearing aid according to the present disclosure, e.g. as described in connection with FIG. 3A, 3B, 3C, which is configured to adapt the processing of an acoustic signal picked up by a microphone of the hearing aid and processed by a signal processing device to optimize the user's speech intelligibility (based on a predictor of a monaural speech intelligibility predictor unit, as proposed by the present disclosure).

[0183] FIG. 5A shows a third scenario for using a hearing aid (HD) comprising an intrusive monaural speech intelligibility predictor according to the present disclosure to improve a hearing aid user's intelligibility of speech from a remote speaker of a telephone conversation using a handsfree telephone set in a car (CAR, WHEEL), where sound from a remote communication partner is wirelessly transmitted to the hearing aid user (U). The scenario of FIG. 5A is similar to the scenario of FIG. 4B apart from the fact that in the scenario of FIG. 5A the voice of the remote communication partner is NOT played by a loudspeaker in the car.

[0184] The clean target signal s is transmitted from the CELL PHONE to the hearing aid HD. The background noise v′ (Noise v′) of the car cabin is captured by the microphone(s) (IT) of the hearing aid. It can be assumed that the background noise v′ as captured is substantially equal to the noise v.sub.ed (Noise v.sub.ed) that is present at the ear drum (Ear drum) of the user (cf. FIG. 5B, 5C). The assumption is of course better the closer to the ear drum the microphone is situated (and/or the more open the ear canal part is). In an embodiment, a microphone of the hearing aid is located in the ear canal, e.g. at the entrance of the ear canal or close to the ear drum (cf. e.g. IT.sub.3 in FIG. 5C).

[0185] FIG. 5B shows an embodiment of a hearing aid comprising an intrusive monaural speech intelligibility predictor for use in the (third) scenario of FIG. 5A. The embodiment of a hearing aid (HD) shown in FIG. 5B comprises the same functional components as the embodiment shown in FIG. 3C. The interconnection of the signal processor (SPU) and the wireless receiver (Rx) and the input transducer (IT) is different, however. As shown in FIG. 5A and as indicated in FIGS. 5B and 5C, the sound output of the loudspeaker (OT) of the hearing aid, equal to the processed signal ƒ(s) from the signal processor (SPU), is acoustically mixed with ‘environmental’ (car cabin) noise v, at the eardrum (Ear drum, cf. Mixture of s and v.sub.ed in FIG. 5B).

[0186] The basic idea of the embodiment of a hearing aid in FIG. 5B is to process the clean version s of the target signal so that the speech intelligibility d is maximized when the processed version of the clean target signal (ƒ(s)). The processed version of the clean target signal (ƒ(s)) can be adaptively controlled, whereas this is not the case for the car cabin noise v.sub.ed at the eardrum (which is given).

[0187] Using a model of speech intelligibility (e.g. as disclosed in the present disclosure) in the configuration of FIG. 5B, an estimate of a user's intelligibility of speech components present in the processed signal ƒ(s) from the signal processor (SPU), when presented to the ear by a loudspeaker (or an acoustic guide element of the hearing aid) and mixed with the noise signal v.sub.ed present at the ear drum, can be determined, cf. signal u=ƒ(s)+v, which is used as the ‘processed/noisy target signal input’ to the hearing loss model (HLM) of the monaural speech intelligibility predictor unit (MSIP′) in FIG. 5B.

[0188] Preferably, the loudspeaker (or alternatively an acoustic guide element) is located in the ear canal, preferably close to the ear drum to deliver the processed signal ƒ(s) to the ear drum. Preferably, the microphone(s) of the hearing device, which is(are) used to pick up background noise v′ (cf. FIG. 5A, 5B), is(are) located close to the ear drum, or at the entrance of the ear canal, or in pinna, or behind the ear. In an embodiment, the noise signal picked up by a given microphone is appropriately modified (in the hearing aid) to account for a transfer function for an acoustic signal from the location of the microphone to the ear drum to thereby give a better estimate of the background noise v.sub.ed received by the user (at the ear drum) for use in the estimate of the speech intelligibility d=d(ƒ(s)+v,s).

[0189] In the configuration of FIG. 5B, the signal processor (SPU) is configured to iteratively modify signal processing of the clean target signal s received from wireless receiver unit (Rx) to provide processed version ƒ(s) the clean target signal s that optimizes speech intelligibility of the (mixed) signal present at the ear drum of the user (in practice here approximated by maximizing the monaural speech intelligibility predictor d(ƒ(s)+v,s) according to the resent disclosure).

[0190] As an alternative to using a speech intelligibility predictor to modify (optimize) s (or as an extreme option of the present disclosure), a simple increase of gain of the clean target signal s (i.e. f(s)=g's, g being a gain factor, e.g. g=10) may be used to increase the signal to noise ratio (SNR) at the ear drum (assuming a constant level of the background (cabin) noise v.sub.ed at the ear drum). In practice, such reliance only on increasing gain of the clean target signal may, however, not be attractive or possible (e.g. due to acoustic feedback problems, maximum power output limitations of the loudspeaker, or uncomfortable levels of the user, etc.). Instead an appropriate frequency dependent shaping of the clean target signal is generally proposed and governed by the monaural speech intelligibility predictor (including the hearing loss model (HLM) preferably defining decisive aspects of a hearing impairment of the user of the hearing aid).

[0191] FIG. 5C illustrates an exemplary hearing aid (HD) formed as a receiver in the ear (RITE) type of hearing aid comprising a part (BTE) adapted for being located behind pinna and a part (ITE) comprising an output transducer (OT, e.g. a loudspeaker/receiver) adapted for being located in an ear canal (Ear canal) of the user (e.g. exemplifying a hearing aid (HD) as shown in FIG. 5A, 5B). The BTE-part (BTE) and the ITE-part (ITE) are connected (e.g. electrically connected) by a connecting element (IC). In the embodiment of a hearing aid of FIG. 5C, the BTE part (BTE) comprises two input units comprising two (individually selectable) input transducers (e.g. microphones) (IT.sub.1, IT.sub.2) each for providing an electric input audio signal representative of an input sound signal from the environment (in the scenario of FIG. 5A, from the car cabin). The hearing device of FIG. 5C further comprises two (individually selectable) wireless receivers (WLR.sub.1, WLR.sub.2) for providing respective directly received auxiliary audio and/or information signals. The hearing aid (HD) further comprises a substrate (SUB) whereon a number of electronic components are mounted, including a configurable signal processor (SPU), a monaural speech intelligibility predictor unit (MSIP), and a hearing loss model unit (HLM, coupled to each other and input and output units via electrical conductors Wx). The configurable signal processor (SPU) provides an enhanced audio signal (cf. signal ƒ(s) in FIG. 5B), which is intended to be presented to a user. In the embodiment of a hearing aid device in FIG. 5C, the ITE part (ITE) comprises an output unit in the form of a loudspeaker (receiver) (OT) for converting an electric signal (ƒ(s) in FIG. 5B) to an acoustic signal. The ITE-part further comprises an input unit comprising an input transducer (e.g. a microphone) (IT.sub.3) for providing an electric input audio signal representative of an input sound signal from the environment in the ear canal (here approximating the noise v.sub.ed from the car cabin at the ear drum (Ear drum) of the user (U) wearing the hearing aid (HD)). In other embodiments, the hearing aid may comprise only the input unit (IT.sub.3) located in or at the ear canal, or the input unit (IT.sub.3) located in or at the ear canal in combination with a an input unit located elsewhere, e.g. in a BTE-part. The ITE-part further comprises a guiding element, e.g. a dome, (DO) for guiding and positioning the ITE-part in the ear canal of the user.

[0192] The hearing aid (HD) exemplified in FIG. 5C is a portable device and further comprises a battery (BAT) for energizing electronic components of the BTE- and ITE-parts.

[0193] In an embodiment, the hearing aid (HD) comprises a directional microphone system (beamformer) adapted to enhance a target acoustic source among a multitude of acoustic sources in the local environment of the user wearing the hearing aid device. In an embodiment, the directional system is adapted to detect (such as adaptively detect) from which direction a particular part of the microphone signal originates.

[0194] The hearing aid of FIG. 5C may form part of a hearing aid and/or a binaural hearing aid system according to the present disclosure (cf. e.g. FIG. 7).

[0195] FIG. 6A shows an embodiment of a binaural hearing system comprising left and right hearing devices (HD.sub.left, HD.sub.right), e.g. hearing aids, in communication with a portable (handheld) auxiliary device (Aux) functioning as a user interface (UI) for the binaural hearing aid system (cf. FIG. 6B). In an embodiment, the binaural hearing system comprises the auxiliary device (Aux, and the user interface UI). In the embodiment of FIG. 6A, wireless links denoted IA-WL (e.g. an inductive link between the left and right hearing devices) and WL-RF (e.g. RF-links (e.g. Bluetooth) between the auxiliary device Aux and the left HD.sub.left, and between the auxiliary device Aux and the right HD.sub.right, hearing aid, respectively) are indicated (implemented in the devices by corresponding antenna and transceiver circuitry, indicated in FIG. 6A in the left and right hearing devices as RF-IA-Rx/Tx-l and RF-IA-Rx/Tx-r, respectively).

[0196] FIG. 6B shows the auxiliary device (Aux) comprising a user interface (UI) in the form of an APP for controlling and displaying data related to the speech intelligibility predictors. The user interface (U) comprises a display (e.g. a touch sensitive display) displaying a screen of a Speech intelligibility SI-APP for controlling the hearing aid system and presenting information to the user. The APP comprises a number of predefined action buttons regarding functionality of the binaural (or monaural) hearing system. In the exemplified (part of the) APP, a user (U) has the option of influencing a mode of operation via the selection of a SI-prediction mode to be a Monaural SIP or Binaural SIP mode. In the screen shown in FIG. 10B, the un-shaded buttons are selected, i.e. SI-enhancement mode (where the processing is adapted to optimize speech intelligibility based on the (monaural or binaural) speech intelligibility predictor) together with a specific Car telephony mode (as described in connection with FIG. 5A, 5B, 5C). Further, a show Current SI-estimate has been activated (resulting in a current predicted value of the binaural speech intelligibility predictor being displayed (in the form of the positive indicator ‘’) together with an indication of the current noise level (indicated as ‘HIGH’)). The grey shaded button Lecture mode (as described in connection with FIG. 4A, 4B) may be selected instead of Car telephony mode.

[0197] FIG. 7 shows an embodiment of a binaural hearing aid system according to the present disclosure comprising a left and right hearing devices (HD.sub.left, HD.sub.right), each comprising a monaural speech intelligibility predictor unit (MSIP′) whose individual predictor values di and d.sub.right are exchanged between the hearing devices and used to influence or control signal processing of respective signal processors (SPU) in the hearing devices to optimize binaural speech intelligibility of the user. The left and right hearing devices (HD.sub.left, HD.sub.right), are e.g. hearing devices as shown in an discussed in connection with FIG. 3C. Each of the left and right hearing aids comprises antenna and transceiver circuitry (IA-Rx/Tx) for allowing a communication link (IA-WLS) to be established and information (d.sub.left, d.sub.right) to be exchanged between said left and right hearing aids. In an embodiment, the binaural hearing aid system comprises a binaural speech intelligibility prediction unit for providing a final binaural speech intelligibility measure d.sub.binaural of the predicted speech intelligibility of the user when exposed to a sound input, wherein the final binaural speech intelligibility measure d.sub.binaural is determined in dependence of the final monaural speech intelligibility predictor values d.sub.left, d.sub.right of the respective left and right hearing aids. The binaural speech intelligibility prediction unit may e.g. be implemented in one or both of the signal processors (SPU) of the left and right hearing devices. In an embodiment, the final binaural speech intelligibility measure d.sub.binaural is determined as the maximum of the speech intelligibility predictor values d.sub.left, d.sub.right of the respective left and right hearing aids: d.sub.binaural=max(d.sub.left, d.sub.right).

[0198] It is intended that the structural features of the devices described above, either in the detailed description and/or in the claims, may be combined with steps of the method, when appropriately substituted by a corresponding process.

[0199] As used, the singular forms “a,” “an,” and “the” are intended to include the plural forms as well (i.e. to have the meaning “at least one”), unless expressly stated otherwise. It will be further understood that the terms “includes,” “comprises,” “including,” and/or “comprising,” when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. It will also be understood that when an element is referred to as being “connected” or “coupled” to another element, it can be directly connected or coupled to the other element but an intervening elements may also be present, unless expressly stated otherwise. Furthermore, “connected” or “coupled” as used herein may include wirelessly connected or coupled. As used herein, the term “and/or” includes any and all combinations of one or more of the associated listed items. The steps of any disclosed method is not limited to the exact order stated herein, unless expressly stated otherwise.

[0200] It should be appreciated that reference throughout this specification to “one embodiment” or “an embodiment” or “an aspect” or features included as “may” means that a particular feature, structure or characteristic described in connection with the embodiment is included in at least one embodiment of the disclosure. Furthermore, the particular features, structures or characteristics may be combined as suitable in one or more embodiments of the disclosure. The previous description is provided to enable any person skilled in the art to practice the various aspects described herein. Various modifications to these aspects will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other aspects.

[0201] The claims are not intended to be limited to the aspects shown herein, but is to be accorded the full scope consistent with the language of the claims, wherein reference to an element in the singular is not intended to mean “one and only one” unless specifically so stated, but rather “one or more.” Unless specifically stated otherwise, the term “some” refers to one or more.

[0202] Accordingly, the scope should be judged in terms of the claims that follow.

REFERENCES

[0203] [1] American National Standards Institute, “ANSI S3.5, Methods for the Calculation of the Speech Intelligibility Index,” New York 1995. [0204] [2] K. S. Rhebergen and N. J. Versfeld, “A speech intelligibility index based approach to predict the speech reception threshold for sentences in fluctuating noise for normal-hearing listeners,” J. Acoust. Soc. Am., vol. 117, no. 4, pp. 2181-2192, 2005. [0205] [3] C. H. Taal, R. C. Hendriks, R. Heusdens, and J. Jensen, “An Algorithm for Intelligibility Prediction of Time-Frequency Weighted Noisy Speech.” IEEE Trans. Audio, Speech, Lang. Process., vol. 19, no. 7, pp. 2125-2136, September 2011. [0206] [4] B. C. J. Moore, “Cochlear Hearing Loss,” Physiological, Psychological and Technical Issues, “Wiley, 2007. [0207] [5] R. Beutelmann and T. Brand, “Prediction of intelligibility in spatial noise and reverberation for normal-hearing and hearing-impaired listeners,” J. Acoust. Soc. Am., Vol. 120, no. 1, pp. 331-342, April 2006.