Methods for processing audio signals and circuit arrangements therefor
09768829 · 2017-09-19
Assignee
Inventors
- Christelle Yemdji (Antibes, FR)
- Nicholas Evans (Valbonne, FR)
- Christophe Beaugeant (Mouans Sartoux, FR)
- Ludovick Lepauloux (Cannes, FR)
Cpc classification
H04B3/20
ELECTRICITY
International classification
H04B3/20
ELECTRICITY
Abstract
A method for processing audio signals is provided comprising outputting an audio signal; receiving the output audio signal via a first receiving path as a first received audio signal; receiving the output audio signal via a second receiving path as a second received audio signal; determining an echo suppression gain based on the first received audio signal and the second received audio signal; and filtering echo suppression of the audio signal based on the first received audio signal and the echo suppression gain.
Claims
1. A method for processing audio signals, the method comprising: outputting an audio signal; receiving the output audio signal via a first receiving path as a first received audio signal; receiving the output audio signal via a second receiving path as a second received audio signal; determining a residual echo power based on the first received audio signal and the second received audio signal; determining an echo suppression gain based on the residual echo power and an estimate of a signal-to-echo ratio of at least a selected received audio signal from the first received audio signal and the second received audio signal; and filtering echo suppression of the audio signal based on the first received audio signal and the echo suppression gain.
2. The method of claim 1, wherein determining an echo suppression gain based on the first audio signal comprises: filtering echo of the first audio signal to produce a first echo error signal; determining the echo suppression gain based on the first echo error signal; and filtering echo suppression of the audio signal based on the first echo error signal.
3. The method of claim 1, wherein determining an echo suppression gain based on the second audio signal comprises: filtering echo of the second audio signal to produce a second echo error signal; and determining the echo suppression gain based on the second echo error signal.
4. The method of claim 2, wherein the filtering of echo comprises an adaptive echo filtering.
5. The method of claim 1, wherein the filtering echo suppression of the audio signal includes ignoring the second received audio signal.
6. The method of claim 1, wherein outputting an audio signal comprises outputting an audio signal via a loudspeaker.
7. The method of claim 1, wherein receiving the audio signal via a first receiving path as a first audio signal comprises receiving the audio signal via a first microphone; and wherein receiving the output audio signal via a second receiving path as a second received audio signal comprises receiving the output audio signal via a second microphone.
8. The method of claim 1, further comprising: beamforming at least one of the audio signal and the audio signal after echo suppression.
9. A method for processing audio signals, the method comprising: outputting an audio signal; determining an echo suppression gain based on a multichannel audio signal information representing received audio signals which are received via different receiving paths; wherein determining the echo suppression gain is based on an estimate of at least a signal-to-echo ratio of a selected set of received audio signals of the received audio signals and a second echo error signal of the received signal not selected for the estimate of the signal-to-echo ratio; and filtering echo suppression of the audio signal based on single-channel audio signal information representing the received audio signal received via a single receiving path and the determined echo suppression gain.
10. The method of claim 9, further comprising: wherein the multichannel audio signal information comprises echo filtering information of the received audio signals.
11. The method of claim 9, further comprising: wherein the multichannel audio signal information comprises multichannel echo filtering information of the received audio signals.
12. The method of claim 9, further comprising: wherein the filtering of echo comprises an adaptive echo filtering.
13. The method of claim 9, wherein outputting an audio signal comprises outputting an audio signal via a loudspeaker.
14. The method of claim 9, wherein the received audio signals are received via at least a first microphone and a second microphone.
15. The method of claim 9, further comprising: beamforming at least one of the audio signal and the audio signal after echo suppression.
16. A circuit arrangement for processing audio signals comprising: an audio device for outputting an audio signal; a first receiving path configured to receive the output audio signal as a first received audio signal; a second receiving path configured to receive the output audio signal as a second received audio signal; a determiner circuit configured to determine a residual echo power based on the first received audio signal and the second received audio signal; wherein the determiner circuit is further configured to determine an echo suppression gain based on the residual echo power and an estimate of at least a signal-to-echo ratio of a selected received audio signal from the first received audio signal and the second received audio signal; and an echo suppression filter coupled to the first receiving path and the determiner circuit configured to suppress echo of the audio signal based on the first received audio signal and the echo suppression gain.
17. The circuit arrangement of claim 16, further comprising: at least one echo filter configured to filter the first received audio signal to produce a first echo error signal; the determiner circuit configured to determine the echo suppression gain based on the first echo error signal; and the echo suppression filter configured to perform echo suppression of the audio signal based on the first echo error signal.
18. The circuit arrangement of claim 17, further comprising: at least one echo filter configured to filter the second audio signal to produce a second echo error signal; the determiner circuit configured to determine the echo suppression gain based on the second echo error signal.
19. The circuit arrangement of claim 17, wherein the at least one echo filter comprises an adaptive echo filter.
20. The circuit arrangement of claim 16, the echo suppression filter configured to ignore the second received audio signal when filtering the audio signal based on the first echo error signal.
21. The circuit arrangement of claim 16, further comprising: a loudspeaker connected to the audio signal output.
22. The circuit arrangement of claim 16, further comprising: a first microphone connected to the first receiving path; and a second microphone connected to the second receiving path.
23. The circuit arrangement of claim 16, further comprising: a beamformer configured to beamform the audio signals or the echo suppression filtered audio signal or both.
24. A circuit arrangement for processing audio signals, the circuit arrangement comprising: an audio device for outputting an audio signal; a plurality of receiving paths coupled to the audio signal output; a determiner circuit configured to determine a residual echo power based on a plurality of received audio signals; wherein the determiner circuit is coupled to the plurality of receiving paths and further configured to determine an echo suppression gain based on at least a selected plurality of received audio signals of the received audio signals; wherein the echo suppression gain is based on the residual echo power, an estimate of a signal-to-echo ratio of the first received audio signal, and a second echo error signal of the received signal not selected for the estimate of the signal-to-echo ratio; and an echo suppression filter coupled to at least one of the plurality of receiving paths and the determiner circuit and configured to filter the audio signal based on a single-channel audio signal information representing a received audio signal received via a single receiving path and the echo suppression gain.
25. The circuit arrangement of claim 24, wherein the multichannel audio signal information comprises echo filtering information of the plurality of audio signals.
26. The circuit arrangement of claim 24, wherein the multichannel audio signal information comprises multichannel echo filtering information of the plurality of audio signals.
27. The circuit arrangement of claim 24, wherein the at least one echo filter comprises an adaptive echo filter.
28. The circuit arrangement of claim 24, further comprising: a loudspeaker connected to the audio signal output.
29. The circuit arrangement of claim 24, further comprising: a first microphone connected to one receiving path of the plurality of receiving paths; and a second microphone connected to another receiving path of the plurality of receiving paths.
30. The circuit arrangement of claim 24, further comprising: a beamformer configured to beamform the audio signals or the echo suppression filtered audio signal or both.
31. The method of claim 1, wherein determining echo suppression gain is also based on a second echo error signal of the received signal not selected for the estimate of the signal-to-echo ratio.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
(1) In the drawings, like reference characters generally refer to the same parts throughout the different views. The drawings are not necessarily to scale, emphasis instead generally being placed upon illustrating the principles of the invention. In the following description, various aspects are described with reference to the following drawings, in which:
(2)
(3)
(4)
(5)
(6)
(7)
(8)
(9)
(10)
(11)
(12)
(13)
(14)
(15)
(16)
DETAILED DESCRIPTION
(17) The following detailed description refers to the accompanying drawings that show, by way of illustration, specific details and aspects in which the invention may be practiced. These aspects are described in sufficient details to enable those skilled in the art to practice the invention. Other aspects may be utilized and structural, logical, and electrical changes may be made without departing from the scope of the invention. The various aspects are not necessarily mutually exclusive, as some aspects can be combined with one or more other aspects to form new aspects.
(18) Approaches to improve speech quality in mobile terminals may include the use of multi-microphone terminals. Multi-microphone terminals may advantageously provide spatial information on the near-end acoustic environment.
(19) In some of the following examples, the dual-channel microphone echo problem is specifically addressed. The adaptive echo cancellation problem may still be solved with classic (or standard) adaptive filters, for example, the normalised least mean squares (NLMS) filters, in particular, two adaptive filters (i.e., one for each microphone path) may be used.
(20) Dual channel echo postfiltering may be provided.
(21) For this the postfilter may use the multi-channel information to compute the power spectral density (PSD) and the echo suppression gains that are applied on one of the error signals to achieve residual echo suppression. In various embodiments, the multi-channel architecture may not necessarily require any beamforming and may keep a moderate computational complexity compared to classic (or standard) single channel echo postfiltering while improving echo suppression performance.
(22) Any beamforming methods may be used in order to improve spatial information.
(23) The dual-channel postfilter may be extended so as to be used with one adaptive filter instead of two. The adaptive filter may be placed on the microphone path on which the echo postfiltering takes place. This may reduce the computational complexity of the echo processing scheme while gaining the advantage of the dual-channel architecture.
(24) Generally, a method for processing audio signals as illustrated in
(25)
(26) The flow diagram 100 illustrates a method for processing audio signals.
(27) In 101, an audio signal is output. For example, an audio signal may be output via a loudspeaker.
(28) In 102, the output audio signal is received via a first receiving path as a first received audio signal. For example, the audio signal may be received via a first microphone.
(29) In 103, the output audio signal is received via a second receiving path as a second received audio signal. For example, the output audio signal may be received via a second microphone.
(30) In 104, an echo suppression gain is determined based on the first received audio signal and the second received audio signal.
(31) In 105, echo suppression of the audio signal is filtered based on the first received audio signal and the echo suppression gain.
(32) In this context, the term “ignored” may refer to not being taken into consideration. The term “determined” may refer to calculated, or estimated, or measured, for example.
(33) In other words, a method of processing audio signal, or more specifically, of performing echo cancellation and echo suppression may be provided. The method may include an output signal from a transducer for example, a loudspeaker producing a sound which then will be reflected back to the device and thereby produce echos which may be captured by microphones along with a desired signal to be input into separate paths for processing. The combined signal (which may be a combination of the desired signal, the output signal and noise) in one of the separated paths may be used to determine or obtain a value to be used on the combined signal in another path such that a resultant signal may be obtained. The resultant signal may have echos (from the output signal) being suppressed and may be similar to the desired signal.
(34) Echo suppression gain being determined based on the first audio signal may include echo of the first audio signal being filtered to produce a first echo error signal, the echo suppression gain being determined based on the first echo error signal and echo suppression of the audio signal being filtered based on the first echo error signal.
(35) The echo suppression gain may for example be determined based on an estimate of the residual echo power spectral density of the first received signal (e.g. at the first microphone) and an estimate of the signal-to-echo ratio of the first received signal.
(36) Echo suppression gain being determined based on the second audio signal may include echo of the second audio signal being filtered to produce a second echo error signal and the echo suppression gain being determined based on the second echo error signal.
(37) The residual echo power spectral density (PSD) of the first received signal and the signal-to-echo ratio of the first received signal are for example determined based on an estimation of a relative transfer function characterizing (e.g. in frequency domain) the dependency of the second echo error signal from the audio signal that has been output in relation to the dependency of the first echo error signal from the audio signal that has been output.
(38) The residual echo power spectral density of the first received signal and the signal-to-echo ratio of the first received signal may for example further be determined based on an estimation of a relative transfer function characterizing (e.g. in frequency domain) the dependency of the second echo error signal from a speech signal to the dependency of the first echo error signal from the speech signal.
(39) The filtering of echo may include an adaptive echo filtering. For example, the first echo error signal is determined by subtracting a first estimate of the echo present in the first received audio signal from the first received audio signal. Similarly, the second echo error signal is for example determined by subtracting a second estimate of the echo present in the second received audio signal from the second received audio signal.
(40) Filtering of the echo suppression of the audio signal may include ignoring the second received audio signal.
(41) Outputting an audio signal may include outputting an audio signal via a loudspeaker.
(42) Receiving the audio signal via a first receiving path as a first audio signal for example includes receiving the audio signal via a first microphone and receiving the output audio signal via a second receiving path as a second received audio signal for example includes receiving the output audio signal via a second microphone.
(43) The method may further include a residual echo power being determined based on the first received audio signal and the second received audio signal. The echo suppression gain may be determined based on the residual echo power.
(44) The multichannel audio signal information for example includes multichannel echo filtering information of the received audio signals.
(45) The filtering of echo for example includes an adaptive echo filtering.
(46) Outputting an audio signal may include outputting an audio signal via a loudspeaker.
(47) The received audio signals are for example received via at least a first microphone and a second microphone.
(48) The method as shown in the flow diagram 100 of
(49) In this context, the term “beamformed” or “beamforming” may generally refer to a signal processing technique used for directional signal transmission or reception.
(50)
(51) The flow diagram 200 illustrates a method for processing audio signals.
(52) In 201, an audio signal may be output. For example, an audio signal may be output via a loudspeaker.
(53) In 202, an echo suppression gain may be determined based on a multichannel audio signal information representing received audio signals which are received via different receiving paths. For example, the received audio signals may be received via at least a first microphone and a second microphone.
(54) In 203, echo suppression of the audio signal may be filtered based on single-channel audio signal information representing a received audio signal received via a single receiving path and the determined echo suppression gain. The determined echo suppression may be determined based on the multi-channel audio signal information.
(55) The multichannel audio signal information may include echo filtering information of the received audio signals or multichannel echo filtering information of the received audio signals. The filtering of echo may include an adaptive echo filtering.
(56) The method as shown in the flow diagram 200 of
(57) Generally a circuit arrangement for processing audio signals as illustrated in
(58)
(59) The circuit arrangement for processing audio signals 300 includes an audio signal output 301 for outputting an audio signal; a first receiving path 302 configured to receive the output audio signal 301 as a first received audio signal; a second receiving path 303 configured to receive the output audio signal 301 as a second received audio signal; a determiner 304 configured to determine an echo suppression gain based on the first received audio signal and the second received audio signal; and an echo suppression filter 305 coupled to the first receiving path 302 and the determiner 304 configured to filter echo suppression of the audio signal based on the first received audio signal and the echo suppression gain.
(60) For example, the echo suppression filter 305 may be configured to ignore the second received audio signal when filtering the audio signal based on the first echo error signal. The term “ignore” and “determine” may similarly be defined as above for the term “ignored” and “determined”, respectively.
(61) The circuit arrangement 300 for example carries out a method as illustrated in
(62) The circuit arrangement for processing audio signals 300 may further include at least one echo filter, e.g. at least one adaptive echo filter, configured to filter the first received audio signal to produce a first echo error signal. The determiner 304 may be configured to determine the echo suppression gain based on the first echo error signal and the echo suppression filter 305 may be configured to filter echo suppression of the audio signal based on the first echo error signal.
(63) The circuit arrangement 300 may further include at least one echo filter configured to filter the second audio signal to produce a second echo error signal. The determiner 304 may be configured to determine the echo suppression gain based on the second echo error signal.
(64) For example, the at least one echo filter may include an adaptive echo filter.
(65) The circuit arrangement 300 may further include a loudspeaker connected to the audio signal output 301.
(66) The circuit arrangement 300 may further include a first microphone connected to the first receiving path 302 and a second microphone connected to the second receiving path 303.
(67) The circuit arrangement 300 may further include a second determiner configured to determine a residual echo based on the first received audio signal and the second received audio signal. The second determiner may be configured to determine a second echo suppression gain based on the second received audio signal. The second determiner may use the first received audio signal and the second received audio signal to determine the second echo suppression gain. The two determiners may use audio signals received via different microphones. Furthermore, the output of the determiner and the second determiner may be applied to a beamforming circuit.
(68) The circuit arrangement 300 may further include a beamformer configured to beamform the audio signals and/or the echo suppression filtered audio signals. For example, the beamformer may be used to beamform all multichannel received signals.
(69)
(70) The circuit arrangement 400 for processing audio signals may include an audio signal output 401 for outputting an audio signal; a plurality of receiving paths 402 coupled to the audio signal output 401; a determiner 403 coupled to the plurality of receiving paths 402 and configured to determine an echo suppression gain based on a multichannel audio signal information representing a plurality of received audio signals received via the receiving paths 402; and an echo suppression filter 404 coupled to at least one of the plurality of receiving paths 402 and the determiner 403 and configured to filter the audio signal based on a single-channel audio signal information representing a received audio signal received via a single receiving path and the echo suppression gain.
(71) The circuit arrangement 400 for example carries out a method as illustrated in
(72) For example, the multichannel audio signal information may include echo filtering information of the plurality of audio signals or multichannel echo filtering information of the plurality of audio signals.
(73) The echo filter may include an adaptive echo filter.
(74) The circuit arrangement 400 may further include a loudspeaker connected to the audio signal output 401.
(75) The circuit arrangement 400 may further include a first microphone connected to one receiving path of the plurality of receiving paths 402; and a second microphone connected to another receiving path of the plurality of receiving paths 402.
(76) The circuit arrangement 400 may further include a beamformer configured to beamform the audio signal and/or the echo suppression filtered audio signal.
(77) It should be noted that aspects and features described in context of the method illustrated in
(78) Computation rules for the echo and gain may be provided. Also a software implementation or a hybrid implementation (partially in hardware and partially in software) of the determination of the echo and the gain may be provided.
(79) Examples for the method illustrated by the flow diagram 100 and the circuit arrangement 300 are described in the following in more detail.
(80) An exemplary system experiencing dual channel echo is described in the following.
(81)
(82) An example of a terminal 500 equipped with one loudspeaker 501 and two microphones 502, 503 is illustrated in
(83) The microphones 502, 503 may record the near-end speaker voice or speech signal 506 and eventually the background noise 508. The near-end speaker voice 507 may also reflect in the environment 505 before being picked up by the microphones 502, 503. Because both microphones 502, 503 may not necessarily be placed at the same position, the acoustic path between the near-end speaker and each microphone may have to be modeled. It should be appreciated that
(84) As an example, the microphones 502, 503 may be placed at corner regions of the terminal 500. The loudspeaker 501 may be placed slightly closer to the one microphone 502 as compared to the other microphone 503. As such, it may be considered that the microphone 502 provides a secondary microphone signal (or observation) and the other microphone 503 provides a primary microphone signal (or observation).
(85) In some examples, the terminal 500 may be a telecommunication terminal that is equipped with one loudspeaker and two or more microphones.
(86) It should also be appreciated that the terminal 500 may not be limited to only a telecommunication terminal and that the terminal 500 may be extended to a laptop or a tablet, which may also experience echo suppression. The terminal 500 may also be a handsfree mobile terminal.
(87) The signal model of the dual channel (DC) echo problem is schematized as shown in
(88)
(89) The primary and secondary microphone signals 600, 601 are provided by the microphones 502, 503 and are denoted y.sub.p(n) and y.sub.s(n) respectively. The signals d.sub.p(n) 602 and d.sub.s(n) 603 represent the echo signal picked up by the primary and secondary microphone 502, 503 respectively. Both are generated by the loudspeaker signal x(n) 604 of the loudspeaker 501 where h.sub.p|s(n) are represented by convolutive blocks 605, 606 accounting for the acoustic path between the loudspeaker 501 and respective microphones 502, 503.
(90) The signals s.sub.p(n) 607 and s.sub.s(n) 608 represent the near-end speech signal picked up by the primary and secondary microphone 502, 503 respectively. Both are generated by the near-end speech signal s(n) 609 (or 507), where g.sub.p|s(n) are represented by convolutive blocks 610, 611 accounting for the acoustic path between the near-end speaker 504 and the primary or secondary microphone 502, 503.
(91) The primary microphone signal y.sub.p(n) 600 is given by the sum, provided by summing block 612, of s.sub.p(n) 607 and d.sub.p(n) 602. The secondary microphone signal y.sub.s(n) 601 is given by the sum, provided by summing block 613, of s.sub.s(n) 608 and d.sub.s(n) 603.
(92) In respect with signal model in
y.sub.p(n)=g.sub.p(n)*s(n)+h.sub.p(n)*x(n) Eq. (1)
y.sub.s(n)=g.sub.s(n)*s(n)+h.sub.s(n)*x(n) Eq. (2)
where: x(n) is the loudspeaker signal 604, y.sub.p|s(n) represents the primary or secondary microphone signals 600, 601 respectively, h.sub.p|s(n) 605, 606 represents the acoustic path between the loudspeaker 501 and the primary or secondary microphone 502, 503 s(n) 609 is the near-end speaker signal g.sub.p|s(n) 610, 611 represents the acoustic path between the near-end speaker 504 and the primary or secondary microphone 502, 503 * represents the convolution operation.
(93) In order to validate the signal model of
(94)
(95)
(96) For achieving single channel (SC) echo cancellation, an echo cancellation circuit 800 considered may include an adaptive filter part, which includes two adaptive filters 801, 802, followed by an echo postfilter 803 as shown in
(97) A “circuit” may be understood as any kind of a logic implementing entity, which may be special purpose circuitry or a processor executing software stored in a memory, firmware, or any combination thereof. Thus, a “circuit” may be a hard-wired logic circuit or a programmable logic circuit such as a programmable processor, e.g. a microprocessor (e.g. a Complex Instruction Set Computer (CISC) processor or a Reduced Instruction Set Computer (RISC) processor). A “circuit” may also be a processor executing software, e.g. any kind of computer program, e.g. a computer program using a virtual machine code such as e.g. Java. Any other kind of implementation of the respective functions which are described may also be understood as a “circuit”. For example, the various components of the circuit arrangement such as the determiner may be implemented by a circuit as described above.
(98)
(99) For each microphone 809, 810, the effect of echo may be considered to be the same as in a SC echo cancellation. Therefore for each microphone signal y.sub.p|s(n) 811, 812, an estimate of the echo signal 813, 814 may be obtained by the use of an adaptive filter 801, 802 as in the SC case.
(100) Although in general, any adaptive echo cancellation process may be applied, e.g. any as such known adaptive echo cancellation algorithm, the standard NLMS algorithm may be used to estimate the echo signals.
(101) For the same reasons that in the SC case, some residual echo may be present in the error signals e.sub.p|s(n) 815, 816 at the output of the acoustic echo cancellations (AECs). The error signal e.sub.p|s(n) 815, 816 may be obtained by the difference, provided by respective summing blocks 817, 818, between the microphone signals y.sub.p|s(n) 811, 812 and the respective estimates of the echo signals 813, 814. The postfilter 803 may be used to achieve further echo suppression. The postfilter 803 may include a filter update block 819 and an echo postfiltering block 820. The filter update block 819 produces an output 821 based on e.sub.p|s(n) 815, 816 and the loudspeaker signal x(n) 822 of the loudspeaker 808. For example in
(102) The circuit 800 may refer to the circuit arrangement 300 of
(103) In a similar manner, the circuit 800 may refer to the circuit arrangement 400 of
(104)
(105)
(106) In
(107) The circuit 900 may refer to the circuit arrangement 300 of
(108) In a similar manner, the circuit 900 may refer to the circuit arrangement 400 of
(109) Generally, the circuit 900 may function in a similar manner as the circuit 800 of
(110) The circuits 800, 900 may be extended to multi-channel m. In multi-channels including a plurality of receiving paths, for example the plurality of receiving paths 402 of
(111) For example, the echo suppression may still be applied only to the primary microphone path. This means existing SC echo suppression gain rules may still be used. The computation of a gain rule may generally require estimates of the residual echo PSD and of the near-end PSD. For example, the following gain rules may be used:
(112)
where Φ.sup.S.sup.
(113) It should be appreciated and understood that computing subband echo postfilter requires the estimation of the residual echo PSD and/or the near-end speech signal PSD. A method to estimate the residual echo and near-end speech PSDs in DC or multichannel case echo postfiltering problem may be introduced.
(114) New estimations of residual echo and near-end PSDs are described below. For dual channel or multichannel echo postfiltering the computation of these PSDs requires the knowledge of relative transfer functions (RTF).
(115) A residual echo and near-end PSD estimate may be provided.
(116) The difference for the computation of the residual echo and near-end PSDs lies in the use of at least two microphone signals instead of one. In the following example, the estimation of these PSDs for the dual channel case is discussed.
(117) Signals equations at the postfilter in the case of two adaptive filters:
(118) (a) Error signals equations in the time domain
(119)
where {tilde over (h)}.sub.p|s(n)=h.sub.p|s(n)−ĥ.sub.p|s(n) represents the echo path misalignment vector.
(120) (b) Error signals in the frequency domain
E.sub.p(k,i)=G.sub.p(k,i).Math.S(k,i)+{tilde over (H)}.sub.p(k,i).Math.X(k,i) Eq. (6)
E.sub.s(k,i)=G.sub.s(k,i).Math.S(k,i)+{tilde over (H)}.sub.s(k,i).Math.X(k,i) Eq. (7)
where: E.sub.p(k,i) and E.sub.s(k,i) are Fourier transform of the error signals of the primary and secondary microphone, respectively k and i respectively represent the frame and frequency bin indexes
(121) In the following, the frame and frequency indexes will be omitted for clarity purposes and will only be used when necessary.
(122) (c) Residual echo signals auto- and cross-psds
(123) Assuming loudspeaker signal and the near-end speech signal are uncorrelated (i.e. their cross-PSD is null Φ.sup.XS=0), the following may be written:
Φ.sup.E.sup.
Φ.sup.E.sup.
Φ.sup.E.sup.
where: Φ.sup.E.sup.
(124) Two RTFs Γ and Θ may be defined as follow:
(125)
(126) Rewriting Eqs. (8) to (10) with the above notations, the following may be obtained:
Φ.sup.E.sup.
Φ.sup.E.sup.
Φ.sup.E.sup.
(127) From Eqs. (12) to (14), new estimates of the residual echo and near-end PSDs may be deduced as follow:
(128)
(129) Another set of PSDs estimate may be derived by taking into account the error signals cross-PSD Φ.sup.E.sup.
(130)
(131) Two sets of PSDs estimates may be used to compute echo postfilter gains in case of DC echo processing. In either case (i.e. the set of Eqs. (15) and (16), or the set of Eqs. (17) and (18)), the computation of Φ.sup.{tilde over (D)}.sup.
(132) The RTFs may need to be estimated. Methods to estimate RTF may include cross-spectral method, mean square or least square error minimization.
(133) (a) Near-end speech acoustic paths RTF estimation
(134) The near-end speech acoustic paths Θ is defined as:
(135)
(136) Θ may also be interpreted as a gain such that:
S.sub.s=Θ.Math.S.sub.p Eq. (20)
(137) Considering near-end only speech activity period (i.e. E.sub.p=S.sub.p=G.sub.p.Math.S and E.sub.s=S.sub.s=G.sub.s.Math.S), an estimate {circumflex over (Θ)} of Θ may be obtained through mean square error (MSE) or least square error (LSE) minimization.
(138) The minimum MSE (MMSE) criteria used for the derivation of the MMSE estimate of {circumflex over (Θ)} is:
(139)
(140) The MMSE estimate of {circumflex over (Θ)} is then given by
(141)
(142) Another estimate in the form of an adaptive filter may be derived from Eq (23) below. In this case, one has many choices for the adaptive filter, for example, LMS, NLMS or FBLMS. It should be understood that as the minimization criteria (Eq (23)) is in the frequency domain using LMS or NLMS may lead to an estimate in the frequency domain. The NLMS solution, which proves to be a relatively stable and robust, is as follow:
(143)
where: {tilde over (e)}(k,i)=E.sub.s(k,i)−Ê.sub.s(k,i)=E.sub.s(k,i)−{circumflex over (Θ)}.sub.NLMS(k,i).Math.E.sub.s(k,i) is the error signal μ is the stepsize which may be set to a fixed value for the simplicity purposes.
(144) The LSE minimization may also be used to estimate the near-end RTF {circumflex over (Θ)}. The LSE estimate of Θ expresses as follows:
(145)
where
(146)
given a set of K measures of β along time.
(147) Details about the derivation of {circumflex over (Θ)}.sub.LSE are presented later on. In any of the cases ({circumflex over (Θ)}.sub.MMSE, {circumflex over (Θ)}.sub.NLMS or {circumflex over (Θ)}.sub.LSE), the update may be performed during near-end only activity period.
(148) An activity detection on the loudspeaker may be detected near-end only activity periods. For example, the activity detection may be achieved by applying a threshold on the loudspeaker and microphone signals energies. The threshold on the loudspeaker energy may avoid adaptation during far-end activity periods whereas the threshold on the microphone signals may avoid adaptation during near-end silence period or on low amplitude microphone signal.
(149) (b) Echo paths RTF estimation
(150) Γ is defined as the ratio between the primary and the secondary residual echo paths:
(151)
(152) Similarly to Θ in Eq. (19) and Eq. (20), Γ defines the link between the residual echo of primary and secondary microphone in the following manner:
{tilde over (D)}.sub.s=Γ.Math.{tilde over (D)}.sub.p Eq. (26)
(153) Introducing Eq. (26) in Eqs. (6) and (7) respectively, the following may be obtained:
E.sub.p=G.sub.p.Math.S+{tilde over (H)}.sub.p.Math.X=S.sub.p+{tilde over (D)}.sub.p Eq. (27)
E.sub.s=G.sub.s.Math.S+{tilde over (H)}.sub.s.Math.X=S.sub.s+Γ.Math.{tilde over (D)}.sub.s Eq. (28)
(154) Using the fact that {tilde over (D)}.sub.s and {tilde over (D)}.sub.p are both generated by the loudspeaker signal x(n), Γ may be estimated through the cross-correlation. Assuming independence of loudspeaker and near-end speech signals (i.e. Φ.sup.XS=0), the cross-correlation estimator of Γ expresses as follows:
(155)
where Φ.sup.XE.sup.
Φ.sup.E.sup.
Φ.sup.E.sup.
(156) The least square may also be used to derive an estimate of the echo RTF Γ. In this case the minimization criterion writes as follows:
(157)
(158) The LS estimate of Γ expresses as follows:
(159)
(160) The derivation of the LS estimate of Γ is presented below. It is noted that {circumflex over (Γ)}.sub.LS matches {circumflex over (Γ)}.sub.CC if only one frame is consider for the least square criterion minimization.
(161) PSD estimates involving multi-microphones may be provided. A communication terminal equipped with one loudspeaker and M microphones may be considered. Each microphone records both the echo signal which is generated by the loudspeaker, and the near-end speech signal. The signal on the m.sup.th microphone signal may be written as follows:
y.sub.m(n)=g.sub.m(n)*s(n)+h.sub.m(n)*x(n) Eq. (33)
where y.sub.m(n) is the signal picked up by the m.sup.th microphone signal h.sub.m(n) is the acoustic path between the loudspeaker and m.sup.th microphone signal g.sub.m(n) is the acoustic path between the near-end speaker and m.sup.th microphone signal.
(162) As for the dual-channel case discussed above, an adaptive filter may be used to estimate the echo signal picked up by the m.sup.th microphone. Therefore, the multi-channel postfilter may take the loudspeaker signal and the microphone and/or error for microphone paths using an adaptive filter signals as inputs. Furthermore, the multi-channel information may only be used in the computation of the echo suppression while echo suppression itself may take place on the m.sup.th microphone path which has an adaptive filter
(163)
(164) The device 1001 may include for each receiving path, an echo PSD and gain update block 1014, 1015 and an echo postfiltering block 1016, 1017 prior to the beamformer 1013.
(165) The loudspeaker signal x(n) 1018 of the loudspeaker 1003 may refer to the audio signal output 301; y.sup.1(n) 1010 may refer to the first receiving path 302; y.sup.2(n) 1011 may refer to the second receiving path 303; the echo PSD and gain update blocks 1014, 1015 may refer to the determiner 304; and the echo postfiltering blocks 1016, 1017 may refer to the echo suppression filter 305.
(166) The loudspeaker signal x(n) 1018 of the loudspeaker 1003 may refer to the audio signal output 401; y.sup.1|2(n) 1010, 1011 may refer to the plurality of receiving paths 402; the echo PSD and gain update blocks 1014, 1015 may refer to the determiner 403; and the echo postfiltering blocks 1016, 1017 may refer to the echo suppression filter 404.
(167) Another device 1100 for multi-channel echo postfiltering may be provided as illustrated in
(168)
(169) The dots 1102, 1103 on the microphone path 1104, 1105 encounters for the presence of adaptive filter (not shown in
(170) As compared to the device 1001 of
(171)
(172) For the scheme illustrated in
(173)
where: Φ.sub.i.sup.Z.sup.
(174) The relative transfer function may be defined as follow:
(175)
(176) With Ĥ.sub.i.sup.m(n) equal to 0 in case no adaptive filter is used on the m.sup.th microphone path.
(177) The function of postfilter may be expressed as follows:
(178)
(179) The previous equations (Eqs. (37) and (38)) show that the computation of the postfilter requires an estimation of the echo PSD on the m.sup.th microphone Φ.sup.D.sup.
(180) In the derivation of the multi-channel PSDs estimate that follows, no adaptive filter is assumed for use on the microphone paths (it is to be noted that this assumption is not limiting and is only made for reasons of simplicity of the explanation). This implies that input microphone signals are y.sub.m(n). Given the microphone observation y.sub.m(n), its Fourier transform may be written as follows:
Y.sub.m=G.sub.m.Math.S+H.sub.m.Math.X Eq. (39)
(181) In the example above, two different estimates for the residual echo and near-end PSDs for the dual-channel terminals are discussed as in Eqs. (17) and (18). Although the use of the estimates in Eqs. (17) and (18) involves phase information which is delicate to handle in speech processing, multi-channel echo and near-end PSD estimates that matches both formalism (the set of Eqs. (15) and (16) and the set of Eqs. (17) and (18)) are presented in the following.
(182) PSD estimates match Eqs. (15) and (16) for M=2. Assuming loudspeaker signal and the near-end speech signal are uncorrelated (i.e. their cross-PSD is null Φ.sup.XS=0), the l.sup.th microphone auto-PSD expresses as follows:
Φ.sup.Y.sup.
where l is the microphone channel index ranging from 1 to M.
(183) By introducing the RTF of Eq. (41)
(184)
into Eq. (40), the following may be obtained:
Φ.sup.Y.sup.
(185) Eq. (42) shows that the l.sup.th microphone auto-PSD may be written as a function of the echo signal and near-end signal PSDs of the m.sup.th microphone, i.e. Φ.sup.D.sup.
(186) Considering all the M microphone signals, Eq. (42) may equivalently be written in a matrix form as follows:
(187)
With the following notations
(188)
which represents the PSDs required for the computation of the echosuppression gains for the m.sup.th microphone path.
(189) From Eq. (44), an estimate of V may be derived as:
{circumflex over (V)}=(A.sup.HA).sup.−1A.sup.HZ. Eq. (45)
(190) The expansion of Eq. (45) leads to the following echo and near-end PSD estimates:
(191)
(192) PSD estimates match Eqs. (17) and (18) for M=2.
(193) By introducing the RTFs defined in Eq. (41) in Eq. (39), the following may be written:
Y.sub.l=Γ.sup.m,l.Math.H.sub.m.Math.X+Θ.sup.m,l.Math.G.sub.m.Math.S=Γ.sup.m,l.Math.D.sub.m+Θ.sup.m,l.Math.S.sub.m Eq. (48)
(194) Eq. (48) shows that the l.sup.th microphone signal may be written as a function of the echo signal and near-end signals received by the m.sup.th microphone channel.
(195) Considering all the M microphone observations, Eq. (48) may equivalently be written in a matrix form as follows:
(196)
(197) From Eq. (49), the microphone PSD matrix may be computed as follows:
Φ.sup.YY=AΦ.sup.VVA.sup.H Eq. (51)
Where Φ.sup.YY=Y.Math.Y.sup.H is an estimate of the microphone power spectrum matrix
(198)
(199) An estimate of Φ.sup.VV is given by
{circumflex over (Φ)}.sup.VV=(A.sup.HA).sup.−1A.sup.HΦ.sup.YYA(A.sup.HA).sup.−1 Eq. (52)
(200) The expansion of Eq. (52) leads to the following echo and near-end PSD estimates:
(201)
(202) The PSD estimates may require the knowledge of the microphone signals auto-PSD. In a real time implementation, an estimate of the microphone signals may be obtained through auto-regressive smoothing.
(203) The PSD estimates may be valid for the case where none of the microphone signals is processed by an adaptive filter before being used by the postfilter.
(204) In another example, the adaptive filter may be placed on some or on all of the microphone paths before the postfilter. The use of an adaptive filter on the m.sup.th microphone path means that in the above equations y.sub.m(n) becomes e.sub.m(n) with
e.sub.m(n)=g.sub.m(n)*s(n)+{tilde over (h)}.sub.m(n)*x(n). Eq. (55)
(205) RTF Estimation may be provided. For example, least square estimate of the near-end RTF may be used. Assuming near-end only activity periods and the presence of some local noise in the near-end acoustic environment, the l.sup.th microphone signal may be written as follows:
y.sub.l(n)=g.sub.l(n)*s(n)+b.sub.l(n), Eq. (56)
or equivalently in the frequency domain:
Y.sub.l=H.sub.l.Math.X+B.sub.l Eq. (57)
where b.sub.l(n) represents the ambient noise received by the l.sup.th microphone and B.sub.l is its Fourier transform.
(206) By introducing the near-end RTF definition as defined in Eq. (41) into Eq. (57), the following may be obtained:
Y.sub.l=Θ.sup.m,l.Math.Y.sub.m+B.sub.l−Θ.sup.m,l.Math.B.sub.m=Θ.sup.m,l.Math.Y.sub.m+{tilde over (B)}.sub.l Eq. (58)
(207) The least square estimate of the near-end RTF may be derived as follows:
(208)
where
(209)
given a set of R measures of β along time.
(210) Least square estimate of the Echo RTF may be provided. Assuming far-end only activity periods and the presence of some local noise in the near-end acoustic environment, the l.sup.th microphone signal may be written as follows:
y.sub.l(n)=h.sub.l(n)*x(n)+b.sub.l(n), Eq. (60)
or equivalently in the frequency domain:
Y.sub.l=H.sub.l.Math.X+B.sub.l Eq. (61)
(211) By introducing the echo RTF definition as defined in Eq. (60) into Eq. (61), the following may be obtained:
Y.sub.l=Γ.sup.m,l.Math.Y.sub.m+B.sub.l−Γ.sup.m,l.Math.B.sub.m=Γ.sup.m,l.Math.Y.sub.m+{tilde over (B)}.sub.l Eq. (62)
(212) The vector [X Y.sub.1 . . . Y.sub.M].sup.T and an observation window which may be subdivised into R frames in the time domain may be considered. Considering the echo RTF is stationary within the observation window, non-stationary of speech signals may be exploited from one frame to another. For each frame r of the observation interval, the following PSD may be written:
Φ.sub.(r).sup.Y.sup.
(213) {tilde over (B)}.sub.l is defined by the ambient noise in the near-end acoustic environment, therefore it may be assumed that it is statistically independent from the loudspeaker (i.e. Φ.sub.(r).sup.{tilde over (B)}.sup.
(214)
(215) Then the LS estimate of the echo RTF defines as follows:
(216)
and expresses as follows:
(217)
where
(218)
given a set of R measures of β along time.
(219) The performance of the dual microphone residual echo PSD estimate described above may be assessed and compared against an existing estimate.
(220) For example, the data recorded with the mock-up phone as previously discussed may be used to generate a test database of speech signals. The microphone signals may contain both echo-only and double-talk periods. The signal-to-echo ratio (SER) may be set and measured for the primary microphone and the secondary microphone may be computed accordingly. The SER may range from −5 dB to 10 dB. The dual channel (DC) echo processing method, for example, the method of
(221) The assessment of the postfilter for the dual channel (DC) echo processing method may be performed in two steps. On one side, the PSD estimator for the dual channel (DC) echo processing method may be assessed in terms of symmetric segmental logarithmic error which may be expressed as follows:
(222)
where K represents the number of frames.
(223) On the other side, the DC postfilter may be compared to the SC postfilter in terms of echo return loss enhancement (ERLE), of speech attenuation (SA), of cepstral distance (CD) and of informal listening tests. The ERLE may represent an amount of echo suppression achieved by the adaptive filter and by the postfilter all together and may be measured during echo-only periods. The SA may be used to measure the amount of speech attenuation introduced by the postfilter on the near-end speech signal during double-talk periods. The SA may be measured for the primary microphone as the attenuation between the clean speech s.sub.p(n) 607 and a weighted speech signal
(224)
where L is length of the frames on which may compute the segmental SA and represents the number of frames during which double talk occurs.
(225) The weighted speech signals
(226) For example, the number of subbands M may be set to 256 and the subband conversion may take place through short term Fourier transform with overlap add.
(227) Residual echo PSD estimate assessment may be performed.
(228)
(229) Residual echo suppression may be provided.
(230)
(231)
(232) Moreover, the DC postfilter 1208 may introduce less near-end speech attenuation during double-talk than the SC postfilter 1209. In the DC case 1208, the speech attenuation increases with the SER while it decreases for the SC case 1209. This difference of behaviour in the speech attenuation may directly reflect on the cepstral distance and may explain its increase for the DC case 1208. Informal listening tests may show that the DC postfilter 1208 yields a slight better intelligibility compared to the SC postfilter 1209 during double-talk periods. The SA introduced by the SC postfilter 1209 may be perceptible and may sometimes lead to complete suppression of the speech.
(233) While the invention has been particularly shown and described with reference to specific aspects, it should be understood by those skilled in the art that various changes in form and detail may be made therein without departing from the spirit and scope of the invention as defined by the appended claims. The scope of the invention is thus indicated by the appended claims and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced.