ECHO CANCELLATION DEVICE, ECHO CANCELLATION METHOD, AND PROGRAM
20220329940 · 2022-10-13
Assignee
Inventors
Cpc classification
H04R3/02
ELECTRICITY
International classification
Abstract
Provided is an echo cancellation apparatus capable of calculating an acoustic coupling amount with high accuracy regardless of the magnitude of the near-end speaker component and without using a double talk detector. The echo cancellation apparatus cancels an echo included in a sound pickup signal picked up by a microphone placed at a near-end and includes an acoustic coupling amount calculation unit that updates and calculates an acoustic coupling amount estimated value of a component of a reproduction signal, which is a signal picked up by a microphone placed at a far-end included in the sound pickup signal, such that an update amount is decreased the greater a magnitude of a component other than an echo component is in the sound pickup signal; a gain calculation unit that calculates a gain coefficient on the basis of the acoustic coupling amount estimated value; and an integration unit that integrates the gain coefficient with the sound pickup signal and generates an echo cancellation signal.
Claims
1. An echo cancellation apparatus for cancelling an echo included in a sound pickup signal picked up by a microphone placed at a near-end, the apparatus comprising: an acoustic coupling amount calculator that updates and calculates an acoustic coupling amount estimated value of a component of a reproduction signal, which is picked up by a microphone placed at a far-end included in the sound pickup signal, such that an update amount is decreased the greater a magnitude of a component other than an echo component is in the sound pickup signal; a gain calculator that calculates a gain coefficient on the basis of the acoustic coupling amount estimated value; and an integrator that integrates the gain coefficient with the sound pickup signal and generates an echo cancellation signal.
2. The echo cancellation apparatus according to claim 1, wherein the acoustic coupling amount calculator determines an update amount by controlling a step size in a case where a formula for obtaining the acoustic coupling amount estimated value is represented by an updated formula with the step size.
3. The echo cancellation apparatus according to claim 2, wherein the acoustic coupling amount estimated value is calculated via
4. The echo cancellation apparatus according to claim 1, wherein the acoustic coupling amount calculator decreases an update amount of the acoustic coupling amount the greater a proportion of the component other than the echo component is when determining an amount to update of the acoustic coupling amount only in a case where a proportion of the component other than the echo component is greater than a predetermined threshold and an average value of a frequency component of the component other than the echo component is greater than a predetermined threshold, and determines an update amount of the acoustic coupling amount without using a proportion of the component other than the echo component in a case where a proportion of the component other than the echo component is equal to or less than a predetermined threshold or in a case where an average value of a frequency component of the component other than the echo component is equal to or less than a predetermined threshold.
5. An echo cancellation method for cancelling an echo included in a sound pickup signal picked up by a microphone placed at a near-end, the apparatus comprising: updating and calculating an acoustic coupling amount estimated value of a component of a reproduction signal, which is picked up by a microphone placed at a far-end included in the sound pickup signal, such that an update amount is decreased the greater a magnitude of a component other than an echo component is in the sound pickup signal; calculating a gain coefficient on the basis of the acoustic coupling amount estimated value; and integrating the gain coefficient with the sound pickup signal and generating an echo cancellation signal.
6. (canceled)
7. The echo cancellation apparatus according to claim 2, wherein the acoustic coupling amount calculator decreases an update amount of the acoustic coupling amount the greater a proportion of the component other than the echo component is when determining an amount to update of the acoustic coupling amount only in a case where a proportion of the component other than the echo component is greater than a predetermined threshold and an average value of a frequency component of the component other than the echo component is greater than a predetermined threshold, and determines an update amount of the acoustic coupling amount without using a proportion of the component other than the echo component in a case where a proportion of the component other than the echo component is equal to or less than a predetermined threshold or in a case where an average value of a frequency component of the component other than the echo component is equal to or less than a predetermined threshold.
8. The echo cancellation apparatus according to claim 3, wherein the acoustic coupling amount calculator decreases an update amount of the acoustic coupling amount the greater a proportion of the component other than the echo component is when determining an amount to update of the acoustic coupling amount only in a case where a proportion of the component other than the echo component is greater than a predetermined threshold and an average value of a frequency component of the component other than the echo component is greater than a predetermined threshold, and determines an update amount of the acoustic coupling amount without using a proportion of the component other than the echo component in a case where a proportion of the component other than the echo component is equal to or less than a predetermined threshold or in a case where an average value of a frequency component of the component other than the echo component is equal to or less than a predetermined threshold.
9. A system for cancelling an echo included in a sound pickup signal picked up by a microphone placed at a near-end comprising: a processor; and a memory storing computer-executable instructions that when executed by the processor cause the system to perform a method, the method comprising: updating and calculating, by an acoustic coupling amount calculator, an acoustic coupling amount estimated value of a component of a reproduction signal, which is picked up by a microphone placed at a far-end included in the sound pickup signal, such that an update amount is decreased the greater a magnitude of a component other than an echo component is in the sound pickup signal; calculating, by a gain calculator, a gain coefficient on the basis of the acoustic coupling amount estimated value; and integrating, by a gain integrator, the gain coefficient with the sound pickup signal and generating an echo cancellation signal.
10. The system according to claim 9 wherein the acoustic coupling amount calculator determines an update amount by controlling a step size in a case where a formula for obtaining the acoustic coupling amount estimated value is represented by an updated formula with the step size.
11. The system according to claim 10, wherein the acoustic coupling amount estimated value is calculated via
12. The system according to claim 9, wherein the acoustic coupling amount calculator decreases an update amount of the acoustic coupling amount the greater a proportion of the component other than the echo component is when determining an amount to update of the acoustic coupling amount only in a case where a proportion of the component other than the echo component is greater than a predetermined threshold and an average value of a frequency component of the component other than the echo component is greater than a predetermined threshold, and determines an update amount of the acoustic coupling amount without using a proportion of the component other than the echo component in a case where a proportion of the component other than the echo component is equal to or less than a predetermined threshold or in a case where an average value of a frequency component of the component other than the echo component is equal to or less than a predetermined threshold.
13. The system according to claim 10, wherein the acoustic coupling amount calculator decreases an update amount of the acoustic coupling amount the greater a proportion of the component other than the echo component is when determining an amount to update of the acoustic coupling amount only in a case where a proportion of the component other than the echo component is greater than a predetermined threshold and an average value of a frequency component of the component other than the echo component is greater than a predetermined threshold, and determines an update amount of the acoustic coupling amount without using a proportion of the component other than the echo component in a case where a proportion of the component other than the echo component is equal to or less than a predetermined threshold or in a case where an average value of a frequency component of the component other than the echo component is equal to or less than a predetermined threshold.
14. The system according to claim 11, wherein the acoustic coupling amount calculator decreases an update amount of the acoustic coupling amount the greater a proportion of the component other than the echo component is when determining an amount to update of the acoustic coupling amount only in a case where a proportion of the component other than the echo component is greater than a predetermined threshold and an average value of a frequency component of the component other than the echo component is greater than a predetermined threshold, and determines an update amount of the acoustic coupling amount without using a proportion of the component other than the echo component in a case where a proportion of the component other than the echo component is equal to or less than a predetermined threshold or in a case where an average value of a frequency component of the component other than the echo component is equal to or less than a predetermined threshold.
Description
BRIEF DESCRIPTION OF DRAWINGS
[0026]
[0027]
[0028]
[0029]
[0030]
DESCRIPTION OF EMBODIMENTS
[0031] An embodiment of the present invention will be described in detail below. Note that components with the same function are given the same number, and redundant descriptions are omitted.
First Embodiment
[0032] The configuration of an echo cancellation apparatus according to the first embodiment will be described below with reference to
[0033] The second acoustic coupling amount calculation unit 203, the second echo power calculation unit 204, the second gain calculation unit 205, the second integration unit 206, and the frequency combining unit 207 are additional elements. The other elements, i.e., the first frequency analysis unit 101, the second frequency analysis unit 102, the first acoustic coupling amount calculation unit 103, the first echo power calculation unit 104, the first gain calculation unit 105, and the first integration unit 106 have the same function as the first frequency analysis unit 101, the second frequency analysis unit 102, the acoustic coupling amount calculation unit 103, the echo power calculation unit 104, the gain calculation unit 105, and the integration unit 106 of the conventional, echo cancellation apparatus 100.
[0034] The operations of the additional configuration requirements not included in the related art will be described in the detail below.
[0035] <Second Acoustic Coupling Amount Calculation Unit 203>
[0036] The second acoustic coupling amount calculation unit 203 updates and calculates an acoustic coupling amount estimated value (a second acoustic coupling amount estimated value |H{circumflex over ( )}.sub.m,i(ω)|.sup.2, described below in detail) of a component of a reproduction signal, which is a signal included in the sound pickup signal spectrum Y.sub.i(ω) picked up by the microphone placed at the far-end so that the update amount is decreased the greater the magnitude of components other than the echo component is in the sound pickup signal spectrum Y.sub.i(ω) (step S203). Note that the components other than the echo component indicates disturbance (normal noise, abnormal noise) of the near-end and particularly indicates abnormal noise of the disturbance of the near-end. This is assuming that normal noise has been cancelled in advance by noise reduction or the like, not illustrated. However, the components other than the echo component may take into account abnormal noise and a component of normal noise that leaked through cancellation.
[0037] The conventional acoustic coupling amount estimation formula represented by Formula (1) is expanded in Formula (6).
[0038] As seen in Formula (6), by factoring out the acoustic coupling amount estimated value of one frame previous from the conventional acoustic coupling amount estimation formula, the acoustic coupling amount estimation formula can be substituted with an updated formula with step size. The step size μ.sub.i−m,ω in Formula (6) is represented by Formula (7).
[0039] With the acoustic coupling amount estimation formula obtained by expanding Formula (6), step size control in which the update amount for each frame can be changed is possible. The second acoustic coupling amount calculation unit 203 is capable of determining the update amount by controlling the step size. Note that in the related art, updating is able to stopped by using a configuration in which the step size is controlled when updating should be continued.
[0040] The second acoustic coupling amount calculation unit 203 is input with the reproduction signal spectrum X.sub.i(ω), the sound pickup signal spectrum Y.sub.i(ω), and an echo cancellation signal spectrum S{circumflex over ( )}(ω) and calculates the second acoustic coupling amount estimated value |H{circumflex over ( )}.sub.m,i(ω)|.sup.2 via Formula (8), for example (step 3203)
[0041] Here, σ[S{circumflex over ( )}.sub.i(ω)] is a parameter that takes a value that is greater the greater the magnitude of the components other than the echo component, such as a near-end speaker component or disturbance, included in the frame of the current time and can be defined via Formula (9), for example.
[0042] Here, υ.sub.1 and υ.sub.2 each represent a threshold and, in a case where the quantization bit number of the signal is 16 bit for example, υ.sub.1=υ.sub.2=1000. Also a fixed parameter may be used or a variable parameter may be used which is a value that increases the greater the magnitude of the input is of the reproduction signal spectrum X.sub.i(ω), the sound pickup signal spectrum Y.sub.i(ω), the echo cancellation signal spectrum S{circumflex over ( )}.sub.i(ω), and the like.
refers to the processing to average the absolute value of the echo cancellation signal spectrum |S{circumflex over ( )}.sub.i(ω)| in the frequency direction.
[0043] Formula (9) represents control to decrease the update amount of the acoustic coupling amount the greater the proportion of the component |S{circumflex over ( )}.sub.i(ω)| other than the echo component is when determining the amount of the acoustic coupling amount to update, only in a case where the proportion of the component |S{circumflex over ( )}.sub.i(ω)| other than the echo component is greater than the predetermined threshold va and the average value
[0044] of the frequency component of the component |S{circumflex over ( )}.sub.i(ω)| other than the echo component is greater than the predetermined threshold υ.sub.2. Formula (9) represents control to determine the update amount of the acoustic coupling amount without using the proportion of the component |S{circumflex over ( )}.sub.i(ω)| other than the echo component in a case where the proportion of the component |S{circumflex over ( )}.sub.i(ω)| other than the echo component is equal to or less than the predetermined threshold υ.sub.1 and the average value
[0045] of the frequency component of the component |S{circumflex over ( )}.sub.i(ω)| other than the echo component is equal to or less than the predetermined threshold υ.sub.2.
[0046] Note that formula (9) is set so that one of and(or), in other words, an and condition or an or condition, can be selected. When the step size is reduced, a large amount of time is needed for updating. Thus, in a case where there is a non-significant amount of disturbance, updating normally is considered to be more efficient, so the thresholds υ.sub.1, υ.sub.2 for whether or not to consider the effects of disturbance are provided, and a determination via an or condition can be performed to further relax the condition.
[0047] <Second Echo Power Calculation Unit 204>
[0048] The second echo power calculation unit 204 is the same as the first echo power calculation unit 104 except in terms of a portion of the input with the first acoustic coupling amount estimated value |H{circumflex over ( )}.sub.m,i(ω)|.sup.2 being substituted with a second acoustic coupling amount estimated value |H{circumflex over ( )}.sub.m,i(ω)|.sup.2 and in terms of output with the first echo power estimated value |D{circumflex over ( )}.sub.i(ω)|.sup.2 being substituted with a second echo power estimated value |D{circumflex over ( )}.sub.i(ω)|.sup.2. In other words, the second echo power calculation unit 204 is input with the reproduction signal spectrum X.sub.i(ω) and the second acoustic coupling amount estimated value |H˜.sub.m,i(ω)|.sup.2 and calculates the second echo power estimated value |D˜.sub.i(ω)|.sup.2 via Formula (10) (step S204).
[Math. 12]
|{tilde over (D)}.sub.i(ω)|.sup.2=Σ.sub.m=0.sup.M−1|{tilde over (H)}.sub.m,i−1(ω)|.sup.2|X.sub.i−m(ω)|.sup.2 (10)
[0049] <Second Gain Calculation Unit 205>
[0050] The second gain calculation unit 205 is the same as the first gain calculation unit 105 except in terms of a portion of the input with the first echo power estimated value |D{circumflex over ( )}.sub.i(ω)|.sup.2 being substituted with a second echo power estimated value |D˜.sub.i(ω)|.sup.2 and in terms of output with the first gain coefficient G.sub.i(ω) being substituted with the second gain coefficient G˜.sub.i(ω). In other words, the second gain calculation unit 205 is input with the second echo power estimated value |D˜.sub.i(ω)|.sup.2 and the sound pickup signal spectrum Y.sub.i(ω) and calculates the second gain coefficient G˜.sub.i(ω) via Formula (11) (step S205).
[0051] <Second Integration Unit 206>
[0052] The second integration unit 206 is the same as the first integration unit 106 except in terms of a portion of the input with the first gain coefficient G.sub.i(ω) being substituted with the second gain coefficient G{circumflex over ( )}.sub.i(ω) and in terms of the output with the first echo cancellation signal spectrum S{circumflex over ( )}.sub.i(ω) being substituted with the second echo cancellation signal spectrum S˜.sub.i(ω). In other words, the second integration unit 206 integrates the second gain coefficient G{circumflex over ( )}.sub.i(ω) with the sound pickup signal spectrum Y.sub.i(ω) and generates and outputs the second echo cancellation signal spectrum S˜.sub.i(ω) (step S206).
[0053] <Frequency Combining Unit 207>
[0054] The frequency combining unit 207 is the same as the frequency combining unit 107 except in terms of the input with the first echo cancellation signal spectrum S{circumflex over ( )}.sub.i(ω) being substituted with the second echo cancellation signal spectrum S˜.sub.i(ω). In other words, the frequency combining unit 207 recombines and outputs the output signal s{circumflex over ( )}(n) of the time domain from the second echo cancellation signal spectrum S˜.sub.i(ω) corresponding to the frequency value ω (step S207).
[0055] <Advantages of Echo Cancellation Apparatus 200 of the First Embodiment>
[0056] According to the echo cancellation apparatus 200 of the first embodiment, when obtaining the acoustic coupling amount by shifting the reproduction signal spectrum with respect to the sound pickup signal spectrum to the past, the step size for determining the acoustic coupling amount estimation update amount is decreased the greater the magnitude of the near-end speaker component (echo cancellation signal spectrum) included in the frame of the current time is. Accordingly, with double talk, an incorrect estimation of the acoustic coupling amount can be prevented without using a double talk detector. Thus, even with double talk, incorrect estimations of the acoustic coupling amount can be reduced, and echo power can be estimated with high accuracy.
[0057] <Simulation Experiment Result>
[0058] The echo cancellation apparatus (echo cancellation method) of the first embodiment described above and a conventional method will now be compared. The method according to NPL 1 is used as the conventional method. To confirm the effectiveness of the echo cancellation apparatus (echo cancellation method) of the first embodiment, the echo cancellation apparatus (echo cancellation method) of the first embodiment and the conventional method were both applied to ER processing and the performances were compared. The placement of the speaker and microphone was in accordance with ITU-T Recommendation P. 340. The reverberation time is approximately 300 ms, the sampling frequency is 16 kHz, the frequency band is from 100 Hz to 7 kHz.
[0059] In the experiment, talk at only the far-end (received single talk) and double talk were evaluated using different metrics. Received single talk was evaluated for the echo suppression amount using echo return loss enhancement (ERLE). The result of the experiment showed that the ERLE of both the echo cancellation apparatus (echo cancellation method) of the first embodiment and the conventional method was 26.32 dB. This result was due to, with the echo cancellation apparatus (echo cancellation method) of the first embodiment, σ[S{circumflex over ( )}.sub.i(ω)] equaled 1 with received single talk and the echo path power spectrum matched that of the conventional method.
[0060] With double talk, the distortion amount in the transmitted audio was evaluated using linear predictive coding (LPC) cepstral distance.
[0061] <Supplement>
[0062] The apparatus of the present invention as a single hardware entity includes, for example, an input unit to which a keyboard or the like can be connected; an output unit to which a liquid crystal display or the like can be connected; a communication unit to which a communication apparatus (for example, a communication cable) capable to communicating outside of the hardware entity can be connected; a central processing unit (CPU) (may be provided with a cache memory, register, or the like); memory such as PAM or ROM; an external storage apparatus; and a bus for connecting the input unit, the output unit, the communication unit, the CPU, the PAM, the ROM, and the external storage apparatus in a manner allowing for data to be passed therebetween. Also, as necessary, the hardware entity may be provided with an apparatus (drive) capable of reading and writing on a storage medium such as a CD-ROM. An example of a physical entity provided with such hardware resources is a general-purpose computer.
[0063] In the external storage apparatus of the hardware entity, a program required for implementing the functions described above and data required for the program processing are stored (this is not limited to an external storage apparatus, and, for example, the program may be stored in a ROM, i.e., a read-only storage apparatus). Also, the data and the like obtained from the program processing is appropriately stored in a RAM, an external storage apparatus, or the like.
[0064] In the hardware entity, the programs stored in the external storage apparatus (or the ROM or the like) and the data required for the processing of the programs are loaded into the memory as necessary and interpreted and execute or processed by a CPU as appropriate. In this manner, the CPU implements the predetermined functions (the configuration requirements labelled as units, means, and the like described above).
[0065] The present invention is not limited to the embodiments described above, and modifications can be made, as appropriate, without departing from the scope of the present invention. Also, the processing in the embodiment described above is not limited to only being executed in the time series according to the order described above and may be executed in parallel or separately depending on the processing capability of the apparatus executing the processing or as necessary.
[0066] As described above, in a case where the processing functions of the hardware entity (apparatus of the present invention) according to the embodiment described above are implemented by a computer, the processing contents of the functions required by the hardware entity are described by a program. Then, by the computer executing the program, the processing functions of the hardware entity described above are implemented on the computer.
[0067] The various types of processing described above can be executed by a program for executing the steps of the method described above being loaded on a recording unit 10020 of a computer illustrated in
[0068] The program with the described processing contents can be stored on a computer-readable storage medium. Examples of a computer-readable storage medium include, for example, a magnetic recording apparatus, an optical disk, a magneto-optical storage medium, a semiconductor memory, and the like. Specifically, for example, for the magnetic recording apparatus, a hard disk apparatus, a flexible disk, a magnetic tape, and the like can be used; for the optical disk, a digital versatile disc (DVD), a DVD random-access memory (RAM), a compact disc read-only memory (CD-ROM), a CD recordable (R)/rewritable (RW), and the like can be used, for the magneto-optical storage medium, a magneto-optical (MO) disc and the like can be used, and for the semiconductor memory, an electrically erasable and programmable read-only memory (EEP-ROM) and the like can be used.
[0069] Also, distribution of the program is performed, for example, by selling, transferring, or lending a portable storage medium, such as a DVD or CD-ROM on which the program is recorded. Furthermore, the program may be distributed by storing the program in a storage apparatus of a server computer and then transferring the program from the server computer to another computer via a network.
[0070] A computer for executing the program in this manner, for example, firstly temporarily stores a program stored in a portable storage medium or a program transferred from a server computer in its own storage apparatus. Then, when processing is executed, the computer reads the program stored in its own storage medium and executes processing in accordance with the read program. In another method of executing the program, a computer may read the program directly from a portable storage medium and execute the processing in accordance with the program, or each time the program is transferred to the computer from a server computer, processing is executed successively in accordance with the received program. Also, the processing described above may be executed by implementing processing functions via only an execution instruction and a result acquisition, without transferring the program from the server computer to the computer, in other words, via an application service, provider (ASP) service. Note that the program of the present embodiment includes information, data, and the like provided for the processing by an electronic computer that conform to the program (data and the like that are not a direct command for the computer but have characteristics specified by the processing of the computer).
[0071] Also, in this embodiment, the hardware entity is configured by executing a predetermined program on a computer. However, at least a portion of the processing contents may be implemented by hardware.