METHOD AND SYSTEM FOR HEAD-RELATED TRANSFER FUNCTION ADAPTATION

Abstract

The disclosure provides a method and a system for head-related transfer function (HRTF) adaptation. The method includes performing a system identification. The system identification includes a pinna identification and a shadowing identification.

Claims

1. A method for head-related transfer function (HRTF) adaptation, comprising: performing a system identification, the system identification comprising a pinna identification and a shadowing identification.

2. The method of claim 1, further comprising: performing a system compensation based on an adapted HRTF obtained from the system identification.

3. The method of claim 2, wherein the system compensation comprises generating an HRTF rendering matrix based on an output of the pinna identification and the shadowing identification.

4. The method of claim 1, wherein the pinna identification comprises: obtaining an adapted HRTF from an ear reference point (ERP) to an ear entrance point (EEP); performing curve fitting to obtain a compensation curve of the HRTF, based on the adapted HRTF; and multiplying an audio signal in a frequency domain (FD) by the compensation curve of the HRTF.

5. The method of claim 4, wherein the compensation curve is an inversed function of the HRTF.

6. The method of claim 1, wherein the pinna identification is implemented by a feedback loop of an active noise canceller (ANC) including an adapted controller.

7. The method of claim 1, wherein the shadowing identification is implemented by a feedforward loop of an active noise canceller (ANC) including an adapted controller.

8. The method of claim 1, wherein the shadowing identification comprises: inputting a first audio signal received from a reference microphone and a second audio signal received from an error microphone into an adapted controller, and obtaining an adapted HRTF.

9. The method of claim 8, further comprising: performing a system compensation by a hybrid output of the adapted HRTF obtained from the pinna identification and the adapted HRTF obtained from the shadowing identification.

10. A system for head-related transfer function (HRTF) adaptation, comprising: a memory configured to store computer-readable instructions; and a processor configured to perform a system identification when executing the computer-readable instructions, the system identification comprising a pinna identification and a shadowing identification.

11. The system of claim 10, wherein the processor is further configured to perform a system compensation based on an adapted HRTF obtained from the system identification.

12. The system of claim 10 or 11, wherein the processor is further configured to generate an HRTF rendering matrix based on an output of the pinna identification and the shadowing identification.

13. The system of claim 10, wherein the processor is further configured to perform the pinna identification by: obtaining an adapted HRTF from an ear reference point (ERP) to an ear entrance point (EEP); performing curve fitting to obtain a compensation curve of the HRTF based on the adapted HRTF; and multiplying an audio signal in a frequency domain (FD) by the compensation curve.

14. The system of claim 13, wherein the compensation curve is an inversed function of the HRTF.

15. The system of claim 10, wherein the processor is further configured to implement the pinna identification by a feedback loop of an active noise canceller (ANC) including an adapted controller.

16. The system of claim 10, wherein the processor is further configured to implement the shadowing identification by a feedforward loop of an active noise canceller (ANC) including an adapted controller.

17. The system of claim 10, wherein the processor is further configured to perform the shadowing identification by: inputting a first audio signal received from a reference microphone and a second audio signal received from an error microphone into an adapted controller, and obtaining an adapted HRTF.

18. The system of claim 17, wherein the processor is further configured to perform a system compensation based on a hybrid output of the adapted HRTF obtained from the pinna identification and the adapted HRTF obtained from the shadowing identification.

19. (canceled)

20. A computer-program product embodied in a non-transitory computer read-able medium that is programmed for providing for head-related transfer function (HRTF) adaptation speech separation, the computer-program product comprising instructions for: performing a system identification, the system identification comprising a pinna identification and a shadowing identification.

21. The computer-program product of claim 20, wherein the pinna identification comprises: obtaining an adapted HRTF from an ear reference point (ERP) to an ear entrance point (EEP); performing curve fitting to obtain a compensation curve of the HRTF, based on the adapted HRTF; and multiplying an audio signal in a frequency domain (FD) by the compensation curve of the HRTF.

Description

DESCRIPTION OF THE DRAWINGS

[0018] The present disclosure can be better understood by reading the following description of non-limiting implementations with reference to the accompanying drawings. The parts in the figures are not necessarily to scale, but the focus is placed on explaining the principle of the present invention. In addition, in the figures, similar or identical reference numerals refer to similar or identical elements.

[0019] FIG. 1 illustrates a schematic diagram of a method and a system of the present disclosure;

[0020] FIG. 2 illustrates a schematic diagram of an ANC feedback loop of an embodiment of the present disclosure;

[0021] FIG. 3 shows a left-ear transfer function (TF) curve graph measured by a method according to an embodiment of the present disclosure;

[0022] FIG. 4 shows a right-ear TF curve graph measured by a method according to an embodiment of the present disclosure;

[0023] FIG. 5 illustrates a schematic diagram of an ANC feedforward loop of another embodiment of the present disclosure;

[0024] FIG. 6 illustrates a schematic diagram of an acoustic echo cancellation system H(Z) implemented in a frequency domain (FD); and

[0025] FIG. 7 illustrates a schematic diagram of acoustic echo cancellation system H(Z) adaptation implemented in an FD.

DETAILED DESCRIPTION

[0026] It should be understood that the following description of the embodiments is given for illustrative purposes only, and not restrictive. The division of examples in the functional blocks, modules, or units shown in the drawings should not be construed as representing these functional blocks, and these modules or units must be implemented as physically separated units. The functional blocks, modules, or units shown or described can be implemented as individual units, circuits, chips, functions, modules, or circuit elements. One or more functional blocks or units can also be implemented in a common circuit, chip, circuit element, or unit.

[0027] Any one or more of the processor, memory, or system described herein includes computer-executable instructions that may be compiled or interpreted from computer programs created using various programming languages and/or technologies. Generally speaking, a processor (such as a microprocessor) receives and executes instructions, for example, from a memory, a computer-readable medium, etc. The processor includes a non-transitory computer-readable storage medium capable of executing instructions of a software program. The computer-readable medium may be, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof.

[0028] FIG. 1 illustrates a schematic diagram of a method and a system of the present disclosure. As shown in FIG. 1, the present disclosure provides a method and a system for HRTF adaptation. The method may include a system identification and a system compensation. The system identification aims to determine a difference between a reference model and a user. The system identification mainly focuses on a pinna difference and a shadowing function. That is, the system identification may include a pinna identification and a shadowing identification. The system compensation aims to use mathematical modeling methods to compensate for a system difference between the reference model and the user. For example, an HRTF rendering matrix is generated based on the output of the pinna identification and the shadowing identification.

[0029] For the system identification, reference may be made to an acoustic echo canceller (AEC) in a telecommunication system for modeling. For the convenience of illustration, FIG. 6 illustrates a schematic diagram of an acoustic echo cancellation system H(Z) implemented in an FD. The principle description will be performed later with reference to FIG. 6. In an embodiment of the present disclosure, by using an echo path identification algorithm, such as a normalized least mean square (NLMS) algorithm, a TF from a speaker (horn) spk to an internal microphone (that is, a pinna identification from an ERP to an EEP) and a TF from an external microphone to the internal microphone (that is, a shadowing identification from a far field to the EEP) can be obtained.

[0030] Pinna Identification (ERP to EEP)

[0031] The pinna identification (ERP to EEP) of one aspect of the present disclosure will be described below with reference to FIG. 2. FIG. 2 illustrates a schematic diagram of an ANC feedback loop. For the convenience of understanding, a system model of a pinna identification in the present disclosure will be described by taking the ANC feedback loop in FIG. 2 as an example. As shown in FIG. 2, for the convenience of description, for example, the position of a speaker (horn) spk is defined as an ERP, the position of an error microphone is defined as an EEP, and an HRTF from the ERP to the EEP is defined as H.sub.0. A controller may be implemented as an AEC system. For a more intuitive understanding, the AEC system shown in FIG. 6 is taken as an example for description. For example, the controller may implement an NLMS-based adapted algorithm.

[0032] In order to describe an individual difference (HRTF) between the pinna (speaker (spk) at the ERP) and the ear canal (error microphone (mic) at the EEP) that affects the spatial fidelity, a feedback loop in FIG. 2 needs to be used to separately apply an HRTF compensation curve (that is, an inversed function of H.sub.0).

[0033] Next, the process of the pinna identification will be described in detail.

[0034] First, the HRTF from the ERP to the EEP (H.sub.0) is obtained. This process may be implemented in two ways. One way includes: capturing any reference audio signal from the earphone spk, recording the signal by the error microphone, and then transforming the signal from a time domain (TD) to an FD through fast Fourier transform (FFT). Another way includes: obtaining the adapted HRTF (H.sub.0) using an AEC adapted loop. For the convenience of understanding, FIG. 6 illustrates an adapted AEC using NLMS. Those skilled in the art can understand that the present disclosure may also use an AEC adapted loop using other adapted algorithms (such as RLS and VLMS).

[0035] Next, an HRTF compensation curve (H.sub.0.sup.−1) is obtained by curve fitting of H.sub.0. For example, if a known filter is given, curve fitting may be modeled as an arbitrary amplitude filter design.

[0036] Finally, an audio signal in the FD is multiplied by the HRTF compensation curve (H.sub.0.sup.−1) before is the audio signal is reproduced through the speaker spk.

[0037] FIGS. 3 and 4 are schematic diagrams of left-ear and right-ear HRTFs of a left-ear pinna identification and a right-ear pinna identification obtained respectively by earless measurement, different user measurement, and artificial head measurement in a method according to an embodiment of the present disclosure, respectively. The earless measurement includes: placing sound-absorbing foam on the top of an earshield of an earphone. It can be seen from FIGS. 3 and 4 that the method for HRTF adaptation of the present disclosure may obtain respective HRTFs (that is, different frequency response curves in the figures) based on earlessness, different users and artificial heads, so that personalized HRTF measurement of different test targets can be implemented.

[0038] Shadowing Identification (Far Field to EEP)

[0039] A system model of the shadowing identification may be the same as a feedforward loop designed by an ANC. FIG. 5 shows a schematic diagram of an ANC feedforward loop. The system model of the shadowing identification of the present disclosure will take the ANC feedforward loop in FIG. 5 as an example for the convenience of understanding.

[0040] Given the combined binaural feedforward ANC, different HRTFs between left and right ERPs/EEPs describe a shadowing effect of the head, which will be separately and adaptively compensated according to a 3D audio.

[0041] A mono feedforward ANC shown in FIG. 5 is taken as an example. A far-field HRTF from a noise source to a reference microphone (ERP) and a near-field HRTF from the ERP to an error microphone (EEP) are shown in FIG. 5. The reference microphone and the error microphone usually have the same characteristics.

[0042] FIG. 5 illustrates various components and signal transmission paths of the mono feedforward ANC. Reference microphone 2 located outside earphone 1 is configured to measure the far-field HRTF. Error microphone 3 located inside earphone 1 is configured to measure the near-field HRTF. Noise 4 entering the system is filtered into signal 5 by the earshield of the earphone. Signal 6 played by an earphone speaker is preferably a reverse signal of signal 5. P(z) in the FIG. 5 represents the far-field HRTF from the noise source to the reference microphone (ERP). N(z) represents the low-pass characteristic of the earshield of the earphone, which has a passive isolation function. H.sub.0 represents the near field HRTF from the earphone speaker (almost at the ERP) to the error microphone (EEP). The controller may be implemented as an AEC system, for example, the AEC system in FIG. 6. Similarly, those skilled in the art can understand that FIG. 6 only illustrates an adapted AEC using NLMS. The present disclosure may also use an AEC adapted loop using other adapted algorithms (such as RLS and VLMS).

[0043] Assuming that the noise source captured by reference microphone 2 will be regarded as a reference signal X(Z) and the signal captured by error microphone 3 will be regarded as an input signal Y(Z), it is finally estimated that an echo cancellation transfer function H(z) will be associated with N(z) (such as a low-pass filter in the FD) and H.sub.0.

[0044] According to an aspect of the present disclosure, a priori estimation may be incorporated into the ANC feedforward loop to obtain better and more stable performance based on measurement results. In actual operation, the low-pass filter (for example, the cutoff frequency is 3 kHz) and H.sub.0 derived through a feedback loop will be multiplied by the reference microphone signal X(Z) in the FD within the NLMS AEC system.

[0045] In a solution of the present disclosure, after a pinna identification and a shadowing identification are performed on the system, an obtained adapted HRTF may be applied to an ANC earphone to achieve accurate and personalized HRTF measurement and adaptation during the use of the ANC earphone. Further, a hybrid (feedback+feedforward) adapted ANC earphone design may be provided to adapt to different adaptation states.

[0046] System Compensation

[0047] For a 3D virtual surround earphone, for example, in order to finally reproduce the HRTF for a customer, reverse mapping is required to measure and map a near-field TF and an incomplete directional shadowing function to a 360-degree model. This process may be modeled as a sparsity problem in the field of statistical analysis. For example, a reference head model is used to collect a large number of near-field and far-field measurements and train a deep neural network (DNN). Data is collected in the form of impulse responses that are located around the space and have different degrees and distances.

[0048] Then, during calculation, the measured shadowing function and pinna response may be used as an input to generate a 360-degree HRTF rendering matrix to achieve a system compensation effect.

[0049] In the binaural hearing modeling research, the HRTF may generally be divided into two free-field spatial characteristics, namely a far field (for example, the distance is greater than 1.0 m) and a near field (for example, the distance is less than 1.0 m) according to the distance from the sound source to the center of the head. The manner in which the source of a free-field sound is determined mainly depends on the following three acoustic cues: (a) interaural time difference (ITD), (b) interaural intensity difference (ILD), and (c) acoustic filtering, that is, a spectrum cue derived from the shapes of the ears, head and body of a person. The near-field HRTF depends on a human body structure, especially an external ear structure composed of the pinna, the ear canal and the ear drum.

[0050] FIGS. 6 and 7 illustrate schematic diagrams of an acoustic echo cancellation system implemented in an FD and an adaptation process of H(Z) implementing the echo cancellation system, respectively. Those skilled in the art can understand that FIGS. 6 and 7 are intended to help understand the technology of the present disclosure, rather than limiting the technology of the present disclosure in a narrow sense. It will be further described below with reference to FIGS. 6 and 7.

[0051] The FD has become the first choice for an AEC, because the FD can implement a high-order adapted filter H(z) with high convergence speed and medium computational complexity. The two basic modules of NLMS AEC filtering and adaptation are shown in FIGS. 6 and 7. FIGS. 6 and 7 illustrate the case of a speaker-peripheral space-microphone (LEM) system. Those skilled in the art can understand that FIGS. 6 and 7 are intended to illustrate the basic principles through examples, rather than specific limitations.

[0052] In the FD, fast convolution/correlation technologies are usually used to implement the AEC. The cross-correlation between an error signal e(i) and a reference signal x(i) in the TD is equal to E(z) in the FD multiplied by X*(z) (X*(z) is the conjugate of X(z)). FIG. 7 shows that the reverse power spectrum density (PSD) of the reference signal x(i)Φ.sub.xx.sup.−1 (z) is used as the normalization of gradient. A step size μ(z) in the FD guarantees the robustness of H(z).

[0053] An AEC version as shown in FIGS. 6 and 7 can only control a linear part of an LEM system, and additional residual echo suppression (RES) is usually used to further reduce echo to keep it within the range of a (linear) AEC errore(i). However, it is well known that the RES characterizes a non-linear signal processing stage and has inherent shortcomings. It may produce acoustic artifacts called musical tones, which need to be avoided.

[0054] The following is a brief description of some basic mathematical principles of NLMS AEC. Those skilled in the art can understand that the following description is only to help understand the basic principles of NLMS AEC, rather than specific limitations.

[0055] Based on Wiener-Khinchin and Parseval theorems,

Φ.sup.xx(z)=X(z).Math.X*(z)

[0056] The adaptation of an echo canceller H(z) is implemented as follows, for example:

[00001]$H\left(z\right)=H\left(z\right)+\mu \left(z\right)\frac{E\left(z\right).Math.X^{*}\left(z\right)}{{\Phi }^{\mathrm{xx}}\left(z\right)}=H\left(z\right)+\mu \left(z\right)\frac{E\left(z\right)}{X\left(z\right)}$

[0057] An optimal step size μ.sub.opt(z) is derived based on a relationship between E(z) and X(z), which will be simulated, analyzed and fine-tuned in practice. In practice, larger μ.sub.opt(z) will converge quickly, but may cause instability. Smaller μ.sub.opt(z) will converge slowly, but sometimes it cannot meet practical applications.

[0058] The present disclosure further provides a system, which includes a memory and a processor. The memory is configured to store computer-readable instructions. The processor is configured to perform a system identification when executing the computer-readable instructions. The system identification includes a pinna identification and a shadowing identification.

[0059] The description of the implementations has been presented for the purposes of illustration and description. Appropriate modifications and changes of the implementations can be implemented in view of the above description or can be obtained through practical methods. For example, unless otherwise indicated, one or more of the methods described may be performed by a combination of suitable devices and/or systems. The method can be performed in the following manner: using one or more logic devices (for example, processors) in combination with one or more additional hardware elements (such as storage devices, memories, circuits, hardware network interfaces, etc.) to perform stored instructions. The method and associated actions can also be executed in parallel and/or simultaneously in various orders other than the order described in this application. The system is illustrative in nature, and may include additional elements and/or omit elements. The subject matter of the present disclosure includes all novel and non-obvious combinations of the disclosed various methods and system configurations and other features, functions, and/or properties.

[0060] As used in this application, an element or step listed in the singular form and preceded by the word “one/a” should be understood as not excluding a plurality of said elements or steps, unless such exclusion is indicated. Furthermore, references to “one implementation” or “an example” of the present disclosure are not intended to be interpreted as excluding the existence of additional implementations that also incorporate the recited features.