Methods for obtaining and reproducing a binaural recording

Abstract

In one aspect, a method for providing a binaural recording to a listener with a head applied in a hearing system, whereas the binaural recording is listened to using a hearing device and whereas the binaural recording consists of a left binaural ear signal intended for a left ear of the listener, and a right binaural ear signal intended for a right ear of the listener, comprises determining a head orientation, determining a source direction of the binaural recording with respect to the head orientation, detecting a change of the head orientation to a new head orientation, adapting the binaural recording considering the source direction of the binaural recording and the new head orientation.

Claims

1. A method for providing a binaural recording to a listener with a head applied in a hearing system, whereas the binaural recording is listened to using a hearing device and whereas the binaural recording consists of a left binaural ear signal intended for a left ear of the listener, and a right binaural ear signal intended for a right ear of the listener, the method comprising: providing a binaural recording consisting of the left binaural ear signal and the right binaural ear signal to the hearing system, determining a head orientation of the listener, determining a source direction of the provided binaural recording with respect to the head orientation, detecting a change of the head orientation to a new head orientation, and adapting the provided binaural recording based on a relationship between the head orientation and the difference between the left binaural ear signal and the right binaural ear signal for a given source direction.

2. A method for recording a binaural recording applied in a hearing system, whereas the binaural recording is recorded using a recording device carried on a head of a recordist, and whereas the binaural recording consists of a left binaural ear signal received by the recording device near and/or at a left ear of the recordist, and a right binaural ear signal received by the recording device near and/or at a right ear of the recordist, the method comprising: recording the binaural recording consisting of the left binaural ear signal and the right binaural ear signal by the recording device, determining a head orientation of the recordist, determining a source direction of the recorded binaural recording with respect to the head orientation, detecting a change of the head orientation to a new head orientation, and adapting the recorded binaural recording based on a relationship between the head orientation and the difference between the left binaural ear signal and the right binaural ear signal for a given source direction.

3. The method according to claim 1, wherein the step of detecting a change of the head orientation to a new head orientation comprises detecting a rotational movement of the head.

4. The method according to claim 1, wherein the step of determining a source direction of the binaural recording with respect to the head orientation, and the step of adapting the binaural recording considering the source direction of the binaural recording and the new head orientation, comprise consulting a mapping table relating the head orientation to a difference between the left binaural ear signal and the right binaural ear signal for a given source direction.

5. The method according to claim 1, wherein the step of determining a source direction of the binaural recording with respect to the head orientation comprises determining a difference between the left binaural ear signal and the right binaural ear signal and comparing this difference to a mapping table relating the head orientation to the difference between the left binaural ear signal and the right binaural ear signal for a given source direction.

6. The method according to claim 1, wherein the step of adapting the binaural recording considering the source direction of the binaural recording and the new head orientation comprises determining the new head orientation with respect to the source direction of the binaural recording and comparing this value of head orientation to a mapping table relating the head orientation to the difference between the left binaural ear signal and the right binaural ear signal for a given source direction.

7. The method according to claim 4, wherein the mapping table relating the head orientation to the difference between the left binaural ear signal and the right binaural ear signal for a given source direction relates the head orientation to the difference in amplitude and/or to the difference in phase between the left binaural ear signal and the right binaural ear signal for a range of frequencies of the right and left binaural ear signals and for a given source direction.

8. The method according to claim 4, wherein the mapping table relating the head orientation to the difference between the left binaural ear signal and the right binaural ear signal for a given source direction is adaptable to the head of the listener and/or to the ears of the listener and/or to an environment of the listener.

9. A method for providing a binaural recording to a listener, whereas the binaural recording is recorded using a recording device carried on a head of a recordist, and whereas the binaural recording consists of a left binaural ear signal received by the recording device near and/or at a left ear of the recordist, and a right binaural ear signal received by the recording device near and/or at a right ear of the recordist, the method comprising: providing a binaural recording consisting of the left binaural ear signal and the right binaural ear signal to the hearing system, determining a head orientation of the listener, determining a source direction of the provided binaural recording with respect to the head orientation, detecting a change of the head orientation to a new head orientation, adapting the provided binaural recording considering the determined source direction of the binaural recording and the detected new head orientation, and wherein the binaural recording is provided to the listener by a method according to claim 1.

10. A hearing system configured to perform the method according to claim 1, comprising a hearing device and/or recording device and a head motion tracking device configured to determine a head orientation and/or a change of the head orientation of the listener and/or recordist.

11. The hearing system according to claim 10, wherein the hearing device and/or recording device is configured to be worn by a listener.

12. The hearing system according to claim 10, wherein the hearing system comprises a storage device for storing a mapping table relating the head orientation to a difference between the left binaural ear signal and the right binaural ear signal for a given source direction.

13. A hearing system configured to perform the method according to claim 2, comprising a hearing device and/or recording device and a head motion tracking device configured to determine a head orientation and/or a change of the head orientation of the listener and/or recordist.

Description

(1) In the drawings:

(2) FIG. 1 shows a flowchart of the steps of the methods, according to a preferred embodiment of the invention,

(3) FIG. 2 shows a recordist with a recording device, recording a binaural recording having two different head orientations, according to a preferred embodiment of the invention,

(4) FIG. 3 shows a listener with a hearing device, listening to a binaural recording having two different head orientations, according to a preferred embodiment of the invention,

(5) FIG. 4 shows a functional diagram of the methods, according to a preferred embodiment of the invention, and

(6) FIG. 5 shows an illustration of two different mapping tables, according to a preferred embodiment of the invention.

(7) FIG. 1 shows a flowchart of the steps of the method for providing a binaural recording to a listener applied in a hearing system, according to a preferred embodiment of the invention. Furthermore, FIG. 1 shows the steps of the method for recording a binaural recording applied in a hearing system, according to a preferred embodiment of the invention. The steps of the two methods are the same, independent of an incorporation in the process of listening to a binaural recording, or of an incorporation in the process of recording a binaural recording.

(8) FIG. 2 shows a recordist 10 with a hearing system 12, according to a preferred embodiment of the invention. The hearing system 12 comprises of a recording device 14, in this embodiment of the invention a headset with two in-ear microphones. Incorporated into the headset is a head tracking device 16, being configured to determine a orientation 18 of the head 20 of the recordist 10. The recordist 10 is in the process of recording a binaural recording of the sound, that is emitted by a sound source 22 with the hearing system 12. The recordist 10 in FIG. 2 has two different orientations of the head 20.

(9) Hereinafter, the individual steps of the method for recording a binaural recording are briefly described with reference to the flowchart in FIG. 1, and the recordist 10 in FIG. 2.

(10) The first step of the method, S100, is to determine the orientation 18 of the head 20 of the recordist 10. This orientation 18 of the head 20 of the recordist 10 is shown on the left side of FIG. 2. In this preferred embodiment of the invention the orientation 18 defines an origin of a reference frame and is therefore zero degrees.

(11) In a next step of the method, S200, a source direction 24 of the binaural recording is determined with respect to the head orientation 18 of the recordist 10. The source direction 24 is a direction towards the sound source 22. It is determined without any knowledge of the environment of the recordist 10, but by analyzing the binaural recoding. After this step, the angle 26 between the orientation 18 of the head 20 of the recordist 10 and the source direction 24 is known.

(12) In a further step of the method, S300, a change of head orientation 28 is detected. This change of head orientation 28 leads to a new orientation 30 of the head 20 of the recordist 10. This new orientation 30 of the head 20 of the recordist 10 is shown on the right side of FIG. 2.

(13) In the final step of the method, S400, the binaural recording is adapted, considering the source direction 24 of the binaural recording and the new head orientation 30. Therefore, this step involves determining the angle 32 between the new head orientation 30 and the source direction 24.

(14) FIG. 3 shows a listener 34 with a hearing system 12, according to another preferred embodiment of the invention. The hearing system 12 comprises of a hearing device 36, in this embodiment of the invention a headset. Incorporated into the headset is a head tracking device 16, being configured to determine a orientation 18 of the head 38 of the listener 34. The listener 34 is in the process of listening to a binaural recording. The listener 34 in FIG. 3 has two different orientations of the head 38.

(15) Hereinafter, the individual steps of the method for providing a binaural recording to a listener 34 are described with reference to the flowchart in FIG. 1, the listener 34 in FIG. 3, the functional diagram in FIG. 4 and the mapping table in FIG. 5.

(16) After having determined the orientation 18 of the head 38 of the listener 34 in the first step of the method S100, and shown on the left side of FIG. 3, in the next step S200 the a source direction 24 of the binaural recording is determined with respect to the head orientation 18 of the listener 34. The source direction 24 is a direction towards the perceived sound source 22. It is determined by analyzing the binaural recoding.

(17) Referring to FIG. 4, for the purpose of analyzing the binaural recoding, the time domain signal x.sub.l(k) intended for the left ear of the listener 34 and the time domain signal x.sub.r(k) intended for the right ear of the listener 34 of the binaural recording are first transformed into a frequency domain signal by a discrete Fourier transformation (DFT). The signals in the frequency domain are denoted X.sub.r(λ, μ) and X.sub.l(λ, μ).

(18) The analysis involves determining an interaural level difference (ILD) and an interaural phase difference (IPD) of the binaural recording:

(19) $\mathrm{ILD}\left(\lambda ,\mu \right)=.Math.\frac{X_r\left(\lambda ,\mu \right)}{X_1\left(\lambda ,\mu \right)}.Math.$$\mathrm{IPD}\left(\lambda ,\mu \right)=\arg \left(\frac{X_r\left(\lambda ,\mu \right)}{X_1\left(\lambda ,\mu \right)}\right)={\varphi }_r\left(\lambda ,\mu \right)-{\varphi }_1\left(\lambda ,\mu \right),$

(20) where X.sub.r,l(λ, μ) denotes the binaural ear signal, in a frequency bin μ, at a time instance A.

(21) The ILD(λ,μ) and IPD(λ, μ) can be compared to and in a mapping table 40.

(22) Referring to FIG. 5, the mapping table 40 creates a link between the head orientation 18, 30 and the difference in amplitude () and the difference in phase (). The specific values for the and that are stored in the mapping table 40 can be generated by an analytic spherical head model and/or by previously measured head-related transfer functions (HRTFs). The HRTFs describe the modification of the signal by the head 38 and/or ears of the listener 34, depending on the direction of sound incidence:
X.sub.l(λ,μ)=H.sub.l(λ,μ).Math.S(λ,μ)
X.sub.r(λ,μ)=H.sub.r(λ,μ).Math.S(λ,μ),

(23) where H.sub.i(λ,μ) with i∈{l, r} represent the HRTFs for the left and right ear, which can be divided into magnitude and phase components as:
H.sub.i(λ,μ)=|H.sub.i(λ,μ).Math.e.sup.iφ.sup.i.sup.(λ,μ),

(24) The mapping table 40 contains the information of measured and or analytically calculated HRTFs, as a function of φ, the angle between the orientation 18, 30 of the head and the source direction 24:

(25) $\left(\mu ,\phi \right)=\frac{.Math.{\hat{H}}_r\left(\mu ,\phi \right).Math.}{.Math.{\hat{H}}_1\left(\mu ,\phi \right).Math.}$$\left(\mu ,\phi \right)=\arg \left(\frac{{\hat{H}}_r\left(\mu ,\phi \right)}{{\hat{H}}_1\left(\mu ,\phi \right)}\right)={\hat{\varphi }}_r\left(\mu ,\phi \right)-{\hat{\varphi }}_1\left(\mu ,\phi \right)$

(26) Not only the resulting and can be stored in the mapping table 40, but also the HRTFs as well.

(27) FIG. 5 shows two examples of mapping tables 40. On the left are the (top) and (bottom) determined from measured HRTFs. On the right are the (top) and (bottom) determined from an analytic model.

(28) The determination of the source direction 24, S200, involves a minimization approach, where the and in the mapping table 40 are determined that best fit the ILD and IPD of the binaural recording. Particularly the minimization has the following form:

(29) ${\phi }_{\mathrm{orig}}\left(\lambda ,\mu \right)=\arg \underset{\phi }{\min }\frac{\left(\mu ,\phi \right)}{\mathrm{ILD}\left(\lambda ,\mu \right)}+\frac{ILD\left(\lambda ,\mu \right)}{\left(\mu ,\phi \right)}-2\cos \left(\mathrm{IPD}\left(\lambda ,\mu \right)-IPD\left(\mu ,\phi \right)\right),$

(30) where φ.sub.orig corresponds to the angle 26 between the orientation 18 of the head 38 of the listener 34 and the source direction 24.

(31) In a further step of the method, S300, a change of head orientation 28 to a new orientation 30 of the head 38 of the listener 34 is detected. Therefore, the angle 32 between the new head orientation 30 and the source direction 24 is known. This new orientation 30 of the head 38 of the listener 34 is shown on the right side of FIG. 3.

(32) In the final step of the method, S400, the binaural recording is adapted, considering the source direction 24 of the binaural recording and the new head orientation 30. Thereby, also the mapping table 40 is consulted. The binaural recoding is modified as follows:
Y.sub.i(λ,μ)=X.sub.i(λ,μ).Math.G.sub.i(λ,μ),

(33) where G.sub.i(λ, μ) are complex coefficients which manipulate the ILD and IPD of the binaural recording X.sub.i(λ, μ) according to
G.sub.i(λ,μ)=G.sub.i.sup.ILD(λ,μ).Math.G.sub.i.sup.IPD(λ,μ)

(34) The phase modifications are

(35) $G_r^{IPD}\left(\lambda ,\mu \right)=e^{-i\frac{\Delta \mathrm{IPD}\left(\lambda ,\mu \right)}{2}}$$G_1^{IPD}\left(\lambda ,\mu \right)=e^{i\frac{\Delta \mathrm{IPD}\left(\lambda ,\mu \right)}{2}},$

(36) whereas the value of ΔIPD(λ, μ) is determined by consulting the mapping table 40:
ΔIPD(λ,μ)=(μ,φ.sub.dest(λ,μ))−(μ,φ.sub.orig(λ,μ)).

(37) φ.sub.dest denotes the angle 32 between the new head orientation 30 and the source direction 24, which was determined in the previous step of the method (S300).

(38) The magnitude of the complex coefficient modifies the ILDs of the binaural recording and is obtained directly from the HRTFs in the mapping table according to

(39) $G_i^{ILD}\left(\lambda ,\mu \right)=\frac{.Math.{\hat{H}}_i\left(\mu ,{\phi }_{dest}\left(\lambda ,\mu \right)\right).Math.}{.Math.{\hat{H}}_i\left(\mu ,{\phi }_{orig}\left(\lambda ,\mu \right)\right).Math.}.$

(40) The modifications get less invasive for smaller values of Δφ and the signal is not modified at all for Δφ=0.

(41) After adapting the binaural signal, the modified signal Y.sub.i(λ, μ) is transformed back to the time domain by applying a inverse Discrete Fourier Transformation (IDFT).

REFERENCE SIGNS

(42) 10 Recordist 12 Hearing system 14 Recording device 16 Head tracking device 18 Orienation of head 20 Head of recordist 22 Sound source 24 Source direction 26 Angle between head orientation and source direction 28 Change of head orientation 30 New head orientation 32 Angle between new head orientation and source direction 34 Listener 36 Hearing device 38 Head of listener 40 Mapping table