Method and device for generating and providing an audio signal for enhancing a hearing impression at live events
10643597 · 2020-05-05
Assignee
Inventors
Cpc classification
G10K2210/3225
PHYSICS
G10K11/17885
PHYSICS
G10K11/17827
PHYSICS
International classification
G10K11/16
PHYSICS
Abstract
A method for generating and providing an audio signal, including receiving a first audio signal via an external microphone of a headphone or earphone, and receiving a second audio signal via a wireless interface. The first audio signal includes a portion reproduced via loudspeakers. The second audio signal corresponds to the portion reproduced via loudspeakers and is received before the corresponding portion of the first audio signal. A propagation time difference is determined between the first audio signal and the second audio signal. The second audio signal is modified by adaptive filtering and temporal shifting such that the propagation time difference between the first and second modified audio signal is substantially compensated. The adaptive filtering models an acoustic transmission of the first audio signal and a modified second audio signal is obtained. The modified second audio signal is inverted, then it is provided via the headphone or earphone.
Claims
1. A method for generating and providing an audio signal, comprising: receiving a first audio signal via a microphone, wherein the microphone is an external microphone of a headphone or earphone, and wherein the first audio signal comprises a portion that is reproduced via loudspeakers; receiving a second audio signal via a wireless interface, the second audio signal corresponding to the portion that is reproduced via loudspeakers and being received prior to the corresponding portion of the first audio signal; determining a propagation time difference between the first audio signal and the second audio signal; modifying the second audio signal by adaptive filtering and temporal shifting such that the propagation time difference between the first and second audio signal is substantially compensated, wherein the adaptive filtering models an acoustic transmission of the first audio signal and wherein a modified second audio signal is obtained; inverting the modified second audio signal, wherein an inverted modified second audio signal is obtained; and providing the inverted modified second audio signal via the headphone or earphone.
2. The method according to claim 1; wherein the second audio signal is additionally provided via the headphone or earphone.
3. The method according to claim 1; wherein the wireless interface is a WLAN interface or a mobile network interface according to the 3G, 4G, or 5G standard.
4. The method according to claim 1; wherein said determining the propagation time difference comprises a cross-correlation in a frequency domain.
5. The method according to claim 1; wherein the second audio signal has two or more audio tracks, and said modifying by adaptive filtering comprises attenuating, removing, or partially removing at least one audio track of the second audio signal.
6. The method according to claim 5; wherein a data signal is additionally received via the wireless interface, the data signal comprising information about the two or more audio tracks of the second audio signal.
7. The method according to claim 1, further comprising: a step of further modifying at least one audio signal of the first and second audio signals prior to the adaptive filtering; wherein the further modifying comprises adding, partially adding, amplifying, attenuating, removing, partially removing, or a combination thereof, at least one adjustable frequency or at least one adjustable frequency range to or from the at least one audio signal.
8. A mobile device configured to execute the method according to claim 1.
9. A non-transient computer-readable storage medium having stored thereon instructions configured to instruct a computer or computer processor to execute the method according to claim 1.
10. A computer software product adapted for configuring a computer or computer processor to execute the method of claim 1.
11. The software product according to claim 10; wherein the computer or computer processor is part of a mobile electronic device.
12. A device for generating and providing an audio signal, the device comprising: a microphone configured to record a first audio signal, wherein the microphone is an external microphone of a headphone or earphone; a wireless interface module configured to receive a second audio signal, wherein the second audio signal corresponds to a portion of the first audio signal and is received prior to the corresponding portion of the first audio signal; an audio processing unit configured to process the second audio signal using the first audio signal; and an output unit configured to provide the processed second audio signal to the headphone or earphone for being reproduced; wherein the audio processing unit comprises: first electronic circuitry configured to determine a propagation time difference between the first and second audio signal; second electronic circuitry configured to modify the second audio signal by adaptive filtering and temporal shifting, such that the determined propagation time difference between the first and the second audio signal is compensated, wherein a modified second audio signal is obtained; and third electronic circuitry configured to invert the modified second audio signal, wherein the processed second audio signal is obtained.
13. The device according to claim 12; wherein the second audio signal is additionally reproduced via the headphone or earphone.
14. The device according to claim 12; wherein the propagation time difference is determined by a cross-correlation in the frequency domain.
15. A method for generating and providing an audio signal, comprising: receiving a first audio signal via a microphone, wherein the first audio signal comprises a portion that is reproduced via loudspeakers; receiving a second audio signal via a wireless interface, the second audio signal corresponding to the portion that is reproduced via loudspeakers and being received prior to the corresponding portion of the first audio signal; determining a propagation time difference between the first audio signal and the second audio signal; modifying the second audio signal, the modifying comprising adaptive filtering and temporal shifting such that the propagation time difference between the first and second modified audio signal is substantially compensated, wherein the adaptive filtering models an acoustic transmission of the first audio signal, and wherein a modified second audio signal is obtained; and providing the modified second audio signal via a headphone or earphone.
16. The method according to claim 15; wherein the wireless interface is a WLAN interface or a mobile network interface according to the 3G, 4G, or 5G standard.
17. The method according to claim 15; wherein said determining the propagation time difference comprises a cross-correlation in a frequency domain.
18. The method according to claim 15; wherein the second audio signal has two or more audio tracks, and said modifying by adaptive filtering comprises attenuating, removing, or partially removing at least one audio track of the second audio signal.
19. The method according to claim 18; wherein a data signal is additionally received via the wireless interface, the data signal comprising information about the two or more audio tracks of the second audio signal.
20. The method according to claim 15, further comprising: a step of further modifying at least one audio signal of the first and the second audio signals prior to the adaptive filtering; wherein the further modifying comprises adding, partially adding, amplifying, attenuating, removing, partially removing, or a combination thereof, at least one adjustable frequency or at least one adjustable frequency range to or from the at least one audio signal.
21. The method according to claim 15, further comprising steps of: receiving a user input for selecting an operating mode; and depending on the received user input, inverting the modified second audio signal; wherein if the user input corresponds to a first operating mode, the modified second audio signal is not inverted before being provided via the headphone or earphone, and if the user input corresponds to a second operating mode, the modified second audio signal is inverted before being provided via the headphone or earphone.
22. A device for generating and providing an audio signal, the device comprising: a microphone configured to record a first audio signal; a wireless interface module configured to receive a second audio signal, wherein the second audio signal corresponds to a portion of the first audio signal and is received prior to the corresponding portion of the first audio signal; an audio processing unit configured to process the second audio signal using the first audio signal; and an output unit configured to provide the processed second audio signal to a headphone or earphone for being reproduced; wherein the audio processing unit comprises: first electronic circuitry configured to determine a propagation time difference between the first and second audio signal; second electronic circuitry configured to modify the second audio signal by adaptive filtering and temporal shifting, such that the determined propagation time difference between the first and the second audio signal is compensated, wherein the processed second audio signal is obtained.
23. The device according to claim 22; wherein the second audio signal is additionally reproduced via the headphone or earphone.
24. The device according to claim 22; wherein the propagation time difference is determined by a cross-correlation in the frequency domain.
25. The method according to claim 22, further comprising: user input means configured to receive user input; operating mode control means configured to select an operating mode, based on the user input received by the user input means; and third electronic circuitry configured to invert the modified second audio signal, depending on the selected operating mode; wherein in a first operating mode the modified second audio signal is not inverted and the processed second audio signal corresponds to the modified second audio signal, and wherein in a second operating mode the modified second audio signal is inverted and the processed second audio signal corresponds to an inverted modified second audio signal.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
(1) Details and further advantageous embodiments may be better understood by those skilled in the art by reference to the accompanying figures.
(2)
(3)
(4)
(5)
(6)
(7)
(8)
(9)
(10)
(11)
(12)
DETAILED DESCRIPTION OF EMBODIMENTS
(13) It is to be understood that the figures and descriptions of the present invention have been simplified to illustrate elements that are relevant for a clear understanding of the present invention, while eliminating, for purposes of clarity, many other elements which are conventional in this art. Those of ordinary skill in the art will recognize that other elements are desirable for implementing the present invention. However, because such elements are well known in the art, and because they do not facilitate a better understanding of the present invention, a discussion of such elements is not provided herein.
(14) The present invention will now be described in detail on the basis of exemplary embodiments.
(15)
(16) The audio mixer 22 then transmits the mixed audio signals obtained from the audio signals that were received from the stage receiver 20 to stage loudspeakers 24. The audience 30 hears the sound signal reproduced by these loudspeakers 24. Further, the audio mixer 22 transmits the audio signals to a wireless interface 26, which radiates the signals wirelessly. These signals correspond to an audio signal that may have a plurality of audio tracks.
(17) In addition, the mixed audio signals are in this example transmitted directly or via the wireless interface 26 to a mobile radio network 28 or a similar service network, which also broadcasts the audio signals having one or more audio tracks by a further wireless interface. Furthermore, viewers 30 are depicted who each carry a mobile device 32 according to the invention, which is set up to receive audio signals either directly via the wireless interface 26 or via the mobile radio network 28. In this example it is also possible to use the mobile radio network 28 for providing additional input signals from remote performers 34 to the audio mixer 22, as well as providing the audio and video signals of the performers 12,34 to a remote audience 36. Thus, different artists in a joint performance need not be on the same stage 14 but may be interconnected electronically via the service network 28.
(18) In some embodiments, the invention enables e.g. two viewers 30a,30b who are at a rock concert near loudspeakers 24 to talk and hear each other, despite a high sound pressure level of the loudspeakers.
(19)
(20) An advantage of the invention in some embodiments is that also the microphones 40 are located on this headphone 48. Therefore, the distance between the microphones 40 and the user's meatus is substantially known, constant and very small. Thus, the ambient sound 38 picked up by the microphones 40 is substantially identical with the ambient sound that the listener 30 hears despite the headphone, and in particular both have the same phase. Especially in a noisy environment like a rock concert, headphones almost always let a part of the ambient sound through. Phase differences between the ambient sound directly heard and the ambient sound picked up by the microphone and (inversely) reproduced by the headphone have a disturbing effect. Such phase differences are frequency dependent and may already be noticeable e.g. if the distance between the microphone 40 and the ear changes by few centimeters. Therefore, this embodiment has the advantage that the phase of the ambient sound signal picked up by the microphone is known and constant, different from another variant where e.g. a microphone of a smartphone is used. In this way, the sound reproduced by the headphone can be better synchronized with the ambient sound.
(21)
(22)
(23)
(24)
(25)
(26) In an embodiment, such device for generating and providing an audio signal comprises at least one microphone 40 for receiving a first audio signal 42, wherein the microphone 40 is an external microphone of a headphone or earphone, a wireless interface 26 for receiving a second audio signal, an audio processing unit 45 with electronic circuitry for modifying the second audio signal using the first audio signal, e.g. by an adaptive filter for filtering the second audio signal and inverting the filtered second audio signal, and an output unit 85 for outputting the inverted filtered second audio signal towards a sound transducer. The audio processing unit 45 may comprise an electronic circuit 50 for determining and modifying the propagation delay difference, and an adaptive filter which in principle fulfils a function comparable to the amplifiers 54 and the adder/subtractor 56. Optionally, the device may comprise further components, e.g. a pre-filter, ADC, DAC, etc. Single, several or all of the above-mentioned components may be implemented on one or more software-configurable processors. In the headphone shown in
(27)
(28) In an embodiment, the first and second audio signals may be additively combined to obtain the modified second audio signal 46. Both parts may be differently weighted herein, leading to different mixing ratios.
(29)
(30)
(31) Generally, embodiments using an external microphone of an earphone or headphone are particularly advantageous if the first and second audio signal are added, while in other cases such as radio-assisted ANC in principle also a microphone of a different mobile device can be used.
(32) In a variant, the invention relates to a method for emitting an audio signal with steps of receiving from an audio mixer a first audio signal having two or more audio tracks, generating or determining information about the two or more audio tracks of the first audio signal, and emitting the received first audio signal as a second audio signal together with a data signal via a wireless interface, the second audio signal having two or more audio tracks and the data signal comprising the generated or determined information. In an embodiment, the wireless interface is a WLAN interface or mobile radio network interface according to the 3G, 4G or 5G standard. In another embodiment, the invention relates to a software product comprising instructions that when executed on a computer or processor configure the computer or processor to execute a method as described above. In a further embodiment, the invention relates to a device for emitting an audio signal that is adapted for executing the method, in particular a mixing console or mixer.
(33) In an embodiment, the invention relates to a storage device or storage medium having stored thereon software instruction for configuring a computer or processor to execute a method as described above. In an embodiment, the invention relates to a system with at least one device for providing an audio signal, as described above, and a device for emitting an audio signal, as described above.
(34) Embodiments described above may be combined if such combination is meaningful. Devices and units described above may be implemented in hardware, software or a combination thereof, such as one or more software-configured processors.
(35) While this invention has been described in conjunction with the specific embodiments outlined above, it is evident that many alternatives, modifications, and variations will be apparent to those skilled in the art. Accordingly, the preferred embodiments of the invention as set forth above are intended to be illustrative, not limiting. Various changes may be made without departing from the spirit and scope of the inventions as defined in the following claims.