Reverberation technique for 3D audio objects

Abstract

Reverberation techniques for 3D audio are disclosed. In an example method, a three-dimensional (3D) reverberation is applied to a sound object placed at a sound object position in a sound room. The sound object originates from a sound object position. A sound object signal is received. A 3D spatial room response (SRR) signal is computed corresponding to the user-selected position. A time convolution operation is performed between an audio signal of the sound object signal and the computed SRR value to generate a reverberated signal.

Claims

1. Method of applying a three-dimensional (3D) reverberation to a sound object as perceived from a listening position in a sound room, the sound object originating from a sound object position, the method comprising: receiving a sound object signal; computing a 3D spatial room response (SRR) signal corresponding to the sound object position, the computing a 3D SRR signal corresponding to the sound object position comprising: interpolating existing SRR signals stored in a database, the interpolating existing SRR values comprising performing a bi-triangular or tetrahedral interpolation between existing SRR values, and, selecting the existing SRR signals based on metadata of the sound object signal; performing a time convolution operation between an audio signal of the sound object signal and the computed SRR value to calculate a reverberated signal.

2. The method according to claim 1, further including: measuring the existing SRR signals by a 3D microphone at distinct distances from the sound source position.

3. The method according to claim 2, further including: measuring the existing SRR signals on positions of a coordinate system.

4. The method according to claim 3 the coordinate system being one of a cylindrical, a Cartesian or a spherical coordinate system.

5. The method according to claim 1, the performing a bi-triangular interpolation comprising: identifying three measurement points on a surface of two neighboring coaxial cylinders, the three measurement points being the closest to the sound object position; performing a triangulation on both neighboring coaxial cylinder surfaces.

6. The method according to claim 5, the performing a triangulation on a cylinder surface comprising combining corresponding SRR signals at the identified points with weights depending on actual distance between SRR measurement position and the sound object position.

7. The method according to claim 1, the performing a tetrahedral interpolation comprising: identifying four measurement points belonging to a surface of two different neighboring coaxial cylinders, the four measurement points being the closest to the sound object position; performing a triangulation on a volume defined by the four measurement points.

8. The method according to claim 7, the performing a triangulation in a tetrahedron comprising combining corresponding SRR signals at the identified points with weights depending on an actual distance between SRR measurement position and the sound object position.

9. The method according to claim 1, the SRR signals being room-impulse-response (RIR) signals in three dimensions.

10. A device to apply a three-dimensional reverberation to a sound object at a sound object position in a sound room, the sound object originating from a sound object position, the device comprising: a receiver to receive the sound object from the sound object position; an SRR logic to compute a 3D spatial room response (SRR) signal corresponding to the sound object position, the SRR logic being configured to interpolate existing SRR signals stored in a database, the SSR logic being further configured to select the existing SRR signals based on metadata of the sound object signal the SSR logic being further configured to interpolate existing SRR signals by performing a bi-triangular or tetrahedral interpolation between existing SRR signals; a reverberation processor to perform a time convolution operation between the sound object and the computed 3D SRR signal.

11. A device according to claim 10, the reverberation processor being configured to perform the time convolution operation between the sound object and the computed 3D SRR signal as the sound object changes position in the sound room.

12. The device according to claim 10, connectable to a database storing existing SRR signals, the SRR logic being configured to identify and retrieve existing SRR signals in the database associated with the sound object position.

13. A computer program product comprising program instructions embodied on a non-transitory medium for causing a computing system to perform a method according to claim 1.

14. A computer program product according to claim 13, embodied on a non-transitory storage medium.

15. Method of applying a three-dimensional (3D) reverberation to a sound object as perceived from a listening position in a sound room, the listening position corresponding to a microphone position at which 3D spatial room responses (SRR) are measured, the sound object originating from a sound object position, the method comprising: receiving a sound object, the sound object comprising an audio signal and associated metadata, the associated metadata comprising the sound object position; computing a 3D spatial room response (SRR) corresponding to the sound object position, the computing a SRR corresponding to the sound object position comprising: selecting existing SRRs to be interpolated based on the sound object position, storing the existing SRRs in a database together with coordinates corresponding to their capture position, and interpolating the selected existing SRRs stored in the database, the interpolating existing SRR values comprising performing a bi-triangular or tetrahedral interpolation between existing SRR values; performing a time convolution operation between the audio signal of the sound object and the computed SRR value to calculate a reverberated signal.

16. The method according claim 15, further comprising: measuring the existing SRRs by a 3D microphone at distinct distances from the sound source position.

17. The method according to claim 16, further comprising measuring the existing SRR signals on positions of a coordinate system.

18. The method according to claim 17, the coordinate system being one of a cylindrical, a Cartesian or a spherical coordinate system.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

(1) Non-limiting examples of the present disclosure will be described in the following, with reference to the appended drawings, in which:

(2) FIG. 1 schematically illustrates a measurement grid in a sound room (auditorium);

(3) FIG. 2 is a block diagram of a device to apply a three-dimensional reverberation to a sound object at a sound object position in a sound room, according to an example;

(4) FIG. 3 is a flow diagram of a method of applying a three-dimensional reverberation to a sound object at a sound object position in a sound room, according to an example.

DETAILED DESCRIPTION OF EXAMPLES

(5) It is noted that the project leading to this application has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 732130″.

(6) Depending on the nature and origin of the SRRs available in the SRRs database, a user is allowed to simulate the acoustics of some particular rooms, by adding the corresponding 3D reverberations of these rooms to some audio objects of his choice.

(7) A set of SRRs of a particular room may be composed of or include several 3D RIRs (meaning RIRs with directional cues) measured from different points of space around a listening position, yielding a ‘cartography’ in 3D of the acoustics of the room as perceived at the listening position.

(8) FIG. 1 schematically illustrates a measurement grid in a sound room.

(9) Distribution of the Measurement Points

(10) The listening position is set at a location 105 where the conductor usually stands, pointing towards the back B of the stage. It is then located on the edge of the stage (meaning that the orchestra stands in front while the audience stands in the back), centered on the left-right axis (L-R), at a height of 2 meters above the stage's floor F. The distribution of the measurement positions (indicated by cross symbols in FIG. 1) is cylindrical: they all belong to the surface of some cylinders of different radii (=different distances from the listening position), whose revolution's axis is vertical and passes by the listening position, at different height. Concretely, in an example implementation for the Auditorium of Barcelona: the distances are: 1 m, 2 m, 5 m, 10 m the azimuth are: 0°, 45°, 90°, 135°, 180°, 225°, 270°, 315° the heights are: −2 m (on the stage's floor), −1 m, 0 m, 1 m, 2 m

(11) Consequently, the SRRs database of the Auditorium of Barcelona is composed of 4×8×5=160 SRRs measurements, constituting a cartography in 3D of the acoustics of this room as perceived from the conductor's location.

(12) As the aim of the described technique is to add the reverberation of particular acoustic spaces to a number of audio objects, the SRRs contain the reverberation of the measured spaces. The reverberation time of a room (duration of the set of subsequent echoes generated by an impulsive sound emitted in the room) depends on its geometry and absorbing properties, so the length of the SRRs varies as a function of the room considered.

(13) In the example of the Auditorium of Barcelona, with the reverberation time being of 1.5 seconds, the SRRs have 72000 samples with a sampling frequency of 48 kHz. Moreover, the SRRs are RIRs in 3D, meaning that some directional cues may be added to the standard RIRs. In the present example, SRRs may be captured by a 3D microphone that features 4 capsules spread along the surface of a rigid sphere. As a result, each SRR may be composed of or include 4 signals of length 72000 samples. The audio samples may be stored in WAV 24-bits format.

(14) Thanks to the set of SRRs described previously, the present technique allows for generating the reverberation pattern in 3D of an audio source placed in any position in between the measurement points, as it would be perceived from the conductor's location. In consequence, the user may be able to position any of his sound objects in the sound room within the limits of the volume covered by the measurement points distribution.

(15) Via the user interface, the user may choose to place a specific sound object (=audio signal) in a specific position of the Auditorium of Barcelona: e.g. distance of 3 m, azimuth of 30°, height of 1.2 m.

(16) The sound object (=audio signal material) may then be aggregated with the following metadata: room name (e.g. Auditorium of Barcelona) coordinates system: cylindrical position (e.g. 3 meters distance, 30° azimuth, 1.2 meters height)

(17) The ‘room’ data allows the system to select the set of SRRs corresponding to the Auditorium of Barcelona from the SRRs database. The ‘coordinates system’ and ‘position’ data allow picking up the subset of adequate SRRs from the set of SRRs of the Auditorium of Barcelona.

(18) FIG. 2 is a block diagram of a system to apply a three-dimensional reverberation to a sound object at a sound object position in a sound room, according to an example. A sound object 205 may be positioned in a sound room within a space already covered by a measurement grid as in FIG. 1. The sound object 205 may include an audio signal and metadata related to the sound room and/or the sound object. The metadata may be sent to a first logic unit 210 (or SRR logic 210) of device 200. The metadata may include, among other information, the room name, the coordinate system and the position of the sound object in the room. The first logic unit 210 may receive the metadata and select SRRs from SRR database 215. SRR database 215 may include SRR measurements of the sound room. The SRR database 215 may form part of the device 200 or may be external and the device 200 may connect to or communicate with the SRR database 215 to retrieve the relevant SRRs. The first logic 210 may thus select the SRR measurements that correspond to positions that are closer to the position of the sound object 205 in the sound room.

(19) Computing the SRR corresponding to the chosen position may include processing the SRRs data of the subset of SRRs extracted in the previous step. This can be seen as an interpolation process, which is achieved by the first logic unit 210.

(20) In the present example, the interpolation method is bi-triangular: over the surface of two neighboring cylinders, the system looks for the 3 measurement points closest to the chosen position so as to achieves a triangulation on both cylinder's surfaces. Next, it achieves a linear interpolation between the two SRRs computed by each triangulation process.

(21) In an example, the selected position is at 3 meters distance, 30° azimuth, 1.2 meters height and the SRRs extracted from the set of SRRs of the Auditorium of Barcelona are the following: (2 m distance, 0° azimuth, 1 m height) (2 m distance, 45° azimuth, 1 m height) (2 m distance, 45° azimuth, 2 m height)
to achieve the triangulation over the surface of the cylinder of radius 2 m, and: (5 m distance, 0° azimuth, 1 m height) (5 m distance, 45° azimuth, 1 m height) (5 m distance, 45° azimuth, 2 m height)
to achieve the triangulation over the surface of the cylinder of radius 5 m.

(22) Each triangulation process may include combining the 3 corresponding SRRs signals with weights depending on the actual distance between the SRR measurement position and the position chosen by the user.

(23) Moreover, since the SRRs are RIRs in 3D, the SRR computed by the triangulation process has a 3D orientation which is different from the 3D orientation of any of the 3 actually measured SRRs. Consequently, in addition to combining the 3 actually measured SRRs, the triangulation process also achieves a mixing of the 4 different channels of the SRRs so as to modify the 3D orientation.

(24) The audio signal of the sound object 205 may be emitted to second logic unit 220. The second logic unit 220 (or reverberation processor 220) may receive the audio signal of the sound object 205 and the selected SRRs from the first logic unit 210 and perform a convolution operation to apply the 3D reverberation to the sound object.

(25) Applying the 3D reverberation to the audio signal of the sound object is done by the second logic unit 220, through a time convolution operation between the audio signal of the sound object and the different channels of the SRR issued from the previous step. This leads to a 3D reverberated sound object composed of or including 4 channels, which is later on decoded by the reproduction system 225. The end listener will then perceive the sound object as if it had been originally recorded in the chosen position (3 meters distance, 30° azimuth, 1.2 meters height, from the conductor's usual location) of the sound room, e.g. the Auditorium of Barcelona.

(26) FIG. 3 is a flow diagram of a method of applying a three-dimensional reverberation to a sound object at a sound object position in a sound room, according to an example. In block 305, a sound object is received from a sound source. Then, in block 310, a 3D SRR signal corresponding to the user-selected position may be computed. In block 315, a time convolution operation may be performed between an audio signal of the sound object and the computed 3D SRR.

(27) Although only a number of examples have been disclosed herein, other alternatives, modifications, uses and/or equivalents thereof are possible. Furthermore, all possible combinations of the described examples are also covered. Thus, the scope of the present disclosure should not be limited by particular examples, but should be determined only by a fair reading of the claims that follow. If reference signs related to drawings are placed in parentheses in a claim, they are solely for attempting to increase the intelligibility of the claim, and shall not be construed as limiting the scope of the claim.

(28) Further, although the examples described with reference to the drawings may include computing apparatus/systems and processes performed in computing apparatus/systems, the developments hereof also extend to computer programs, including computer programs on or in a carrier, adapted for putting the system into practice.