Method and arrangement for detecting acoustic and optical information as well as a corresponding computer program and a corresponding computer-readable storage medium

Abstract

The invention relates to a method and an arrangement for detecting acoustic and optical information as well as a corresponding computer program and a corresponding computer-readable storage medium, which can in particular be used to generate three-dimensional sound maps. The sound maps can be visualized and provided with information about acoustic sources, sound power and emitter characteristics. For this purpose, it is proposed to use for the detection of acoustic and optical information at least one microphone array and at least one device for detecting optical geometry data, wherein the at least one microphone array and the at least one device for detecting optical geometry data are arranged in a defined positional relationship. Acoustic information emitted from an object and the geometry of the object are detected by moving the at least one microphone array, the at least one device for detecting optical geometry data and the object relative to each other.

Claims

1. A method for detecting acoustic and optical information, wherein at least one microphone array and at least one device for detecting optical geometry data is used, and wherein the at least one microphone array and the at least one device for detecting optical geometry data are arranged in a defined positional relationship, characterized in that acoustic information emitted from an object and the geometry of the object are detected by moving the at least one microphone array, the at least one device for detecting optical geometry data and the object relative to each other, wherein an individual sound map is generated in each case from the acoustic information captured by the at least one microphone array in at least part of the positions and a common sound map is calculated from at least a portion of the individual sound maps, wherein preferably acoustic characteristics and/or sound power levels of acoustic sources are calculated.

2. The method of claim 1, wherein the detection of acoustic and optical information is carried out with an unchanged positional relationship, or wherein the detection of the acoustic and optical information is carried out with different positional relationships.

3. The method of claim 1, wherein the at least one microphone array, the at least one device for detecting optical geometry data and/or the object are moved manually or automatically.

4. The method according to claim 1, wherein location and/or angular coordinates are acquired for at least some positions of the at least one microphone array and of the at least one device for detecting optical geometry data and preferably recorded as a trajectory.

5. The method according to claim 1, wherein a 3D-model is generated from depth images generated from detected optical geometry data, preferably computer-aided and automatically.

6. The method according to claim 4, wherein the position coordinates, the angular coordinates, the individual depth images and/or the individual sound maps are detected at discrete detection positions or continuously.

7. An arrangement for capturing acoustic and optical information, comprising: at least one data processing device, at least on microphone array, and at least one device for detecting optical geometry data, wherein the arrangement is configured to execute a method according to claim 1.

8. A non-transitory computer-readable storage medium having stored thereon a program which enables a data processing device, after the program has been loaded into memory means of the data processing device, to execute in cooperation with at least one microphone array and at least one device for detecting optical geometry data a method according to claim 1.

Description

(1) The invention will be explained below in more detail with reference to an exemplary embodiment illustrated the figures of the drawings, wherein:

(2) FIG. 1 shows a schematic diagram of the method;

(3) FIG. 2 shows an exemplary visualization of the sound field of two acoustic sources recorded with a 48-channel microphone array, and

(4) FIG. 3 shows an exemplary visualization of the sound field of two acoustic sources recorded with a 120-channel microphone array.

(5) The invention will now be explained in more detail with reference to an exemplary embodiment. According to the embodiment, an annular microphone array 102, 102, which is arranged in a defined positional relationship to one or more 3D-scanners 104, 104, is moved around an object 106. Linear, cross-shaped, spherical microphone array or arrays with randomly distributed microphones 108, 108 may be used as a microphone array in place of the annular microphone array 102, 102. Likewise, one or more video cameras can be used in place of the at least one 3D-scanner 104, 104. When using video cameras, the sound-emitting object is acquired two-dimensionally and the 3D-model of the object is, for example, generated by photogrammetric 3D-reconstruction, as described, for example, in the publication: Rodehorst, Volker: Photogrammetric 3D-reconstruction in the near-field by auto-calibration with projective geometry, Wissenschaftlicher Verlag, Berlin 2004.

(6) Likewise, the exemplary method can also be modified by moving the object with respect to a stationary arranged microphone array 102, 102 instead of moving the microphone array 102, 102 around the object 106. However, both the microphone array 102, 102 and the object 106 can also move during capture of the acoustic and optical information.

(7) The invention proposes a device and a method which enable a significantly increase in the number of spatial sampling points, without actually increasing the number of microphones 108, 108 physically present. For this purpose, microphones 108, 108 are mechanically connected in a defined relative position with one or more 3D-scanners 104, 104 with integrated position sensing. Preferably, the microphones 108, 108 and the at least one 3D-scanner 104, 104 are connected to one another by way of a fixed, but releasable mechanical connection. Furthermore, the positional relationship of the microphones 108, 108 with respect to the at least one 3D-scanner 104, 104 is preferably not changed while the acoustic and optical information is being detected. The device 110, 110 composed of microphones 108, 108 and 3D-scanner(s) 104, 104 is now moved manually or automatically around the object 106 to be measured in order to scan the sound field 112 with the microphones 108, 108 at many different locations and from many different directions. In this situation, the object 106 should not move and the acoustic sources 114 of the object 106 should be stationary at least repetitively. At the same time, the sound emitting object 106 is detected with the integrated 3D-scanners 104, 104 three-dimensionally, wherein the recorded individual depth images are computed into an overall model (stitching) during the scanning process. The position and direction of the 3D-scanner 104, 104 (position detection) and hence the microphone positions in relation to the 3D-object 106 are also recorded during the measurement (motion trajectory 116). For this purpose, the device 110, 110 is preferably associated with a coordinate system 118, 118. The local microphone coordinates (location and angle) and the local scanner coordinates (location and angle) are known in relation to the coordinate system 118, 118. The location (position) and orientation (angle) of the coordinate system 118, 118 is also captured in the detection positions, i.e. in the positions where acoustic and/or optical information is captured, and stored.

(8) For example, a camera used for the motion control of computer games (e.g. a Kinect camera from Microsoft) can be used as the 3D-scanner 106; this scanner also provides normal camera images (video) during the measurement, allowing the microphone array 102, 102 to be used also for 2D-applications without modifications.

(9) In the subsequent evaluation, the corresponding microphone coordinates are now determined for each position of the 3D-scanner 104, 104. A sound map is then computed for each position of the microphone array 102, 102 by using the known beam forming algorithms and projected onto the 3D-model of the object 106. Obscured regions (shadowing, which cannot be detected by the 3D-scanner 104, 104 from the respective position) are not computed. The individual sound maps computed from the different array positions are summed in a weighted form and averaged. The radiation characteristics of the acoustic sources 114 into the different directions can then be calculated.

(10) It will be understood that the device 110, 110 can also be used while stationary, in which case the objects 106 to be scanned move. In another embodiment, both the device 110, 110 and the measured objects 106 may be arranged statically. The scene is then captured only from a single position in three dimensions. Depth information is then supplied for each image in addition to a photo or video of the scene.

(11) The invention in its embodiment is not limited to the aforedescribed preferred exemplary embodiments. Instead, a number of variants are conceivable that make use of the inventive arrangement, the inventive method, the inventive computer program and the inventive computer-readable storage medium even in fundamentally different embodiments.

REFERENCE SYMBOLS

(12) 102 Microphone Array

(13) 102 Microphone Array

(14) 104 3D-Scanner

(15) 104 3D-scanner

(16) 106 Object

(17) 108 Microphone

(18) 108 Microphone

(19) 110 Device

(20) 110 Device

(21) 112 Sound field

(22) 114 Acoustic sources

(23) 116 Motion trajectory

(24) 118 Coordinate System

(25) 118 Coordinate system