Determination of Room Reverberation for Signal Enhancement

Abstract

A hearing prosthesis arrangement is described for a hearing assisted patient. A microphone senses an acoustic environment around the hearing assisted patient and generates a corresponding microphone output signal. An audio signal processor processes the microphone output signal and produces a corresponding prosthesis stimulation signal to the patient for audio perception. The audio signal processor includes a dereverberation process that measures a dedicated reverberation reference signal produced in the acoustic environment to determine reverberation characteristics of the acoustic environment, and reduces reverberation effects in the hearing prosthesis stimulation signal based on the reverberation characteristics.

Claims

1. A hearing prosthesis arrangement for a hearing assisted patient, the arrangement comprising: a microphone configured to sense an acoustic environment around the hearing assisted patient and generate a corresponding microphone output signal; and an audio signal processor configured to process the microphone output signal and produce a corresponding hearing prosthesis stimulation signal to the hearing assisted patient for audio perception, wherein the audio signal processor includes a dereverberation process configured to: a. measure a dedicated reverberation reference signal produced in the acoustic environment to determine reverberation characteristics of the acoustic environment, and b. reduce reverberation effects in the hearing prosthesis stimulation signal based on the reverberation characteristics.

2. The arrangement according to claim 1, further comprising: a remote control separate from the audio signal processor and configured to produce the dedicated reverberation reference signal.

3. The arrangement according to claim 2, wherein the remote control is a hand held prosthesis remote control device.

4. The arrangement according to claim 1, wherein the dereverberation process further is configured to redetermine the reverberation characteristics of the acoustic environment when the hearing assisted patient moves to a new location within the acoustic environment.

5. The arrangement according to claim 1, wherein the hearing prosthesis stimulation signal is a cochlear implant stimulation signal.

6. A method of audio signal processing in a hearing prosthesis arrangement that includes: a microphone configured to sense an acoustic environment around the hearing assisted patient and generates a corresponding microphone output signal; and an audio signal processor configured to process the microphone output signal and produce a corresponding hearing prosthesis stimulation signal to the hearing assisted patient for audio perception, wherein the method comprises: using the audio signal processor to perform a dereverberation process comprising: a. measuring a dedicated reverberation reference signal produced in the acoustic environment to determine reverberation characteristics of the acoustic environment, and b. reducing reverberation effects in the hearing prosthesis stimulation signal based on the reverberation characteristics.

7. The method according to claim 6, wherein the dedicated reverberation reference signal is produced by a remote control separated from the audio signal processor.

8. The method according to claim 6, wherein the remote control is a hand held prosthesis remote control device.

9. The method according to claim 6, wherein the dereverberation process further comprises redetermining the reverberation characteristics of the acoustic environment when the hearing assisted patient moves to a new location within the acoustic environment.

10. The method according to claim 6, wherein the hearing prosthesis stimulation signal is a cochlear implant stimulation signal.

11. A hearing prosthesis arrangement for a hearing assisted patient, the arrangement comprising: a microphone configured to sense an acoustic environment around the hearing assisted patient and generate a corresponding microphone output signal; and an audio signal processor configured to process the microphone output signal and produce a corresponding hearing prosthesis stimulation signal to the hearing assisted patient for audio perception, wherein the audio signal processor includes a dereverberation process configured to: a. obtain environment-specific dereverberation data from an external database, and b. reduce reverberation effects in the hearing prosthesis stimulation signal based on the reverberation characteristics.

12. The arrangement according to claim 11, wherein the external database is distributed across a plurality of external devices.

13. The arrangement according to claim 11, wherein obtaining environment-specific dereverberation data from an external database requires satisfying one or more access restriction requirements of the external database.

14. The arrangement according to claim 11, wherein the dereverberation process further is configured to redetermine the reverberation characteristics of the acoustic environment when the hearing assisted patient moves to a new location within the acoustic environment.

15. The arrangement according to claim 11, wherein the hearing prosthesis stimulation signal is a cochlear implant stimulation signal.

16. A method of audio signal processing in a hearing prosthesis arrangement that includes: a microphone configured to sense an acoustic environment around the hearing assisted patient and generate a corresponding microphone output signal; and an audio signal processor configured to process the microphone output signal and produce a corresponding hearing prosthesis stimulation signal to the hearing assisted patient for audio perception, wherein the method comprises: using the audio signal processor to perform a dereverberation process comprising: a. obtaining environment-specific dereverberation data from an external database, and b. reducing reverberation effects in the hearing prosthesis stimulation signal based on the reverberation characteristics.

17. The method according to claim 16, wherein the external database is distributed across a plurality of external devices.

18. The method according to claim 16, wherein obtaining environment-specific dereverberation data from an external database requires satisfying one or more access restriction requirements of the external database.

19. The method according to claim 16, wherein the dereverberation process further comprises redetermining the reverberation characteristics of the acoustic environment when the hearing assisted patient moves to a new location within the acoustic environment.

20. The method according to claim 16, wherein the hearing prosthesis stimulation signal is a cochlear implant stimulation signal.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0015] FIG. 1 shows functional blocks in a blind algorithm dereverberation process.

[0016] FIG. 2 shows functional blocks in an inverse filtering arrangement for dereverberation.

[0017] FIG. 3 shows a system for dereverberation using a reference reverberation signal according to an embodiment of the present invention.

[0018] FIG. 4 shows various functional steps in a method of signal processing to reduce reverberation based on a reference reverberation signal.

[0019] FIG. 5A-5B shows an arrangement for dereverberation using environment-specific dereverberation data from an external database according to an embodiment of the present invention.

[0020] FIG. 6 shows various functional steps in a method of signal processing to reduce reverberation based on environment-specific dereverberation data from an external database.

[0021] FIG. 7 shows detailed functional steps in a specific embodiment for reducing reverberation for a hearing prosthesis system.

DETAILED DESCRIPTION

[0022] Various embodiments of the present invention are directed to hearing prosthesis arrangements for a hearing assisted patient that circumvent the problem of estimation errors in blind dereverberation algorithms by direct measurement of a reference reverberation signal produced in the acoustic environment. In the specific context of a hearing prosthesis system, the reference reverberation signal can be produced by a user remote control and then the reflected reference reverberation signal is measured by the sensing microphone in the system audio processor device.

[0023] FIG. 3 shows a system for reduced reverberation using a reference reverberation signal, and FIG. 4 shows various functional steps in such a method of signal processing. A behind the ear (BTE) device 304 of a hearing prosthesis system (e.g., a cochlear implant system) includes a sensing microphone that senses the acoustic environment 300 around a hearing impaired patient (i.e., the room where the patient is located). The BTE device 304 also contains an audio signal processor that processes the microphone output signal as an audio input signal and produces a corresponding prosthesis stimulation signal to the patient for audio perception, for example, by transcutaneous transmission of a cochlear implant stimulation signal to implanted components of a cochlear implant hearing prosthesis system. The hearing prosthesis system also includes a user remote control 301 that provides conveniently accessible user controls for user adjustment of the system. The remote control 301 also includes an audio output transducer 302 that can generate an audio output signal into the acoustic environment 300 (e.g., loudspeaker device).

[0024] Although FIG. 3 shows a BTE device 304 and a remote control 301 for a cochlear implant, embodiments of the invention are not limited to those specific devices. In other specific applications, other mobile devices can serve as audio signal sender and receiver (preferably two separated devices), which are equipped with a loudspeaker/audio output transducer and/or a sensing microphone and audio processing software. In fact, even non-mobile fixed sound processing systems such as Public Address (PA) systems, audio or video conferencing systems, and etc. may be used.

[0025] To reduce reverberation effects in the hearing prosthesis stimulation signal, the audio signal processor in the BTE device 304 and the remote control 301 initially are time synchronized, step 401, to begin a dereverberation process. The audio output transducer 302 in the remote control 301 then produces a dedicated reverberation reference signal 305 in the acoustic environment 300, step 402, that interacts with the structures in the acoustic environment 300, for example, reflecting off the walls 303.

[0026] A separate hardware device is time synchronized with the transmission of the reverberation reference signal 305 by the remote control 301 and measures the reflected reverberation reference signal 306, step 403, that contains acoustic information regarding the acoustic environment 300, i.e., room-specific features enabling improved audio signal processing that outperforms blind algorithms. In the embodiment shown in FIG. 3, the reflected reverberation reference signal 306 is measured by the BTE device 304 with its sensing microphone, and then the hearing system's audio processor within that device is conveniently available to perform the dereverberation processing. In addition, the location of the BTE device 304 with respect to the room geometry of the acoustic environment 300 is ideal for the purpose of dereverberation in the sense that it is also where the normal audio input signal for the system is sensed. Another advantage of conducting the dereverberation process using the audio processor in the BTE device 304 is that it is always available and ready-to-use; i.e. no hardware modification of existing processors is necessary. Still, other mobile audio recording devices may be used to perform the dereverberation process as long as the distance to the user's head is taken into account. Preserving this spatial arrangement is important, since the perceptive impression of reverberation is not constant within a room, but depends on the positions of the remote control 301 as audio source and the BTE device 304 as receiver. The inventors have found that instead of using the same device for transmission and reception of the reverberation reference signal 305, the reverberation parameters determined according to the invention are as good as and in various conditions better suited.

[0027] Time synchronization, step 401, may be established by a protocol sending a message between the BTE device 304 and the remote control 301 over a communication link, such as a wireless link. For example, the remote control 301 may send a start of transmission of the reverberation reference signal 305 to the BTE device 304. The BTE device upon reception of the message starts measuring the reflected reverberation reference signal 306 in step 403. In one embodiment, time synchronization, step 401, may be established by the BTE device 304 sending a message requesting transmission of the reverberation reference signal 305 to remote control 301. This allows that the BTE device 304 does not need to listen for messages from remote control 301 all the time and can keep the communication module in a sleep mode activated only during short transmission and/or reception periods to reduce power consumption.

[0028] The dereverberation process being run by the audio processor in the BTE device 304 determines reverberation characteristics of the acoustic environment, step 404, from the measured reflected reverberation reference signal 306. The reverberation characteristics typically are combined with other relevant reverberation information such as direct-to-reverberation ratio (DRR), reverberation time (RT60), etc. to form a dereverberation model that is applied to the operating audio input signal that is affected by reverberation, step 405, to calculate an estimate of the clean speech signal with reduced reverberation effects. It would further be useful to redetermine the reverberation characteristics of the acoustic environment 300 when the hearing assisted patient moves to a new location within the acoustic environment 300.

[0029] Using physically separated and separate devices to generate and send a clean reference signal with one of the devices and measure the reverberation reference signal with the other one of the devices has the advantage that the measured microphone output signal is not overshadowed by the much larger transmitted sound signal. This is particularly significant because the dynamic range of the sensing microphone is limited, and by not recording the large transmission signal component, the measurement of the reflected reverberation reference signal can be performed with better accuracy/resolution and the room reverberation can be better determined.

[0030] A typical real-life scenario for an embodiment of the invention might be a cochlear implant user who enters a room such as classroom or a church. The user would point the remote control in the direction of the teacher or minister, and press a button that causes the remote control to send out the dedicated reverberation reference signal. Especially in the classroom scenario, it is also easily possible to hand over the remote control to the teacher such that the clean signal is sent out at the position of the target signal (i.e. the voice of the teacher), which later is to be enhanced by the dereverberation procedure. The BTE device of the cochlear implant system then measures the reflected reverberation reference signal. In some embodiments, the BTE device might also receive a clean (unreflected) version of the transmitted reverberation reference by a wireless link from the remote control. Alternatively, the clean reference signal may already be stored in the BTE and provided to the remote control for sending out as a dedicated reverberation reference signal. The dereverberation process in the BTE device then calculates an estimate of the room reverberation from the clean reference and the measured reflected reverberation reference signal.

[0031] The patient user can evaluate and adjust the system performance to optimize the operation for the user. Such adjustments can be done, for example, during a fitting procedure with an experienced audiologist, or during actual usage of the system to adjust to different real life situations. This adjustment can be very useful since the measurement of the dereverberation could further lead to influence and change the stimulation processing.

[0032] It is known that if the input audio signal contains reverberation, channel-specific dereverberation information can be used as a stimulation selection criteria instead of or in combination with the band pass energy that is typically used. The acoustic reverberation information that is determined by the arrangement also can be applied exclusively or in combination with other channel-specific information (e.g., energy, signal-to-noise ratio SNR) as an input for an N-of-M selection process; or as an input for a carrier- and envelope-triggered cochlear stimulation strategy as for example described in U.S. Pat. No. 8,019,429B2 incorporated herein by reference to manipulate the threshold (e.g., lower threshold for less reverberations) or the filter coefficients (e.g., modify the slope of the gating signal depending on the reverberation information for calculating the threshold); or as an input for modifying the channel-specific inhibition time of a pulsatile implant stimulation as for example described in U.S. Pat. No. 8,417,348 to Schleich, which is incorporated herein by reference.

[0033] The stimulation coding strategy can evaluate and use the dereverberation information when it is beneficial. In an open-field (unenclosed) environment, the reverberation information is not relevant and need not be used to enhance the quality of the coding strategy. Under those conditions, a normal coding strategy is used.

[0034] Such arrangements for dereverberation in assisted hearing applications are highly accurate with improved adaptation to actual acoustic conditions. Predefined configurations can be exploited that provide computationally low complexity dereverberation with reduced estimation errors.

[0035] Embodiments of the present invention also are directed to hearing prosthesis arrangements for a hearing assisted patient with a dereverberation process based on incorporating a priori room information obtained from environment-specific dereverberation data from an external database.

[0036] FIG. 5A-5B shows a typical such embodiment and FIG. 6 shows various functional steps in such a method of signal processing. The patient user 501 is located in a room 500 that represents a specific acoustic environment. The patient user 501 has a mobile device 502 that includes a user interface with controls for user adjustment of the entire system including the dereverberation process. The mobile device 502 includes an audio output transducer (e.g., loudspeaker device) that can send out a clean reference signal 505 into the room 500 thereby resulting in a dedicated reverberation reference signal 506. The reflected reverberation reference signal 506 reflected from the walls 503 and other surfaces of the room 500 is sensed by a sensing microphone within a behind the ear (BTE) device 504 of a hearing prosthesis system (e.g., a cochlear implant system) worn by the hearing impaired patient user 501. The BTE device 504 also contains an audio signal processor that processes the microphone output signal as an audio input signal and produces a corresponding prosthesis stimulation signal to the patient for audio perception, for example, by transcutaneous transmission of a cochlear implant stimulation signal to implanted components of a cochlear implant hearing prosthesis system.

[0037] The mobile device 502 also provides access by wireless link 507 to a remote database server 510 in for example an internet cloud 508 which controls a reverberation parameter database 509 containing dereverberation data for different rooms/different acoustic environments. The reverberation parameter database 509 may be updated and extended continuously, i.e., any new room may be added to the database at any time, room acoustics changes may be updated as needed. Furthermore, the reverberation parameter database 509 may be one single large database or be a distributed, decentralised system. In either configuration, rooms having one or more common features may be clustered/categorised, e.g., rooms belonging to the same building (for example the class rooms of a school).

[0038] The dereverberation data in the reverberation parameter database 509 may be generated in various ways; for example, by direct acoustic measurements of the room to determine room impulse response measurements; by room simulations using computer program simulation software to calculate relevant room parameters, especially for missing parameters where some room response parameters are already available, by parameter mapping and completion between rooms—when the database entry of one room is incomplete but the information is sufficient to identify similar rooms in the reverberation parameter database 509, one or more missing parameters may be obtained by transformation, interpolation, extrapolation, pattern matching, averaging or likewise mathematical operations; and/or by employing both parameters of the desired room and those of similar rooms in room simulation software—for example, if a patient user enters a new room, he/she may estimate the room dimensions and a similar room type (e.g., office, class room, hall), already listed in the reverberation parameter database 509 may provide information about room acoustics (e.g., absorption coefficient, reverberation time). The patient user may also estimate and provide other relevant parameters for de-reverberation parameter estimation, such as e.g. the distance between the audio source and the listener (i.e., himself), or further details such as the positions of the audio source and the listener relative to the room walls.

[0039] Maintenance of the reverberation parameter database 509 may be restricted to a selected group of persons (e.g., database administrators), or patient users may be allowed to modify data in the reverberation parameter database 509. For example, a user taking a given location inside a room and using the de-reverberation process on any portable electronic device may use the same program/application to modify, fine tune, update, etc. specific room parameters, and afterwards pass (upload) the altered data set to the reverberation parameter database 509 in the database server 510. Any such database update process may be managed automatically and may include a plausibility check of the new/edited data parameters.

[0040] To reduce reverberation effects in the hearing prosthesis stimulation signal, the audio signal processor in the BTE device 504 and the mobile device 502 initially are connected or paired, step 601, and the mobile device 502 connects by the wireless link 507 to the reverberation parameter database 509 controlled by the remote database server 510 in the cloud 508, step 602. The database server 510 provides the room dereverberation data for that room 500 from the reverberation parameter database 509 via the wireless link 507 back to the mobile device 502, step 603. Obtaining the room dereverberation data may require satisfying one or more access restriction requirements (e.g., privacy controls limiting data access to approved users) of the reverberation parameter database 509. This permission-controlled data acquisition can be important particularly for private buildings. The mobile device 502 provides that reverberation data to the audio processor in the BTE device 504 which uses it to calculate room reverberation parameters, step 604, and generate a room dereverberation model, block 605. Alternatively, the mobile device 508 may itself calculate the relevant parameters for the de-reverberation process and may provide these parameters to the BTE device 504 via, e.g., a wireless link. Alternatively, the BTE device 504 can directly connect to the database server 510 by e.g. wireless link 507 to access the reverberation parameter database 509. The dereverberation model can be applied directly at the database server 510 which then returns to the BTE device 504 a corresponding prosthesis stimulation signal, or the database server 510 provides the room reverberation parameters to the to the BTE device 504 which includes the dereverberation model.

[0041] The dereverberation process run by the audio processor in the BTE device 504 determines reverberation characteristics of the room 500 to calculate an estimate of the clean speech signal with reduced reverberation effects. As with the earlier arrangement in FIG. 3, it may be useful to redetermine the reverberation characteristics of the room 500 when the patient user 501 moves to a new location within the room 500.

[0042] In different specific embodiments, the signal dereverberation processing can be performed at the remote database server 510, on the mobile device 502, on a tablet, a hearing system remote control, the BTE device 504, or some other external device.

[0043] There should be some provision made to correctly select the proper room database entry in the reverberation parameter database 509. In different specific embodiments, this can be done manually by the user, or by an automatic selection procedure. Some public buildings could be equipped with sensors that interact (i.e., exchange data) with the dereverberation processing software and/or the remote database server 510 automatically as soon as a user enters the room 500.

[0044] FIG. 7 shows detailed functional steps in a specific embodiment for reducing reverberation for a hearing prosthesis system. When the system initially is exposed to a new acoustic environment, the room must first be identified, block 701, for example, based on position data from a global or local positioning system, block 702, and/or room identification data from an in-room sensor system, block 703. Once the room identification has been made, the algorithm accesses a cloud server reverberation database, block 704, typically using a wireless connection, and retrieves the data set for the room if it is in the database, block 705.

[0045] If the cloud server database does not have an existing data set for that specific room, the algorithm decides if such a room data set should be created, block 706. If not, the dereverberation process stops because no data is available, but otherwise, a new room data set can be created in the cloud server database by collecting as much room information as possible, block 707. That collected information can be used directly in further dereverberation processing, or it can be processed and used to calculate additional dereverberation parameters, block 708, and storing these locally, block 709, and/or globally in the cloud server, block 710. And again that additionally processed and calculated dereverberation parameter data can be used directly in further dereverberation processing, and/or serve to create a new room data set for storage in the cloud server, blocks 711, 712, and 713.

[0046] Whatever dereverberation data and parameters are available, are then exploited together with any available local user position data, block 714, for room-and user-specific dereverberation of the audio input signals to the hearing system. The system may then further check back with the user to determine if the dereverberation processing actually provides improved perceptive quality, block 715, and if not, the reverberation parameters and data may be refined further, block 716, until there is such an improvement in perception quality for which the system is stable and signal processing occurs using the existing data, block 717.

[0047] In specific embodiments, the data communication between the user and remote database may be realised as a simple computer program or an application for, e.g., a smart phone, a tablet, a laptop, or the remote control of the user's hearing device. Besides bi-directional data transfer of room parameters and/or audio signals, the communication between the hand-held (portable) device and the cloud server also may provide means for exchanging additional meta data. For example, one type of meta data denotes information about the actual location of the user within the room. In addition, the position of the primary sound source of the user's focus may also be useful. For example, during class in a classroom, a user pupil remains at his/her seat while the teacher who represents the desired speech source stays in front of the room around the desk and the blackboard. It can be useful to specify one or more secondary sound sources. Permanently installed or hand-held portable tracking systems (e.g., using optical tracking systems or other techniques) can be used to capture position information within a room with respect to room geometry as well as between the user and the sound source(s).

[0048] Embodiments of the invention may be implemented in part in any conventional computer programming language such as C, C++, C#, Java, VHDL, SystemC, Verilog, ASM, etc. Alternative embodiments of the invention may be implemented as pre-programmed hardware elements, other related components, or as a combination of hardware and software components.

[0049] Embodiments can be implemented in part as a computer program product for use with a computer system. Such implementation may include a series of computer instructions fixed either on a tangible medium, such as a computer readable medium (e.g., a diskette, CD-ROM, ROM, or fixed disk) or transmittable to a computer system, via a modem or other interface device, such as a communications adapter connected to a network over a medium. The medium may be either a tangible medium (e.g., optical or analog communications lines) or a medium implemented with wireless techniques (e.g., microwave, infrared or other transmission techniques). The series of computer instructions embodies all or part of the functionality previously described herein with respect to the system. Those skilled in the art should appreciate that such computer instructions can be written in a number of programming languages for use with many computer architectures or operating systems. Furthermore, such instructions may be stored in any memory device, such as semiconductor, magnetic, optical or other memory devices, and may be transmitted using any communications technology, such as optical, infrared, microwave, or other transmission technologies. It is expected that such a computer program product may be distributed as a removable medium with accompanying printed or electronic documentation (e.g., shrink wrapped software), preloaded with a computer system (e.g., on system ROM or fixed disk), or distributed from a server or electronic bulletin board over the network (e.g., the Internet or World Wide Web). Of course, some embodiments of the invention may be implemented as a combination of both software (e.g., a computer program product) and hardware. Still other embodiments of the invention are implemented as entirely hardware, or entirely software (e.g., a computer program product).

[0050] Although various exemplary embodiments of the invention have been disclosed, it should be apparent to those skilled in the art that various changes and modifications can be made which will achieve some of the advantages of the invention without departing from the true scope of the invention.