ENHANCED AUDIO LANDSCAPE

Abstract

The inventive EAL may generally enhance a driver's spatial awareness by providing the driver with intuitive/familiar audible warnings with directionality which can be enhanced with visual warnings as well as other technologies such as ANC and Zoned audio. Thus, the driver can understand the meaning of the warning and where it is coming from so he can react to it faster. In a specific embodiment, a motor vehicle includes a plurality of outside microphones mounted in association with an outer surface of the vehicle. A plurality of loudspeakers are disposed within the passenger compartment of the vehicle. A processing device identifies a sound of interest in microphone signals received from the outside microphones. It is determined from the microphone signals a direction in which the sound of interest is traveling. This may also be done with sensors, GPS, V2X, etc. In response to the identification of the sound of interest, playing at least one of the microphone signals on the loudspeakers is begun such that the sound of interest in the microphone signal and heard by the driver on the loudspeakers is perceived by a driver of the vehicle to be traveling in the determined direction.

Claims

1. A motor vehicle, comprising: a plurality of outside microphones mounted in association with an outer surface of the vehicle; a plurality of loudspeakers disposed within a passenger compartment of the vehicle; and a processing device configured to: identify a sound of interest in outside microphone signals received from the outside microphones; determine from the outside microphone signals a direction in which the sound of interest is approaching the vehicle; and in response to the identification of the sound of interest, begin playing at least one of the outside microphone signals on the loudspeakers such that the sound of interest in the outside microphone signal and heard by a driver of the vehicle on the loudspeakers is perceived by the driver to be coming from the determined direction.

2. The motor vehicle of claim 1 wherein the processing device is configured to attenuate sounds other than the sound of interest in the outside microphone signal.

3. The motor vehicle of claim 1 further comprising an inside microphone disposed within a passenger compartment of the vehicle, the processing device including an active noise controller configured to use a signal from the inside microphone to attenuate: sounds other than the sound of interest in the outside microphone signal; and/or sounds originating in the vehicle.

4. The motor vehicle of claim 1 further comprising an audio system transmitting an audio signal to the loudspeakers, the processing device being configured to, in response to the identification of the sound of interest, attenuate the audio signal being played on the loudspeakers.

5. The motor vehicle of claim 1 wherein the outside microphones are mounted in positions substantially evenly spaced around a periphery of the vehicle.

6. The motor vehicle of claim 1 wherein the processing device is configured to identify the sound of interest in the microphone signals by detecting a frequency characteristic of the sound of interest in the microphone signals.

7. The motor vehicle of claim 1 wherein the sound of interest includes a noise caused by hard braking, a sound of a siren, or a sound of a crash.

8. A method of producing sounds in a motor vehicle, the method comprising: mounting a plurality of outside microphones in association with an outer surface of the vehicle; providing a plurality of loudspeakers within a passenger compartment of the vehicle; identifying a sound of interest in outside microphone signals received from the outside microphones; determining from the outside microphone signals a direction in which the sound of interest is approaching the vehicle; and in response to the identifying of the sound of interest, commencing playing at least one of the outside microphone signals on the loudspeakers such that the sound of interest in the outside microphone signals and heard by a driver of the vehicle on the loudspeakers is perceived by the driver to be coming from the determined direction.

9. The method of claim 8 further comprising attenuating sounds other that the sound of interest in the outside microphone signal.

10. The method of claim 8 further comprising: providing an inside microphone disposed within a passenger compartment of the vehicle; and using a signal from the inside microphone to attenuate: sounds other than the sound of interest in the outside microphone signal; and/or sounds originating in the vehicle.

11. The method of claim 8 further comprising: transmitting an audio signal from an audio system to the loudspeakers; and in response to the identifying of the sound of interest, attenuating the audio signal being played on the loudspeakers.

12. The method of claim 8 wherein the outside microphones are mounted in positions substantially evenly spaced around a periphery of the vehicle.

13. The method of claim 8 wherein the sound of interest in the microphone signals is identified by detecting a frequency characteristic of the sound of interest in the microphone signals.

14. The method of claim 8 wherein the sound of interest includes a noise caused by hard braking, a sound of a siren, or a sound of a crash.

15. A motor vehicle, comprising: a plurality of outside microphones mounted in association with an outer surface of the vehicle; an inside microphone disposed within a passenger compartment of the vehicle; a plurality of loudspeakers configured to emit sounds into a passenger compartment of the vehicle; and a processing device configured to: identify a sound of interest in outside microphone signals received from the outside microphones; determine from the outside microphone signals a direction in which the sound of interest is approaching the vehicle; and in response to the identification of the sound of interest, begin playing at least one of the outside microphone signals on the loudspeakers such that the sound of interest in the outside microphone signal and heard by a driver of the vehicle on the loudspeakers is perceived by the driver to be coming from the determined direction; wherein the processing device includes an active noise controller configured to receive a signal from the inside microphone and to attenuate: sounds other than the sound of interest in the outside microphone signal; and sounds originating in the vehicle.

16. The motor vehicle of claim 15 wherein the processing device is configured to attenuate sounds other than the sound of interest in the outside microphone signal.

17. The motor vehicle of claim 15 further comprising an audio system transmitting an audio signal to the loudspeakers, the processing device being configured to, in response to the identification of the sound of interest, attenuate the audio signal being played on the loudspeakers.

18. The motor vehicle of claim 15 wherein the outside microphones are mounted in positions substantially evenly spaced around a periphery of the vehicle.

19. The motor vehicle of claim 15 wherein the processing device is configured to identify the sound of interest in the microphone signals by detecting a frequency characteristic of the sound of interest in the microphone signals.

20. The motor vehicle of claim 15 wherein the sound of interest includes a noise caused by hard braking, a sound of a siren, a sound of a vehicle horn, or a sound of a vehicular crash.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0015] A better understanding of the present invention will be had upon reference to the following description in conjunction with the accompanying drawings.

[0016] FIG. 1 is a block diagram of one embodiment of a vehicle of the present invention including an enhanced audio landscape arrangement.

[0017] FIG. 2 is a schematic diagram of types of sounds that may be attenuated and blocked over time according to one embodiment of the invention.

[0018] FIG. 3 is a flow chart of one embodiment of a method of the invention for producing sounds in a motor vehicle.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0019] FIG. 1 illustrates one embodiment of a vehicle 10 of the present invention including an enhanced audio landscape arrangement 12. Arrangement 12 includes one or more inside microphones 14 disposed inside the passenger compartment, and one or more outside microphones 16 disposed at various locations along the outside surface of vehicle 10. For example, three outside microphones 16 may be spaced 120 degrees apart around the lateral periphery of vehicle 10; or four outside microphones 16 may be spaced 90 degrees apart around the lateral periphery of vehicle 10. Arrangement 12 also includes an audio system 18, which may include a radio, compact disc player, etc.; an electronic processor 20, an active noise controller 22, and a plurality of loudspeakers 24 disposed within the passenger compartment. A V2X module 26 or connectivity to the cloud can also be included to provide information about objects and activity around the car.

[0020] During use, inside microphones 14 may pick up sounds inside the passenger compartment, such as conversations (e.g., passenger phone calls), music (e.g., from audio system 18), engine noise, etc. Outside microphones 16 may pick up sounds outside the vehicle, which may include both noise (e.g., steady-state traffic noise, other steady-state noises, wind, conversations, etc.) and sounds of interest (e.g., screeching noises from hard braking, sirens, crashing sounds, etc.).

[0021] Active noise controller 22 may sense, via inside microphones 14, noise in the passenger compartment and cancel out or attenuate the noise by sending a canceling signal to speakers 24 via processor 20 and audio system 18. Thus, only the non-noise signal from audio system 18 is played on speakers 24.

[0022] Processor 20 may monitor the outputs of outside microphones 16 for sounds of interest. Each type of sound of interest may have its own frequency characteristic and/or time domain signature, which may be identified by processor 20 in the outputs of outside microphones 16. The sounds of interest may have been simulated or pre-recorded in a laboratory in order to determine what frequency characteristics and/or time domain signatures that processor 20 should look for in the outputs of outside microphones 16. After detecting the sound of interest, or in the process of detecting the sound of interest, processor 20 may determine the direction in which the sound of interest approaches vehicle 10. More particularly, processor 20 may determine the magnitude of the sound of interest at each of outside microphones 16 individually. By comparing these magnitudes at each outside microphone 16, processor 20 may calculate the direction in which the sound of interest is approaching vehicle 10. Outside microphones 16 may be unidirectional microphones all facing in an outward direction in order to better distinguish the traveling direction of the sound of interest.

[0023] Next, processor 20 may reproduce and possibly amplify the sound of interest within the passenger compartment via speakers 24. By playing the sound of interest at different volume levels on at least two of speakers 24, processor 20 may replicate within the passenger compartment the direction of the sound of interest outside vehicle 10. Thus, the sound of interest, as heard and perceived by the driver through speakers 24, may come from the same direction as the source of the sound of interest outside vehicle 10, and may move with the sound of interest (e.g., surround sound).

[0024] Accordingly, the driver may instinctively look in the perceived direction of the sound of interest, which may match the direction in which the driver may see the source of the sound of interest outside vehicle 10. Other sounds (e.g., music) or noises being played through speakers 24 may be attenuated while the sound of interest is being played through speakers 24 so that the sound of interest can be more easily heard by the driver and can more easily catch the attention of the driver.

[0025] FIG. 2 illustrates different types of sounds that may be attenuated, blocked passed, or amplified versus time according to one embodiment of the invention. From time=0 to time=t1, exterior sounds and the interior radio may be played on speakers 24. Beginning at time=t1, a phone call by a passenger of the car may begin, and that phone call too may be played on speakers 24.

[0026] Beginning at time=t2, processor 20 may recognize a sound of interest picked up by outside microphones 16. The sound of interest may then be digitally isolated by processor 20 from the other exterior sounds, and processor 20 may begin playing and possibly amplifying the sound of interest on speakers 24. The digital isolation of the sound of interest may include using ANC 22 to produce canceling signals to cancel out the exterior sounds other than the sound of interest picked up by outside microphones 16. Other sounds, such as exterior sounds, the interior radio, and the ongoing telephone call, may be attenuated or blocked from being played on speakers 24.

[0027] Beginning at time=t3, processor 20 may recognize that the sound of interest has ceased and may stop isolating, playing and/or amplifying the sound of interest on speakers 24. Other sounds, such as exterior sounds, the interior radio, and the ongoing telephone call, may no longer be attenuated or blocked from being played on speakers 24.

[0028] The invention has been described herein as recognizing an external “sound of interest” from a potential danger, such as a siren sound, a braking sound, or a sound of a vehicle that is in the driver's blind spot, and then reproducing the sound within the vehicle. It is also possible within the scope of the invention to use other vehicle systems to identify a potential danger and, in response thereto, produce an audio signal to draw the driver's attention appropriately. In one embodiment, the vehicle may have a radar-based blind spot warning system that confirms or first discovers the presence of another vehicle in the driver's blind spot, and responds thereto by causing the sound of the other vehicle to be reproduced in the vehicle from the direction of the blind spot. Moreover, a driver monitoring system may detect the positions of the driver's head and eyes and trigger the blind spot warning system more quickly or with a lower detection threshold if the positions of the driver's head and eyes indicate that the driver is not paying full attention to the road.

[0029] In another embodiment, a vehicle attention management system may draw the driver's attention in a direction in front of the vehicle if there is sensed a distraction that might pull his attention from the road. Such a distraction may include an incoming telephone call. The driver's attention may be drawn in a direction in front of the vehicle by a tone or beep that sounds like it is coming from a direction forward of the vehicle.

[0030] In yet another embodiment, a forward-facing vehicle camera may determine when a traffic light turns green when the vehicle is stopped at a red light. A driver monitoring system may detect the positions of the driver's head and eyes and trigger an audio cue to remind the driver to accelerate if the positions of the driver's head and eyes indicates that the driver's eyes are not directed forward, or that the driver is not paying full attention to the traffic light and road.

[0031] FIG. 3 illustrates one embodiment of a method 300 of the invention for producing sounds in a motor vehicle. In a first step 302, a plurality of outside microphones are mounted in association with an outer surface of the vehicle. For example, outside microphones 16 may be mounted such that they are evenly spaced apart along an outside surface of vehicle 10 and such that each of microphones 16 is approximately at a same vertical level.

[0032] In a next step 304, a plurality of loudspeakers are provided within a passenger compartment of the vehicle. For example, loudspeakers 24 may be spaced out around the interior of vehicle 10.

[0033] Next, in step 306, a sound of interest in outside microphone signals received from the outside microphones is identified. For example, processor 20 may recognize in signals from microphones 16 a frequency characteristic or time domain signature of a sound of interest, such as screeching noises from hard braking, sirens, crashing sounds, etc.

[0034] In step 308, a direction in which the sound of interest is approaching the vehicle is determined from the outside microphone signals. For example, processor 20 may determine from the different magnitudes of the outside microphone signals, and from the different locations of outside microphones 16, a direction in which the sound of interest is approaching vehicle 10.

[0035] In a final step 310, in response to the identifying of the sound of interest, at least one of the outside microphone signals is begun to be played on the loudspeakers such that the sound of interest in the outside microphone signals and heard by a driver of the vehicle on the loudspeakers is perceived by the driver to be coming from the determined direction. For example, in response to processor 20 recognizing the outside microphone signal as a sound of interest, processor 20 may transmit respective signals to speakers 24 such that the signals may have the same content, but different amplitudes so that speakers 24 may each play the signal with a different loudness. The amplitudes of the signals sent to speakers 24 may be set such that the driver hears speakers 24 and perceives the combined sound as coming from the same direction as the actual real-life sound outside of vehicle 10 comes from.

[0036] The invention has been described herein as including two methods of capturing the sound. One method includes using microphones on the exterior of the vehicle. The other method includes using V2X or Cloud services. In another method, a vehicle equipped sensor is used such as a camera, radar, lidar, ultrasonic or any other suitable sensing device.

[0037] The invention has been described herein as including directionality but not depth. However, the scope of the invention includes using depth and directionality working together and/or independently. For example, the presence of a school zone may not have directionality but it does have changing depth.

[0038] The word “intuitive” has been used above to describe features of the invention. It is to be understood that the word “intuitive” may refer to features having a behavior associated with a particular chosen sound.

[0039] The foregoing description may refer to “motor vehicle”, “automobile”, “automotive”, or similar expressions. It is to be understood that these terms are not intended to limit the invention to any particular type of transportation vehicle. Rather, the invention may be applied to any type of transportation vehicle whether traveling by air, water, or ground, such as airplanes, boats, etc.

[0040] The foregoing detailed description is given primarily for clearness of understanding and no unnecessary limitations are to be understood therefrom for modifications can be made by those skilled in the art upon reading this disclosure and may be made without departing from the spirit of the invention.