Active noise control and customized audio system

Abstract

An acoustic customization system to enhance a user's audio environment. One type of enhancement would allow a user to wear headphones and specify what ambient audio and source audio will be transmitted to the headphones. Added enhancements may include the display of an image representing the location of one or more audio sources referenced to a user, an audio source, or other location and/or the ability to select one or more of the sources and to record audio in the direction of the selected source(s). The system may take advantage of an ability to identify the location of an acoustic source or a directionally discriminating acoustic sensor, track an acoustic source, isolate acoustic signals based on location, source and/or nature of the acoustic signal, and identify an acoustic source. In addition, ultrasound may be serve as an acoustic source and communication medium. The audio customization system may be responsive to one or more inputs that enhance aspects of an audio output and one or more inputs that diminish aspects of an audio output. The system may lessen the influence of ambient audio or in some situations enhance ambient audio over source audio. The system may specify aspects of audio to be modified by specification of filtering algorithm, characterization of audio samples, monitored distortion, user selection, location specification or environmental specification.

Claims

1. A device for detecting audio and locating a source of said audio in three dimensional space comprising: a base associated with a mobile computing device; three or more microphones mounted on said base; wherein said microphones are mounted in a configuration with a first microphone mounted in a position that is not co-linear with a second microphone and a third microphone; and a fourth microphone mounted in a location that is not co-planar with said first microphone, said second microphone and said third microphone.

2. The device according to claim 1 further comprising: a beam-forming unit responsive to said microphones; and a beam steering unit responsive to said microphones.

3. The device according to claim 1 wherein said base is an outer housing of said mobile computing device.

4. The device according to claim 1 wherein said base is a protective case configured to receive a mobile computing device.

5. The device according to claim 4 further comprising an auxiliary power supply mounted in said protective case.

6. The device according to claim 4 further comprising a detachable module mating with a housing of said protective case and wherein said microphones are mounted in said detachable module.

7. The device according to claim 6 further comprising an auxiliary power supply mounted in said detachable module.

8. The device according to claim 1 wherein said fourth microphone is mounted on a boom.

9. The device according to claim 8 wherein said boom is pivot mounted on said base.

10. The device according to claim 9 wherein said boom is a telescoping boom.

11. The device according to claim 8 further comprising three or more legs pivot mounted on said base.

12. The device according to claim 11 further comprising resilient material mounted on said legs.

13. The device according to claim 12 wherein said resilient material is vibration damping.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

(1) FIG. 1 shows an embodiment in the form of an auxiliary box allowing for personal tuning of an active noise reduction system.

(2) FIG. 2 shows an embodiment implemented on a personal electronic device, particularly a tablet.

(3) FIG. 3 shows an embodiment with two noise-sensing microphones mounted on a set of headphones.

(4) FIG. 4 shows a schematic of an embodiment.

(5) FIG. 5 shows an illustration of an adaptive filter.

(6) FIG. 6 shows a non-audio based identification input.

(7) FIG. 7 shows an embodiment of an audio customization system.

(8) FIG. 8A shows an embodiment.

(9) FIG. 8B shows an embodiment.

(10) FIG. 8C shows an embodiment.

(11) FIG. 8D shows an embodiment.

(12) FIG. 8E shows an embodiment of the invention.

(13) FIG. 9A shows an embodiment of a user control interface.

(14) FIG. 9B shows an embodiment of a user control interface.

(15) FIG. 9C shows an embodiment of a user control interface.

(16) FIG. 9D shows an embodiment of a user control interface.

(17) FIG. 9E shows an embodiment of a user control interface.

(18) FIG. 9F shows an embodiment of a user control interface.

(19) FIG. 9G shows an embodiment of a user control interface.

(20) FIG. 10 shows a system layout according to an embodiment.

(21) FIG. 11 shows a system for management, acquisition and creation of audio profiles for use in customizing audio.

(22) FIG. 12 shows a schematic of an embodiment of the custom audio system using an adaptive filter as an audio customization engine.

(23) FIG. 13 shows an embodiment of an audio customization system.

(24) FIG. 14 shows a system layout according to an embodiment of the invention.

(25) FIG. 15 shows an illustration of networked embodiment of a communications system.

(26) FIG. 16A shows an example of a registration process of an embodiment of a communication system.

(27) FIG. 16B shows an example of a configuration process of an embodiment of a communication system.

(28) FIG. 16C shows an example of the operation process of an embodiment of a communication system.

(29) FIG. 17 shows an embodiment of a mutual permission customized audio source according to an embodiment of the invention.

(30) FIG. 18 shows a communications table which may be utilized in an embodiment of the invention.

(31) FIG. 19 shows an authorization table which may be used in an embodiment of the invention.

(32) FIG. 20 shows an embodiment of a mutual permission audio connection system acting in cooperation with a social networking system.

(33) FIG. 21 shows a pair of headphones with an embodiment of a microphone array.

(34) FIG. 22 shows a top view of a pair of headphones with a microphone array.

(35) FIG. 23 shows a collar-mounted microphone array.

(36) FIG. 24 illustrates a collar-mounted microphone array positioned on a user.

(37) FIG. 25 illustrates a hat-mounted microphone array.

(38) FIG. 26 shows a further embodiment of a microphone array.

(39) FIG. 27 shows a top view of a mounting substrate.

(40) FIG. 28 shows a microphone array in an audio source location and isolation system.

(41) FIG. 29 shows a front view of headphones with a multi-planar microphone array.

(42) FIG. 30 shows an embodiment of the audio source location, tracking, and isolation system.

(43) FIG. 31 shows an embodiment of the audio source location, tracking, and isolation system and particularly sensors and a location processor.

(44) FIG. 32 shows a pair of headphones with microphone arrays.

(45) FIG. 33 shows an audio source location and isolation system.

(46) FIG. 34 shows an audio source imaging system.

(47) FIG. 35 shows an adaptive audio spatialization system.

(48) FIG. 36 shows a representative shared music session.

(49) FIG. 37 shows an embodiment of a PCD during a shared music session.

(50) FIG. 38 shows a content selection system.

(51) FIG. 39 shows an embodiment of a personalized lighting display system.

(52) FIG. 40 shows a schematic of a narrowcast messaging system.

(53) FIG. 41 shows an embodiment of a permissioning subsystem.

(54) FIG. 42 shows a schematic of an embodiment of a location generation unit.

(55) FIG. 43 shows a helmet-mounted multi-directional array.

(56) FIG. 44A shows a front view of a headphone mounted array.

(57) FIG. 44B shows a jacket-mounted multi-directional array.

(58) FIG. 45 shows a smartphone with an integrated microphone.

(59) FIG. 46 shows a smartphone or smartphone case with an integrated microphone array.

(60) FIG. 47 shows a smartphone case with an integrated microphone array and an auxiliary power supply.

(61) FIG. 48 illustrates a smartphone case with a removable microphone array and battery module.

(62) FIG. 49 illustrates a smartphone or smartphone case with an integrated microphone array having pivot-mounted legs and aerial.

(63) FIG. 50 shows a smartphone or smartphone case according to FIG. 5 in a deployed configuration.

(64) FIG. 51 shows a cross-section of an interface connector.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

(65) Before the presently disclosed system is described in further detail, it is to be understood that the invention is not limited to the particular embodiments described, as such may, of course, vary. It is also to be understood that the terminology used herein is for the purpose of describing particular embodiments only, and is not intended to be limiting, since the scope of the present invention will be limited only by the appended claims.

(66) Where a range of values is provided, it is understood that each intervening value, to the tenth of the unit of the lower limit unless the context clearly dictates otherwise, between the upper and lower limit of that range and any other stated or intervening value in that stated range is encompassed within the invention. The upper and lower limits of these smaller ranges may independently be included in the smaller ranges is also encompassed within the invention, subject to any specifically excluded limit in the stated range. Where the stated range includes one or both of the limits, ranges excluding either or both of those included limits are also included in the invention.

(67) Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Although any methods and materials similar or equivalent to those described herein can also be used in the practice or testing of the present invention, a limited number of the exemplary methods and materials are described herein.

(68) It must be noted that as used herein and in the appended claims, the singular forms “a”, “an”, and “the” include plural referents unless the context clearly dictates otherwise.

(69) All publications mentioned herein are incorporated herein by reference to disclose and describe the methods and/or materials in connection with which the publications are cited. The publications discussed herein are provided solely for their disclosure prior to the filing date of the present application. Nothing herein is to be construed as an admission that the present invention is not entitled to antedate such publication by virtue of prior invention. Further, the dates of publication provided may be different from the actual publication dates, which may need to be independently confirmed.

(70) FIG. 1 shows a personally tunable custom audio system 101 which may be suitable for Adaptive Noise Cancellation. The system may be implemented in a housing 102. The housing may be portable and have a clip for attaching to a belt, garment or exercise equipment.

(71) Alternatively, the housing may be integrated with a case for a personal electronic device such as a smartphone or tablet.

(72) The system may be implemented in a personal electronic device such as a smartphone or tablet.

(73) The system may have or be connected to a noise-detecting sensor or microphone 110. The sensor may be integrated with the housing or be remote. In the case of a personal electronic device, the system may have a jack 103 for a remote noise-detecting sensor.

(74) The system may be connected to or integrated with a sound reproduction device such as one or more speakers or headphones. The connection may be by a speaker jack 104.

(75) The system may be connected to an audio source, for example, a personal media player such as an MP3 player. The connection may use jack 105.

(76) The system may be provided with an on/off switch 106 and one or more user controls 107. The controls may be for one or more channels such as a left channel tune adjustment 108 and a right channel tune adjustment 109. There may be one or more controls for frequency bands per channel. Alternatively, the controls may be for degree in balance in one or more frequency bands.

(77) FIG. 2 shows an embodiment implemented on a personal electronic device, 201, such as a tablet or smartphone. The device may have a touch screen 202 and a mechanical control 203. The device shown in FIG. 2 may be implemented in an application. FIG. 2 shows three level sliders 204, 205 and 206 for three frequency bands for the left channel and three level sliders 207, 208 and 209 for three frequency bands for the right channel. There is an on/off switch 210 that is also a touch control. The tablet 201 may have an on-board microphone 211 and a stereo headphone jack 212. Audio input may be provided by an onboard radio player or an external input.

(78) FIG. 3 shows an embodiment with a housing 301. The housing provided with an input jack 302 which may be connected to an audio source such as an MP3 player 303. The housing 301 is provided with an audio output jack 304. Headphones 305 may be connected by a cable to the jack 304. The housing may be connected to two noise-sensing microphones 307 and 308. The microphones may be hard-wired or connected with a jack.

(79) The microphones 307 and 308 may be affixed to the headphone earpieces in a manner to approximate location of the user's ears. The housing may also include a left channel control 309, a right channel control 310, and an on/off switch 311.

(80) The system may be used with or without an audio source. The system may enhance the user's listening experience by reducing the impact of external and ambient noise and sounds when used with an audio source. When used without an audio source, the system still operates to reduce the impact of external sounds and ambient noise.

(81) FIG. 4 shows a schematic of an embodiment of the custom audio system according to the system which may be an adaptive noise cancellation system.

(82) According to an embodiment of the system, audio is delivered to a user with a perceived reduction of noise. In addition the audio characteristics may be tailored according to a profile selected by a user, a profile determined by audio analysis, a profile indicated by a non-audio input, and/or a preset profile.

(83) Customized audio according to an embodiment of the system may be implemented by the use of an adaptive filter. The adaptive filter may be hardware or software implemented. A software implementation may be executed using an appropriate processor and advantageously by a digital signal processor (DSP).

(84) An adaptive filter is a filter system that has a transfer function controlled by variable parameters. According to embodiments of the system, an adaptive filter may allow improved control over the adjustment of the parameters.

(85) User controlled adjustment; audio analysis driven adjustment; and/or non-audio analysis driven adjustment may be used to customize audio input. The adjustment types can be used individually, in combination with each other and/or in combination with other types of adjustment.

(86) According to an embodiment illustrated in FIG. 4, an adaptive noise cancellation system 401 may receive a source audio signal 402 from an audio source 403 which may provide live or pre-recorded audio. Live audio may be obtained from an audio signal generator or an audio transducer, such as a microphone and analog to digital converter.

(87) The adaptive noise cancellation system may receive an ambient audio signal 404 from an ambient audio source 405.

(88) The ambient audio source may include one or more audio transducers such as a microphone(s) for detecting noise. According to one embodiment, two microphones may be used in positions corresponding to a user's ears. According to a different embodiment, a single microphone may be used. The single microphone may be in or connected to the system housing 102, associated with headphones in the form of a headset, or remotely located in a fixed or mobile position.

(89) Alternatively, the ambient audio source may be an artificial source designed to provide a signal that acts as the base of the cancellation.

(90) The active noise reduction system has a control unit 406. The control unit 406 provides parameters which define or influence the transfer function.

(91) FIG. 5 shows a more detailed illustration of the adaptive filter 505 and filter control system 506. The filter control system 506 responds to user variable input parameter control 501, audio analysis based variable control 502, and identification based variable parameter control 503.

(92) The filtration control unit 504 mixes the variable parameters to create an adaptive filter control signal 507. The adaptive filter control signal defines the transfer function used by the adaptive filter 505.

(93) User-set variable input parameter controls 501 are useful to tune the transfer function by the user to the preference of the user. The user set variable input parameter controls 501 may be established to permit the user to select a profile for the transfer function. Various profile controls can be provided to the user. For example, a profile specifically tuned to the environment inside of a passenger train. A profile specifically tuned to the environment in a jet airliner, a profile specifically tuned to the environment inside a subway train. The user adjustable controls may be a single control or multiple controls. They may correlate to conventional audio parameters such as bass, treble, frequency response. The user control parameters may be specifically engineered to modify the response of the adaptive filter according to conventional or non-conventional parameters. The user set variable input parameter controls may be controlled through switches and/or knobs on a connected interface or through a software implemented display interface such as a touchscreen. The touchscreen may be on a dedicated interface device or may be implemented in a personal electronic device such as a smart phone.

(94) Audio analysis based variable controls may be based on a computerized assessment of the ambient audio source signal. The analysis of the ambient source audio may provide input to the filtration control unit 504 to modify the adaptive filter response based on analysis of background noise and/or dominant noise. For example, the audio analysis may assess the background noise typically present on a city street and the result of that analysis is used to influence the filtration control unit 504. The audio analysis may also detect dominant noise, in this example a jackhammer being operated at a construction site, to further influence the filtration control to provide an input to the adaptive filter to compensate for the dominant noise source.

(95) The identification based variable parameter input unit 503 may provide input to the filtration control unit 504 to influence the response of the adaptive filter 505. Identification based variable parameters are further described in connection with FIG. 6.

(96) The environmental identification may be provided in the form of a local radio beacon transmitting identification based variables. The local beacon may be transmitting Bluetooth, Wi-Fi or other radio signals. The identification may also be based on location services such as those available in an iOS or Android device. The available variables are provided to the filtration control unit 504 which combines or mixes the signals to generate an adaptive filter control signal 507. The adaptive filter control signal 507 is provided to the adaptive filter 505 and defines the transformation applied to the audio source 403.

(97) FIG. 6 illustrates identification based adaptation non-audio-based variable parameter input unit 503 in order to provide an input to the filtration control unit 504. The identification based variable parameter input unit 503 obtains non-audio environmental identification signals. These non-audio environmental identification signals may serve as an index to noise profile compensation control. The noise profile compensation control may be generic or specific to a particular location. Examples of generic profiles include a passenger train, a bus, a city street, etc. Examples of specific profiles, for example, the main dining in Del Frisco's restaurant in New York City. Or inside of a 1970 Chevelle SS with a well-tuned 396 cubic inch V8 engine.

(98) FIG. 7 shows an audio customization system. The system includes an audio divider 701. The audio divider has one or more audio inputs 702. The audio inputs may be digital or analog signals. According to the preferred embodiment, analog signals may be digitized using an analog to digital converter. The analog inputs may be connected to microphones, instruments, pre-recorded audio or one or more audio source inputs like a board feed. The audio divider 701 may include one or more demultiplexers in order to separate different audio signals on the same input. The audio divider 701 also includes the capacity to divide input signals into multiple channels, for example, frequency domain channels.

(99) The audio divider 701 may be implemented in a multi-channel audio processor such as an STA311B available from ST Microelectronics. The STA311B has an automode that may divide an audio signal into eight frequency bands. Audio input signals may be divided, shaped or transferred according to controllable frequency bands or in any other manner that may be accomplished by a digital signal processor or other circuitry. The audio divider may have matrix switching capabilities to allow control of selecting which input(s) is connected to which channel output(s) 703.

(100) The audio divider 701 may be connected to an audio controller 704 which may dictate the manner in which the audio input signals 702 are handled. Alternatively, the audio divider 701 may be static and transform the audio inputs 702 to channel outputs 703 according to a predefined scheme. In addition the audio divider 701 is connected to a storage unit 705 which may contain pre-recorded audio or audio profiles. The channel outputs 703 of the audio divider 701 are connected to the inputs 706 of an audio processing unit 707. The audio processing unit 707 is responsive to audio controller 704, and contains one or more adaptive filters to combine audio input signals 706. The audio controller 704 dictates which inputs are combined and the manner of combination. The audio processing unit 707 is connected to a mixing unit 708 which combines the channel outputs 703 of the audio processing unit 707 in a manner dictated by audio controller 704. The mixing unit 708 has one or more audio outputs (709). According to one embodiment, the mixing unit 708 may have a two-channel output for connection to a headphone (not shown).

(101) Mixing may be accomplished using a digital signal processor. For example a Cirrus Logic C54700xx Audio-System-on-a-chip (ASOC) processor may be used to mix the outputs 710 of audio processing unit 707.

(102) In practical implementation a single digital signal processor may be used to perform the functions of the audio divider 701, audio processing unit 707 and mixing unit 708.

(103) FIG. 8 shows an illustration of an embodiment of the system. FIG. 8A shows an integrated input/output headset 801. The headset may include left speaker 802 and right speaker 803. Speakers 802 and 803 may advantageously be connected by a headband 804. A microphone array 805 may be carried on the headband 804 and may include multiple microphones 806. Advantageously, the microphones 806 are directional.

(104) FIG. 8B shows an alternative embodiment of an input/output unit with microphones 806 located in a neckpiece housing 807 and including earphones 808.

(105) A third embodiment is illustrated in FIG. 8C. Conventional headphones 810 may be used as an audio output device. A microphone array 809 carrying a plurality of directional microphones 806 may be attached to the headband of a headphone 810.

(106) FIG. 8D shows an interface with a housing 811 designed to be connected to a belt or other structure by clip 812. The housing 811 may include one or more microphones 806, an input jack 813, and an output jack 814. The input jack 813 may be connected to an audio source such as an mp3 player. The output jack 814 may be connected to speakers, an earphone set or a headphone set.

(107) A further embodiment shown in FIG. 8E includes a housing 815 configured for connection to a smartphone such as an iPhone or Android phone. The housing 815 may be integrated with or connected to a smartphone case. The device shown in FIG. 8E may include one or more sensor microphones 806. Advantageously, a plurality of directional microphones may be used. Alternatively, one or more omni-directional microphones may be used. The housing 815 may have a connector 816 suitable for electrically connecting the device to a smartphone. In the smartphone embodiment shown in FIG. 8E, the smartphone or other portable electronic device (not shown) may include application software operating as a user control. The signal processing capability may be incorporated into the smartphone or be performed by a separate processor located in the housing.

(108) In each of the embodiments 8A, 8B, 8C, 8D, and 8E, user controls may be provided for in a connected input/output device such as a smartphone or by controls mounted on any of housings 805, 807, 809, 811 or 815. In addition, an audio divider 702 and mixing unit 708 may be provided for either within the microphone housings or control unit. In addition, connections between the input/output devices, audio inputs, audio processing unit, and mixing unit may be by wired or wireless connections. The same holds true for the controller and audio divider and/or storage if utilized.

(109) FIG. 9A-G shows alternative aspects of a user control interface for use and connection with the audio optimization system according to the system.

(110) FIG. 9A shows a user control interface useful to control noise cancellation according to direction of noise source.

(111) FIG. 9B shows a user control interface suitable for controlling direction and distance of audio subject to noise cancellation.

(112) FIG. 9C shows a user control interface to facilitate a user capturing audio to serve as a model for enhancement or cancellation. The interface of FIG. 9B to record a sample audio that is to be exempted from cancellation, enhanced or specifically subject to cancellation. For example a particular ringtone or alarm may be recorded and stored to serve as a profile to permit the same or similar audio to be transferred to the audio output.

(113) The user control interface may also include controls for channels, volume, bass, treble, midrange, other frequency ranges, selection of cancellation algorithm or profile, selection of enhancement algorithm or profile, feature on/off switches, etc.

(114) FIG. 9D shows a user control interface including a display of a representation of an ambient sound and sliders to change or customize audible parameters in an audio library.

(115) FIG. 9E shows a user control interface designed for microphone selection.

(116) FIG. 9F shows a user control interface including a display allowing selection of distance from ambient sound source and/or microphone array.

(117) FIG. 9G shows a user control interface including a display corresponding to a noise cancellation algorithm and user input controls.

(118) FIG. 10 shows a system layout according to an embodiment of the system. An adaptive noise controller 1001 is provided. The adaptive noise controller 1001 may be connected to a reference microphone array 1002 and to a set of digital filters 1003. The reference microphone array 1002 may also be connected to the digital filters 1003. The digital filters 1003 may rely on ambient sound profiles stored in an ambient sound library 1004 also connected to the adaptive noise controller 1001. A source signal 1005 may be connected to digital filters 1006 which in turn are connected to ambient sound library 1004 and adaptive noise controller 1001. Output devices such as earphone/headphone 1007 may be connected to the adaptive noise controller 1001 and may be connected to a speaker driver 1008. One or more error microphones 1009 may be connected to the adaptive noise controller 1001 and/or the headphone/earphone array 1007.

(119) An embodiment of the system may operate to allow a user to select audio received in a headphone. The system may include a programmable audio processor which transmits audio selected by a user to an audio transducer, such as a headphone. The selection of audio can be by audio source and can be particular aspects or portions of an audio signal. It is a recognized problem that when audio is being played through headphones a user can become isolated from his audio environment. Noise canceling headphones designed to increase the perceived quality of audio to a user increase the level of isolation. The embodiment of the system may be designed to allow a user to selectively decrease audio isolation from the user's environment.

(120) The system may include audio profiles that are selected to control customization of audio provided to a user. FIG. 11 shows a system for management, acquisition and creation of audio profiles for use in customizing audio.

(121) The system may include an audio customization engine 1101. One or more audio sources 1102 may be connected to the audio customization engine 1101. The audio sources advantageously include local audio sensor(s) such as one or more microphones or microphone arrays. The system may have microphones to detect local audio which may be used by the audio customization engine 1101 for active noise control.

(122) One or more active profiles 1103 may be used by the audio customization engine 1101 to customize audio signals provided to an audio output device 1104, for example, headphones.

(123) A user control interface 1105 operates with a profile manager 1106 to designate a set of active profiles. The profile manager 1106 can assemble audio profiles to be in active profiles 1103. The active profiles 1103 may be from one or more sources. The active profiles 1103 may include one or more default profile such as car horns or police sirens.

(124) The system may have a user profile storage cache 1107 containing profiles obtained or generated by a user. Selected audio profiles may be from user profile storage cache 1107, may be transferred or copied to the active profiles 1103 for use by the audio customization engine. Another potential source of audio profiles is library 1108. The library 1108 may contain audio profiles indexed by a directory to allow a user to select an audio profile from a remote source. The library 1108 may contain profiles for individuals, environments, specified sounds or other audio components.

(125) Audio profiles may also be stored in the contacts for a user or organization. The profile manager 1106 may access a contacts application to obtain audio profiles contained in a contacts application.

(126) A profile generator 1110 may be present and connected to profile manager 106. The profile generator 1110 may sample audio from a microphone 1111 and process the sampled audio to generate an audio profile. The generated profile may be placed directly in the active profiles 1103, added to a contact 1109 or stored in user profile storage cache 1107 or library 1108. The audio profiles may be associated with appropriate metadata to facilitate location, identification and use.

(127) An invitation system 1112 may be connected to the profile manager 1106 in order to invite another user or system to provide an audio profile or sample audio to generate a profile. The user control interface 1105 may control operation of the profile manager 1106 and audio customization engine 1101.

(128) The system described herein may be implemented in a personal electronic device such as a smartphone or tablet. The system may be implemented and computation allocated between server and client devices depending on computational, communications, and power resources available.

(129) The system may have or be connected to one or more microphones or microphone arrays, integrated with the housing of a user device or be remote. In the case of a personal electronic device, the system may have a jack to connect an audio sensor. The system may be connected to or integrated with a sound reproduction device such as one or more speakers or headphones. The connection may be by a speaker jack 1104. The system may be connected to an audio source, for example, a personal media player such as an MP3 player. The connection may use jack 105.

(130) The system may be provided with an on/off switch and one or more user controls. The controls may be for one or more channels such as a left channel tune adjustment and a right channel tune adjustment. There may be one or more controls for frequency bands per channel. Alternatively, the controls may be for degree in balance in one or more frequency bands. The user controls may be applied to control operations on a server or local operation on a user device.

(131) FIG. 12 shows a schematic of an embodiment of the custom audio system using an adaptive filter 1201 as an audio customization engine.

(132) The adaptive filter 1201 may act on one or more audio input signals 1202, 1204 to condition the audio information for delivery of a modified or customized audio signal to a user. The audio characteristics may be tailored according to a profile selected by a user, a profile determined by audio analysis, a profile indicated by a non-audio input, and/or a preset profile. The adaptive filter may be hardware or software implemented. A software implementation may be executed using an appropriate processor and advantageously by a digital signal processor (DSP). An adaptive filter is a filter system that has a transfer function controlled by variable parameters. An adaptive filter may allow improved control over the adjustment of the parameters.

(133) One or more sources 1203, 1205 may be connected to adaptive filter 1201 to provide audio signals 1202, 1204. Audio source 1203 may be local or remote. Audio source 1205 may provide local ambient audio information from one or more audio transducers such as microphones or microphone arrays. Other audio sources may be from remote or specialized audio transducers, mp3 or other audio players, or audio streams, or any other audio source.

(134) The adaptive filter 1201 may be connected and responsive to a control unit 206. The control unit 1206 may provide parameters which define or influence the transfer function executed by the adaptive filter 1201.

(135) FIG. 13 shows an embodiment of an audio customization system 1306 showing profile manager 1304. The profile manager 1304 may be associated with profiles 1301, 1302, 1303.

(136) The profiles 1301, 1302, and 1303 may be mixed and used to control the adaptive filter to create an adaptive filter control signal 1307. The profile manager 1304 may perform this function. The adaptive filter control signal 1307 defines the transfer function used by the adaptive filter 1305. For illustration, FIG. 13 shows an audio source(s) 1308 which is representative of one or more audio inputs, including, but not limited to, local microphone(s)/microphone array(s); local audio player; cloud-based audio player; and/or network connected devices etc. The system is not limited by the source(s) or type of source(s). The adaptive filter 1305 applies the transfer function defined by the profile manager 1304 to the audio sources 1308 and outputs to an audio output 1309. The mixing function may also be performed in the adaptive filter itself, depending on implementation choices.

(137) FIG. 14 shows a system layout. An adaptive audio controller 1401 may be provided. The adaptive audio controller 1401 may be connected to an audio source(s) 1402 which may be one or more microphones or other audio sources including an ambient microphone array. The adaptive audio controller may also be connected to a set of active audio profiles 1403. The active audio profiles 1403 may be selected from profiles stored in the sound library 1404. The sound library 1404 may contain audio profiles created by sampling audio information detected by the ambient microphone. If a user wants to establish a profile for certain characteristic audio, the audio may be sampled and characterized in order to create a profile. The sample audio may be used to create an audio profile such as a specific voice, machinery, or other noise. Profiles for a noise, such as a jackhammer or a person the user does not want to hear may be created, as well as profiles to a noise or person the user especially want to hear may be created by isolating and analyzing the specified audio to characterize the audio and establish a profile that can be used by the adaptive audio controller 1401, to either enhance or attenuate audio corresponding to the characteristics of the sample.

(138) The adaptive audio controller 1401 may be implemented in a multi-channel audio processor, a digital signal processor, for example an Audio-System-On-A-Chip (ASOC) processor. The audio processor may have an auto mode that may divide an audio signal into eight frequency bands. Audio input signals may be divided, shaped or transferred according to controllable frequency bands or in any other manner that may be accomplished by a digital signal processor or other circuitry.

(139) The audio divider may be connected to an audio controller implemented by the DSP which may dictate the manner in which the divided audio input signals are handled. The processed audio channels may then be mixed down to a mono or stereo output. The stereo or two-channel output may connect to a headphone.

(140) Output device 1407 may be connected to the adaptive audio controller 1401. The audio source(s) 1402 may also include one or more error microphones 1405 for noise detection and cancellation purposes.

(141) The customization may be used and managed in a networked system. FIG. 15 illustrates an embodiment of a networked communications system for establishing and providing preferred audio. According to an embodiment of the system, a social networking system may be established where members of the network may authorize and/or request access to enhanced communication with others in the network. The communications may occur over a network or may occur in a non-networked fashion, i.e., people talking within “earshot” of each other. One system implementation is shown in FIG. 15. The system is managed by a control processor 1501. A subscriber interface 502 may be utilized by the subscriber's or network members. The subscribers may establish a transformation to be used for their own accessible audio. Subscribers may create their own audio profiles. Subscribers may authorize others to include the subscribers in transformations. A network connection 1503 is illustrated, however, processing and communications resources may suggest whether indicated processes are performed centrally on servers or distributed to user devices.

(142) An audio acquisition system 1504 may be connected to the control processor 1501. The audio acquisition system is used to sample audio. The subscriber interface may include a microphone and a subscriber advantageously will record voice samples which will be processed through the audio acquisition system 1504 and provided to the profile generation system 1505. The profile generation system is utilized to characterize the nature of the acquired audio in order to establish a generalized filter useful for distinguishing audio content having the same characteristics for use in specifying a transformation. Certain audio signals may exhibit characteristic properties which facilitate establishment of a profile for use in transformation. For example, a telephone dial tone may have a particular narrow frequency which could be measured and profiled. The profile would be used in the transformation in order to filter out that particular frequency. Other audio sources are more complex but may still be characterized for filter generation. Complex audio sources such as individual voices will typically require substantial processing, and as such, centralized server processing may be appropriate. Profiles generated by the profile generation system may be stored in a profile library 1506. The subscriber interface 1502 may be utilized to identify and select profiles contained in the profile library for incorporation in a subscriber transformation. Advantageously a profile library may include subscriber profiles and generic profiles which may be useful such as police siren profiles, car horn profiles, alarm profiles, etc.

(143) FIGS. 16A, 16B, and 16C illustrate operations of an embodiment of the communications system. FIG. 16A illustrates the registration process for the system. Registration is initiated by acquisition operations 1601. The acquisition operations acquire information for use in the system for each subscriber. The acquisition process includes acquiring subscriber identification and registering credentials. The acquisition process also involves setting permissions. Setting permissions as a process to establish which subscribers may have access to subscriber profiles. The acquisition process 1601 also includes acquiring audio samples from the subscriber. Process 1602 serves to generate an audio profile on the basis of audio acquired in process 1601. Process 1603 generates a subscriber record which includes or links subscriber identifications, subscriber permissions and subscriber audio profiles. Process 1604 operates to store the subscriber record in a library for use by the subscribers and those authorized by the subscriber. FIG. 16B illustrates the configuration operation for subscribers. Configuration is initiated when a subscriber connects and submits acceptable credentials for identification and establishing authorization to access the system. The credentials are submitted and verified at process 1605. Process 1606 illustrates operations to manage profiles. A subscriber, once connected to the configuration system, can manage the profiles which are utilized to generate the subscriber audio transformation. The manage profile operation 1606 includes search; request authorization; add profiles; and delete profiles. The search function is a mechanism for a subscriber to search for other subscribers and available profiles. The request authorization function may be initiated on the basis of the results of a subscribers search, or on the basis of input on a subscriber identification. The request authorization function initiates an authorization request to another subscriber for access to the other subscriber's audio profile. Once a subscriber has access to the audio profile of another subscriber, the first subscriber may use that audio profile in a transformation to enhance or attenuate audio information having matching characteristics.

(144) The request authorization operation initiates an authorization request to another subscriber. Once that subscriber receives the request, it may be accepted, rejected, or ignored. According to an embodiment, once the request is accepted, the subscriber record of the accepting subscriber is updated to reflect permission granted to the request of the subscriber for use of the audio profile.

(145) The managed profile operation also includes an add profile function whereby a subscriber can select profiles to be activated for that subscriber. Profiles including permissions which are added by a subscriber are then included in the active profiles and utilized to generate a transformation that will be applied to audio information received by that subscriber.

(146) The manage profiles operation 1606 also includes a delete profiles function. The delete profiles function serves to deactivate and remove a particular profile from the subscriber's active profiles. The update active lists function 1607 operates to modify the subscriber's active audio profiles in accordance with the add profiles function and delete profiles function of the manage profiles operation 1606.

(147) FIG. 16C illustrates the operations function of the communications system. Operations are initiated by acquisition of the subscriber's active profiles 1608. Once the active profiles are acquired for a session, the system carries out a configure transformation operation 1609. The configure transformation operation 1609 combines the active profiles into a transformation which may be used by the adaptive audio profiler 1401, the adaptive filter 1305, or the audio customization engine 1101. The system includes a sample audio operation 610 which advantageously utilizes one or more microphones to “listen” to the ambient environment and may include local or networked audio signals combined with the ambient signals.

(148) One or more of the audio signals are provided to an audio processor which provides the audio transformation 1611 which is created by the configure transformation operation 1609. The transformed audio may be provided to a transducer such as a speaker, and preferably headphones.

(149) The techniques, processes and apparatus described may be utilized to control operation of any device and conserve use of resources based on conditions detected or applicable to the device.

(150) Headphones are a pair of small speakers that are designed to be held in place close to a user's ears. They may be electroacoustic transducers which convert an electrical signal to a corresponding sound in the user's ear. Headphones are designed to allow a single user to listen to an audio source privately, in contrast to a loudspeaker which emits sound into the open air, allowing anyone nearby to listen. Earbuds or earphones are in-ear versions of headphones.

(151) The system may be controlled so that a particular communication station will be in audio communication with one or more other communications stations 1701. The control station 1702 may require permissions from one or more of the communications stations 1701 to establish and maintain audio communications. The permissions may be designated at a control station 1702. Advantageously the control stations 1702 may be client applications running on a desktop or other computing platform. A user may log into a control station 1702 in order to manage and control audio communications to stations which the user is authorized to manage.

(152) The control station may be connected by a network 1705 such as the internet to a connection manager 1706. The connection manager 1706 may contain logic facilitating the identification of audio sources that each communications station has requested. The audio sources may be other subscriber stations which must be set up by their users to authorize communications. In addition the audio sources may include static audio sources such as radio stations or other broadcast facilities and signaling stations to provide information of a more general interest. Examples of signaling stations may include weather alerts, AMBER alerts, or school closing notifications. A control station 1702 may be utilized to program the connection manager 1706 to designate the sources that the communications station 1701 is requesting.

(153) Each individual computing device may have a physical or logical identification. The physical or logical identifications may be IP addresses, MAC addresses, telephone numbers, user numbers or any other identification token. When the communication manager 1706 has received sufficient permissions to authorize a communication connection, the connection manager informs the connection matrix 1707 of the enabled connection. The connection matrix 1707 is connected to and controls a matrix switching system 1708 which establishes authorized connections between communications stations 1701.

(154) It may be desirable to control the nature of or aspects of audio information which is communicated between communications stations 1701. FIG. 17 illustrates an audio suppression system 1709 between the communications station 1701 and the matrix switching system 1708. The audio suppression system 1709 may advantageously be controlled according to instructions from a control station 1702 provided to a communications manager 1706. The communications manager 1706 may provide control instructions to the audio suppression system 1709.

(155) Audio suppression system 1709 may be in place to attenuate background noise or other portions of the audio information being communicated. Depending on the application, the audio suppression may be applied to inbound communications to a communications station 1701 or outbound communications from a communication system 1701.

(156) The control station 1702 may be used to populate a communications table 1710 as shown in FIG. 18. The communications table 1810 may have a set of records 1834 that include a requesting station ID field 1831, a requested station ID field 1832 and a mutual authorization flag field 1811. FIG. 19 shows an authorization table 1912 with records containing a transmitting station ID field 1933 and transmit authorization flag 1935. The control station 1702 may provide an identification of a station and an identification of each station that the station wishes to include in its communications group. The communications table 1810 may also include a flag field 1811 to signal a mutual authorization to establish communications. The mutual authorization field 1811 is activated when a station initiates a communication request to a second station which has previously been authorized by the second station. An authorization table 1912 may include records identifying communications stations that do not require explicit authorization to establish communications. For example, a radio station could be set up so that it does not require authorization, for example, a subscription-based station. A radio station may also be set up so that it does require authorization. The radio station's subscription management system 1913 would be responsible for communicating authorized identifications to the communications manager 1706.

(157) An entry may be created in a communications table 1810 when an authorized request is made for a first communications station to be in communication with a second communications station. The entry 1834 will include the ID of the first station as the requesting station ID 1831 and the ID of the second station as a requested station ID 1832. If an authorized request for the second station to be in communication with the first station had not been previously made an entry is created in the communications table 1810, an invitation may be transmitted to the second station to establish communication. If that invitation is accepted, a second entry may be created in the communications table 1810 indicating the ID of the second station seeking authorization to establish communications. A process may be used to determine when complementary entries exist in the communications table 110, and if so, set the authorization flags 1811 to authorize communications and having an authorized field set.

(158) If a station requests communication authorization with a second station which had previously authorized communication, a record may be entered in the communications table 1810 indicating the communication pair and setting the authorization flag 1811. The communications manager 1706 identifies all communication pairs which have been mutually authorized either by specific action or by default and places an entry in the connection matrix 1707. The connection matrix 107 controls the matrix switching system 1708 to establish a communication channel between the stations of the communication pair.

(159) According to an advantageous feature, an address book may be provided in or in connection with each station. The address book may be a personal look-up table to identify a correlation between a user-identifiable information, like a name, and a logical identification like a station identification number.

(160) In this fashion, a system can be established where a group of friends request communications. Each friend can listen in on audio originating from a paired communications station. The friends may modify the authorizations on an ad hoc basis.

(161) According to an advantageous feature, each station may include a communication activation control. In this fashion, the user of each station may control whether the station broadcasts, receives broadcasts, broadcasts and receives or does not broadcast and does not receive. The control interface may be an application.

(162) FIG. 20 shows an embodiment of a mutual permission audio connection system acting in cooperation with a social networking system. In the embodiment of FIG. 20 a mutual permission audio communication system is shown working in connection with a social network platform. An example of an established social network platform is the Facebook platform. The Facebook platform facilitates add-on systems which may take advantage of the Facebook functionality for certain operations such as registration and log-on. In the embodiment illustrated in FIG. 20 an established social network platform 2001 may be controlled or operated through a user interface 2002. The user interface may include an audio communication control station user interface 2003, along with the intrinsic social network user interface 2004.

(163) The operation of the communication system may be controlled through an audio communication subsystem 2005 which may be associated with the established social network platform 2001 or may be independent, connected through a communications network 2008. In either case the audio communication control station user interface 2003 may be separate from the social network user interface 2004, freestanding and connected through communications network 2008. Communication stations 1701, previously described, may be connected through communications network 2008. A connection matrix 1707 and matrix switching system 1708 along with audio suppression system 1709, all previously described, may also be connected to the communications stations and audio communication subsystem through a communications network 2008. The established social network platform 2001 may be connected to an intrinsic permissioning system 2006. The connection manager 2007, having the functionality previously described for connection manager 1706, may be incorporated in the permissioning system 2006 of the established social network platform 2001, or connected to connection matrix 1707.

(164) FIG. 21 and FIG. 22 show a pair of headphones with an embodiment of a microphone array. FIG. 22 shows a top view of a pair of headphones with a microphone array.

(165) The headphones 2101 may include a headband 2102. The headband 2102 may form an arc which, when in use, sits over the user's head. The headphones 2101 may also include ear speakers 2103 and 2104 connected to the headband 2102. The ear speakers 2103 and 2104 are colloquially referred to as “cans.” A plurality of microphones 2105 may be mounted on the headband 2102. There may be three or more microphones where at least one of the microphones is not positioned co-linearly with the other two microphones in order to identify azimuth.

(166) The microphones in the microphone array may be mounted such that they are not obstructed by the structure of the headphones or the user's body. Advantageously the microphone array is configured to have a 360-degree field. An obstruction exists when a point in the space around the array is not within the field of sensitivity of at least two microphones in the array. An accelerometer 2106 may be mounted in an ear speaker housing 2103.

(167) FIG. 23 and FIG. 24 show a collar-mounted microphone array 2301.

(168) FIG. 24 illustrates the collar-mounted microphone array 2301 positioned on a user. A collar-band 2302 adapted to be worn by a user is shown. The collar-band 2302 is a mounting substrate for a plurality of microphones 2303. The microphones 2303 may be circumferentially-distributed on the collar-band 2302, and may have a geometric configuration which may permit the array to have a 360-degree range with no obstructions caused by the collar-band 2302 or the user. The collar-band 2302 may also include an accelerometer 2304 rigidly-mounted on or in the collar band 2302.

(169) FIG. 25 illustrates a hat-mounted microphone array. FIG. 25 illustrates a hat 2501. The hat 2501 serves as the mounting substrate for a plurality of microphones 2502. The microphones 2502 may be circumferentially-distributed around the hat or on the top of the hat in a fashion that avoids the hat or any body parts from being a significant obstruction to the view of the array. The hat 2501 may also carry on accelerometer 2504. The accelerometer 2504 may be mounted on a visor 2503 of the hat 2501. The hat mounted array in FIG. 25 is suitable for a 360-degree view (azimuth), but not necessarily elevation.

(170) FIG. 26 shows a further embodiment of a microphone array. A substrate is adapted to be mounted on a headband of a set of headphones. The substrate may include three or more microphones 2702. A substrate 2603 may be adapted to be mounted on headphone headband 2102. The substrate 2603 may be connected to the headband 2102 by mounting legs 2604 and 2605. The mounting legs 2604 and 2605 may be resilient in order to absorb vibration induced by the ear speakers and isolate microphones and an accelerometer in the array.

(171) FIG. 27 shows a top view of a mounting substrate 2603. Microphones 2702 are mounted on the substrate 2603. Advantageously an accelerometer 2701 is also mounted on the substrate 2603. The microphones alternatively may be mounted around the rim 2604 of the substrate 2603. According to an embodiment, there may be three microphones 2702 mounted on the substrate 2603 where a first microphones is not co-linear with a second and third microphone. Line 2705 runs through microphone 2702B and 2702C. As illustrated in FIG. 27, the location of microphone 2702A is not co-linear with the locations of microphones 2702B and 2702C as it does not fall on the line defined by the location of microphones 2702B and 2702C. Microphones 2702A, 2702B and 2702C define a plane. A microphone array of two omni-directional microphones 2702B and 2702C cannot distinguish between locations 2706 and 2707. The addition of a third microphone 2702A may be utilized to differentiate between points equidistant from line 2705 that fall on a line perpendicular to line 2705.

(172) According an advantageous feature, a motion detector such as Gyroscope, and/or a compass may be provided in connection with a microphone array. Because the microphone array is configured to be carried by a person, and because people move, a motion detector may be used to ascertain change in position and/or orientation of the microphone array. It is advantageous that the motion sensor, for example accelerometer, be in a fixed position relative to the microphones 502 in the array, but need not be directly mounted on a microphone array substrate. An accelerometer 304 may be mounted on the collar-band 2302 as illustrated in FIG. 24. An accelerometer may be mounted in a fixed position on the hat 2501 illustrated in FIG. 25, for example, on a visor 2503. The accelerometer may be mounted in any position. The position 2504 of the accelerometer is not critical.

(173) FIG. 28 shows a microphone array 2801 in an audio source location and isolation system. A beam-forming unit 2803 is responsive to a microphone array 2801. The beamforming unit 2803 may process the signals from two or more microphones in the microphone array 2801 to determine the location of an audio source, preferably the location of the audio source relative to the microphone array. A location processor 2804 may receive location information from the beam-forming system 2803. The location information may be provided to a beam-steering unit 2805 to process the signals obtained from two or more microphones in the microphone array 2801 to isolate audio emanating from the identified location. A two-dimensional array is generally suitable for identifying an azimuth direction of the source. An accelerometer 2806 may be mechanically coupled to the microphone array 2801. The accelerometer 2806 may provide information indicative of a change in location or orientation of the microphone array. This information may be provided to the location processor 2804 and utilized to narrow a location search by eliminating change in the array position and orientation from any adjustment of beam-forming and beam-scanning direction due to change in location of the audio source. The use of an accelerometer to ascertain change in position and/or change in orientation of the microphone array 2801 may reduce the computational resources required for beam forming and beam scanning.

(174) FIG. 29 shows a front view of a headphone fitted with a microphone array suitable for sensing audio information to locate an audio object in three-dimensional space.

(175) An azimuthal microphone array 2603 may be mounted on headphones. An additional microphone array 2906 may be mounted on ear speaker 2103. Microphone array 2906 may include one or more microphones 2702 and may be acoustically and/or vibrationally isolated by a damping mount from the earphone housing. According to an embodiment, there may be more than one microphone 2702. The microphones may be dispersed in the same configuration illustrated in FIG. 27.

(176) A microphone array 2907 may be mounted on ear speaker 2104. Microphone array 2907 may have the same configuration as microphone array 2906.

(177) Microphones may be embedded in the ear speaker housing and the ear speaker housing may also include noise and vibration damping insulation to isolate or insulate the microphone arrays 2906 and 2907 from the acoustic transducer in the ear speakers 2103 and 2104.

(178) Three non-co-linear microphones in an array may define a plane. A microphone array that defines a plane may be utilized for source detection according to azimuth, but not according to elevation. At least one additional microphone 108 may be provided in order to permit source location in three-dimensional space. The microphone 108 and two other microphones define a second plane that intersects the first plane. The spatial relationship between the microphones defining the two planes is a factor, along with sensitivity, processing accuracy, and distance between the microphones that contributes to the ability to identify an audio source in a three-dimensional space.

(179) In a physical embodiment mounted on headphones, a configuration with microphones on both ear speaker housings reduces interference with location finding caused by the structure of the headphones and the user. Accuracy may be enhanced by providing a plurality of microphones on or in connection with each ear speaker.

(180) FIG. 30 shows an audio source location tracking and isolation system. The system includes a sensor array 3001. Sensor array 3001 may be stationary. According to a particularly useful embodiment the sensor array 3001 may be body-mounted or adapted for mobility. The sensor array 3001 may include a microphone array. The microphone array may have two or more microphones. The sensor array may have three microphones in order to be capable of a 360-degree azimuth range. The sensor array may have four or more microphones in order to have a 360-degree azimuth and an elevation range. The 360-degree azimuth requires that the three microphones be non-co-linear and the elevation-capable array must have at least three non-co-linear microphones defining a first plane and at least three non-co-linear microphones defining a second plane intersecting the first plane provided that two of the three microphones defining the second plane may be two of the three microphones also defining the first plane.

(181) In the event that the sensor array 3001 is adapted to be portable or mobile, it is advantageous to also include a motion sensor rigidly-linked to the sensor array.

(182) A wide source locating unit 3002 may be responsive to the sensor array. The wide source locating unit 3002 is able to detect audio sources and their general vicinities. Advantageously the wide source locating unit 3002 has a full range of search. The wide source locating unit may be configured to generally identify the direction and/or location of an audio source and record the general location in a location table 3003. The system is also provided with a narrow source locating unit 3004 also connected to sensor array 3001. The narrow source locating unit 3004 operates on the basis of locations previously stored in the location table 3003. The narrow source locating unit 3004 will ascertain a pinpoint location of an audio source in the general vicinity identified by the entries in a location table 3003. The pinpoint location may be based on narrow source locations previously stored in the location table or wide source locations previously stored in the location table. The narrow source location identified by the narrow source locating unit 3004 may be stored in the location table 3003 and replaced the prior entry that formed a basis for the narrow source locating unit scan. The system may also be provided with a beam steering audio capture unit 3005. The beam steering audio capture unit 3005 responds to the pinpoint location stored in the location table 3003. The beam steering audio capture unit 3005 may be connected to the sensor array 3001 and captures audio from the pinpoint locations set forth in the location table 3003.

(183) The location table may be updated on the basis of new pinpoint locations identified by the narrow source locating unit 3004 and on the basis of an array displacement compensation unit 3006 and/or a source movement prediction unit 3007. The array displacement compensation unit 3006 may be responsive to the accelerometer rigidly attached to the sensor array 3001. The array displacement compensation unit 3006 ascertains the change in position and orientation of the sensor array to identify a location compensation parameter. The location compensation parameter may be provided to the location table 3003 to update the pinpoint location of the audio sources relative to the new position of the sensor array.

(184) Source movement prediction unit 3007 may also be provided to calculate a location compensation for pinpoint locations stored in the location table. The source movement prediction unit 3007 can track the interval changes in the pinpoint location of the audio sources identified and tracked by the narrow source locating unit 3004 as stored in the location table 3003. The source movement prediction unit 3007 may identify a trajectory over time and predict the source location at any given time. The source movement prediction unit 3007 may operate to update the pinpoint locations in the location table 3003.

(185) The audio information captured from the pinpoint location by the beam steering audio capture unit 3005 may be analyzed in accordance with an instruction stored in the location table 3003. Upon establishment of a pinpoint location stored in the location table 3003, it may be advantageous to identify the analysis level as gross characterization. The gross characterization unit 3008 operates to assess the audio sample captured from the pinpoint location using a first set of analysis routines. The first set of analysis routines may be computationally non-intensive routines such as analysis for repetition and frequency band. The analysis may be voice detection, cadence, frequencies, or a beacon. The audio analysis routines will query the gross rules 3009. The gross rules may indicate that the audio satisfying the rules is known and should be included in an audio output, known and should be excluded from an audio output or unknown. If the gross rules indicate that the audio is of a known type that should be included in an audio output, the location table is updated and the instruction set to output audio coming from that pinpoint location. If the gross rules indicate that the audio is known and should not be included, the location table may be updated either by deleting the location so as to avoid further pinpoint scans or simply marking the location entry to be ignored for further pinpoint scans.

(186) If the result of the analysis by the gross characterization unit 3008 and the application of rules 3009 is of unknown audio type, then the location table 3003 may be updated with an instruction for multi-channel characterization. Audio captured from a location where the location table 3003 instruction is for multi-channel analysis, audio may be passed to the multi-channel/multi-domain characterization unit 3010. The multi-channel/multi-domain characterization unit 3010 carries out a second set of audio analysis routines. It is contemplated that the second set of audio analysis routines is more computationally intensive than the first set of audio analysis routines. For this reason the second set of analysis routines is only performed for locations which the audio has not been successfully identified by the first set of audio analysis routines. The result of the second set of audio analysis routines is applied to the multi-channel/multi-domain rules 3011. The rules may indicate that the audio from that source is known and suitable for output, known and unsuitable for output or unknown. If the multi-channel/multi-domain rules indicate that the audio is known and suitable for output, the location table may be updated with an output instruction. If the multi-channel/multi-domain rules indicate that the audio is unknown or known and not suitable for output, then the corresponding entry in the location table is updated to either indicate that the pinpoint location is to be ignored in future scans and captures, or by deletion of the pinpoint location entry.

(187) When the beam steering audio capture unit 3005 captures audio from a location stored in location table 3003 and is with an instruction as suitable for output, the captured audio from the beam steering audio capture unit 3005 is connected to an audio output 3012.

(188) As illustrated in FIG. 31, the location of microphone 2702A is not co-linear with the locations of microphones 2702B and 2702C as it does not fall on the line defined by the location of microphones 2702B and 2702C. Microphones 2702A, 2702B and 2702C define a plane. A microphone array of two omni-directional microphones 2702B and 2702C cannot distinguish between locations 2706 and 2707. The addition of a third microphone 2702A may be utilized to differentiate between points equidistant from line 2705 that fall on a line perpendicular to line 2705.

(189) A motion sensor may be provided in connection with a microphone array. The motion sensor may be an accelerometer 2701. The motion sensor may include an accelerometer, a gyroscope and/or a magnetometer/compass. A 9-axis motion sensor may be used. Because the microphone array is configured to be carried by a person, and because people move, a motion sensor may be used to ascertain change in position and/or orientation of the microphone array. It is advantageous that the motion sensor be in a fixed position relative to the microphones 2702 in the array, but need not be directly mounted on a microphone array substrate. A microphone array is useful as an audio sensor capable of multi-directional sensing. Other multi-directional sensors may be used.

(190) FIG. 31 shows an audio source location tracking and isolation system. The system includes a sensor array 3001. Sensor array 3001 may be stationary. The sensor array 3001 may also be body-mounted or adapted for mobility. The sensor array 3001 may include a microphone array or other multi-directional acoustic sensor. The multi-directional acoustic sensor may be two or three dimension capable.

(191) In the event that the sensor array 3001 is adapted to be portable or mobile, it is advantageous to also include a motion sensor rigidly-linked to the sensor array.

(192) A wide source locating unit 3002 may be responsive to the sensor array. The wide source locating unit 3002 is able to detect audio sources and their general vicinities. Advantageously the wide source locating unit 3002 has a full range of search. The wide source locating unit may be configured to generally identify the direction and/or location of an audio source and record the general location in a location table 3003. The system is also provided with a narrow source locating unit 3004 also connected to sensor array 3001. The narrow source locating unit 3004 operates on the basis of locations previously stored in the location table 3003. The narrow source locating unit 3004 will ascertain a pinpoint location of an audio source in the general vicinity identified by the entries in a location table 3003. The pinpoint location may be based on narrow source locations previously stored in the location table or wide source locations previously stored in the location table. The narrow source location identified by the narrow source locating unit 3004 may be stored in the location table 3003 and replace the prior entry that formed a basis for the narrow source locating unit scan. The system may also be provided with a beam steering audio capture unit 3005. The beam steering audio capture unit 3005 responds to the pinpoint location stored in the location table 3003. The beam steering audio capture unit 3005 may be connected to the sensor array 3001 and captures audio from the pinpoint locations set forth in the location table 3003.

(193) The location table may be updated on the basis of new pinpoint locations identified by the narrow source locating unit 3004 and on the basis of an array displacement compensation unit 3006 and/or a source movement prediction unit 3007. The array displacement compensation unit 3006 may be responsive to the accelerometer rigidly attached to the sensor array 3001. The array displacement compensation unit 3006 ascertains the change in position and orientation of the sensor array to identify a location compensation parameter. The location compensation parameter may be provided to the location table 3003 to update the pinpoint location of the audio sources relative to the new position of the sensor array. The location table 3003 output may be used for the directional cues 3101 stored in the digital audio storage unit 3307.

(194) Source movement prediction unit 3007 may also be provided to calculate a location compensation for pinpoint locations stored in the location table. The source movement prediction unit 3007 can track the interval changes in the pinpoint location of the audio sources identified and tracked by the narrow source locating unit 3004 as stored in the location table 3003. The source movement prediction unit 3007 may identify a trajectory over time and predict the source location at any given time. The source movement prediction unit 3007 may operate to update the pinpoint locations in the location table 3003.

(195) The audio information captured from the pinpoint location by the beam steering audio capture unit 3005 may be analyzed in accordance with an instruction stored in the location table 3003. Upon establishment of a pinpoint location stored in the location table 3003, it may be advantageous to identify the analysis level as gross characterization. The gross characterization unit 3008 operates to assess the audio sample captured from the pinpoint location using a first set of analysis routines. The first set of analysis routines may be computationally non-intensive routines such as analysis for repetition and frequency band. The analysis may be voice detection, cadence, frequencies, or a beacon. The audio analysis routines will query the gross rules 3009. The gross rules may indicate that the audio satisfying the rules is known and should be included in an audio output, known and should be excluded from an audio output or unknown. If the gross rules indicate that the audio is of a known type that should be included in an audio output, the location table is updated and the instruction set to output audio coming from that pinpoint location. If the gross rules indicate that the audio is known and should not be included, the location table may be updated either by deleting the location so as to avoid further pinpoint scans or simply marking the location entry to be ignored for further pinpoint scans.

(196) If the result of the analysis by the gross characterization unit 3008 and the application of rules 3009 is of unknown audio type, then the location table 3003 may be updated with an instruction for multi-channel characterization. Audio captured from a location where the location table 3003 instruction is for multi-channel analysis, audio may be passed to the multi-channel/multi-domain characterization unit 3010. The multi-channel/multi-domain characterization unit 3010 carries out a second set of audio analysis routines. It is contemplated that the second set of audio analysis routines is more computationally intensive than the first set of audio analysis routines. For this reason the second set of analysis routines is only performed for locations which the audio has not been successfully identified by the first set of audio analysis routines. The result of the second set of audio analysis routines is applied to the multi-channel/multi-domain rules 3011. The rules may indicate that the audio from that source is known and suitable for output, known and unsuitable for output or unknown. If the multi-channel/multi-domain rules indicate that the audio is known and suitable for output, the location table may be updated with an output instruction. If the multi-channel/multi-domain rules indicate that the audio is unknown or known and not suitable for output, then the corresponding entry in the location table is updated to either indicate that the pinpoint location is to be ignored in future scans and captures, or by deletion of the pinpoint location entry.

(197) When the beam steering audio capture unit 3005 captures audio from a location stored in location table 3003 and is with an instruction as suitable for output, the captured audio from the beam steering audio capture unit 3005 is connected to an audio output 3012.

(198) FIG. 32 shows a pair of headphones with multi-planar multi-directional acoustic sensors such as microphone arrays. FIG. 33 shows a top view of a substrate with a microphone array which may be part of the headphones of FIG. 32.

(199) The headphones 3201 may include a headband 3202. The headband 3202 may form an arc which, when in use, sits over the user's head. The headphones 3201 may also include ear speakers 3203 and 3204 connected to the headband 3202. The ear speakers 3203 and 3204 are colloquially referred to as “cans.”

(200) A substrate is adapted to be mounted on a headband of a set of headphones. The substrate may include three or more microphones 3208.

(201) A substrate 3205 may be adapted to be mounted on headphone headband 3202. The substrate 3205 may be connected to the headband 3202 by mounting legs 3206 and 3207. The mounting legs 3206 and 3207 may be resilient in order to absorb vibration induced by the ear speakers or otherwise and isolate acoustic transducers and an accelerometer. A beacon 3216 may be mounted on the headphones 3201. The beacon may be an acoustic or radio beacon. Acoustic beacons may be audible or inaudible. An inaudible beacon may emit ultrasound. A radio beacon may be a Bluetooth Low Energy (BLE) beacon, for example, according to the iBeacon standard.

(202) FIG. 33 shows a microphone array 3301 in an audio source location and isolation system. A beam-forming unit 3303 is responsive to a microphone array 3301. The beamforming unit 3303 may process the signals from two or more microphones in the microphone array 3301 to determine the location of an audio source, preferably the location of the audio source relative to the microphone array. A location processor 3304 may receive location information from the beam-forming system 3303. The location information may be provided to a beam-steering unit 3305 to process the signals obtained from two or more microphones in the microphone array 3301 to isolate audio emanating from the identified location. A two-dimensional array is generally suitable for identifying an azimuth direction of the source. An accelerometer 3306 may be mechanically coupled to the microphone array 3301. The accelerometer 3306 may provide information indicative of a change in location or orientation of the microphone array. This information may be provided to the location processor 3304 and utilized to narrow a location search by eliminating change in the array position and orientation from any adjustment of beam-forming and beam-scanning direction due to change in location of the audio source. The use of an accelerometer to ascertain change in position and/or change in orientation of the microphone array 3301 may reduce the computational resources required for beam forming and beam scanning.

(203) FIG. 34 shows an audio source imaging system.

(204) A location table 3003 as described in connection with FIG. 30 stores, inter alia, the location of audio sources being tracked by an audio source location system in a format suitable for the audio source location and isolation system. The format of the data indicating relative location stored in location table 3003 is not suitable for output directly to a display device. A display image translation unit 3401 is connected to the location table 3003. The display image translation unit 3401 transforms the data contained in location table 3003 to a format which is suitable for output directly or indirectly to an image display. The display image translation unit 3401 has an output suitable for use by an image display. The output of the display image translation unit 3401 is or may be converted in a conventional manner to an image 3402 referenced to sensor array position. Image 3402 is particularly suitable for displaying to a user the tracked audio sources from the point of view of the sensor array. The image may be a two-dimensional, a simulated three-dimensional image, or an actually three-dimensional image display. Such images may be suitable to display on a wearable display such as a wrist-mounted display, a Google Glass-style display or any heads-up display.

(205) The images referenced to the sensor array position 3402 may also be provided to an audio source station translation unit 3403. The audio source station translation unit 3403 may translate the image 3402 referenced to the sensor array position to an image 3404 referenced to one of the audio sources tracked in location table 3003. The audio source translation station may use a vector inversion process to translate the sensor array referenced image 3402 to an audio source referenced image 3404. For example, the image 3402 referenced to sensor array position may express the location of each audio source contained in location table 3003 as a vector with its origin at the sensor array and each source being expressed in terms of a direction and distance. If, for example, the sensor array is located at Point A and the location of an audio source B is identified by direction and distance, for example, the image 3402 referenced to sensor array position may reflect that audio source B is in the northwest direction at a distance of 20 feet. Audio source translation unit 3403 may transform the origin of the vector to a location referenced to the location of audio source B. For example, the sensor array would therefore be located 20 feet from audio source B in the southeast direction. This type of translation may be accomplished to translate an image 3402 referenced to a sensor array position to an image 3404 referenced to any audio source location contained in location table 3003.

(206) According to an alternative or additional feature, the image 3402 referenced to a sensor array position can be translated to a referenced image 3407 for any known position. A mapping station translation unit 3405 may utilize information obtained from an array position sensor 3406 and the image 3402 referenced to the sensor array in order to transform the image 3402 referenced to sensor array to a referenced image 3407 referenced to any position correlated to a location identified by an array position sensor 3406.

(207) Array position sensor 3406 may utilize transducers in order to identify the position of the sensor array in relation to a known reference point. The position sensor 3406 may be co-located with the sensor array and may utilize location services or other position sensitive transducers in order to sense the position of the sensor array. The array position sensor may be responsive to a beacon located in a known position. An example of the transformation of an image 3402 referenced to an array to an image 3407 referenced to Point O is, the position sensor determines that the sensor array is 10 feet to the west of Point O and determines that the location of audio source B is 20 feet west of the sensor array, then the mapping station translation unit may select Point O as a reference point and determine that the location of audio source B is 30 feet west of Point O. In a similar fashion the mapping station translation unit 3405 may translate the image 3402 referenced to the sensor array position to an image 3407 referenced to any location in a known direction and distance from the origin, Point O.

(208) The image generated by the audio source imaging system may be useful for any application where a particular reference position is desirable. For example, the image reference to the sensor array where the sensor array is mounted on the headband of headphones may be utilized for a heads-up image projection from a wearable display such as a Google Glass-type display unit or as an image for a wrist-mounted display unit. An image referenced to an audio source may be useful for any application where the audio source is the desired point of view. For example, an operative or team member may be outfitted to emit an audio signal as a beacon. The image referenced to the sensor array will include the position of the audio beacon and the audio source station translation unit 3403 may output the image reference to the audio source to a heads-up display worn or carried by the operative at Location B. In this manner, the operative receives a display of the audio sources being tracked by the location table 3003 but from its own point of view.

(209) Using the sensor array and known distance between a first sensor location and a second sensor location, the distance to an audio source can be ascertained by one of ordinary skill knowing (i) the angles between a line extending from a first sensor location to a second sensor location (the “base line”), and a line extending from said second sensor location to an audio source, (ii) the angle between a line extending from said first sensor location to the audio source and the base line, and (iii) the distance between the first sensor location and the second sensor location. Because of the inherent nature of sensor elements, beamforming identifies a direction in terms of a range of directions the variations within the range affects accuracy of the determinations. The distance determinations may be enhanced by increasing the distance between the sensor locations. This is done using at least a known distance between sensor locations that is large enough to overcome uncertainty in the distance caused by uncertainty in the directions.

(210) FIG. 35 shows an adaptive audio spatialization system. The system may be responsive to an audio source 3501. The audio source may be live or pre-recorded. Audio from the source may be captured with a multi-directional acoustic sensor, also referred to as a directionally discriminated acoustic sensor. An example of a multi-directional audio sensor is a microphone array. Audio from the audio source 3501 is processed by the audio spatialization engine 3502. The audio spatialization engine may apply a perceived spatial component to the audio obtained from the direction of the source. The application of the perceived spatial component may use head-related transfer functions (HRTF) applied to the audio so that the user perceives the audio source as emanating from the applied direction. The audio spatialization engine 3502 may be responsive to audio source directional cues 3503. The audio source directional cues may be provided on the basis of the relative position of an audio source or on an artificial position or direction. The audio spatialization engine 3502 may also be responsive to a listener position/orientation unit 3503. The listener position/orientation unit 3503 generates a signal representative of the listener position/orientation and is responsive to a motion sensor 3505. The motion sensor 3505 may advantageously be rigidly linked to the personal audio output device and provides a signal indicative of the position or orientation of a user or changes in the position or orientation of the user. The motion sensor may be one or more of a compass, a gyroscope, and/or an accelerometer. According to one embodiment, a nine-access motion sensor may be utilized.

(211) The audio spatialization engine 3502 has an output representing a spatialized audio signal. The output is connected to an audio output stage 3506. The audio output stage 3506 may operate as a pre-amplifier and/or amplifier for the audio signal. In addition, the audio output stage 3506 may mix other audio signals so that audio information from more than one audio source is provided to the personal speakers. The audio source directional cues 3503 may be a location table as shown in FIG. 30.

(212) It is possible that the audio cues provided are not as specific as the location specified by the location table. The reason for this is that the beam steering functionality is optimized by having a very accurate location or direction to isolate. By contrast, in many applications, the precision of the spatialization is less important to a listener than the precision required for optimum beam steering functionality. The use of less precise directionality in the monitoring of user position and orientation and application of spatialization can conserve computational resources and may not be perceptually significant to a user.

(213) The system may be used, for example, amongst a group of people each using a personal communication device linked to a customized audio delivery system in a multifaceted event. In an exemplary environment they may be participating in an event that may be spread across a large geographic area. In other cases participants may be densely assembled. Examples of multifaceted events include, but are not limited to arena venues, festival events, fairs, and conventions/exhibitions. Information may be passed between personal communication devices of the participants using point-to-point wireless communication, a distributed network of computers such as the Internet, a wireless communication network, small cell LTE, Wi-Fi, and so on. In any case, information received at the personal communications devices can include an identification of the event and an indication of available content or identification of one or more other participants possibly according to some specified criteria that can be passed to a participant's personal communication device. The system can be implemented as part of a communication system for establishing and providing preferred audio and/or a mutual permission customized audio source connection system

(214) In the described embodiments, the personal communication device can take the form of a portable media player, cellular phone, or as a handheld computing device such as a tablet computer. In any case, the personal communication device can be configured to wirelessly receive and in some cases may send a signal that can contain information that can include a menu of available content, requests for content and/or communication with or to facilitate communications with other participants and/or event updates or news flashes (announcements). The information can include a snippet or chunk of data that can be broadcasted by one or more devices to other devices that are within the transmission range of the broadcasting device(s). In one embodiment, the snippet or chunk of data can take the form of a token that can be used to seed a group of personal communication devices with the menu of available content. The token can be stored in a personal communication device and concurrently broadcasted to any other personal communication device using, for example, short message service (SMS) messaging or a Wi-Fi RF transmission. In this way, by broadcasting the information, each personal communication device can be made aware of the available content, event updates, and announcements at about the same time.

(215) In the described embodiments, the signal received at the personal communication device can include information other than the available content, event updates, and announcements. Such information can include any personal communication device identifiers, or PCDIDs, indicating the identity of those personal communication devices that have already received the information. In this way, a personal communication device can retrieve not only information related to the available content, event updates, and announcements, but other information related to those personal communication devices participating in the multifaceted event. One of the features of the PCDID is the ability to facilitate social networking within the group. In any case, the unique identifier (including any personalized information associated therewith) can be associated with the PCDID of the personal communication device and be passed between various other personal communication devices. In this way, a dynamic social network can be formed independent of or in conjunction with the available content, event updates, and announcements.

(216) In addition to available content, event updates, and announcements, and any PCDIDs used to identity personal communication devices, the information (or the token for that matter) can include other information such as a time counter used to specify a start time and a stop time for a particular music session.

(217) The menu of available content can be used to select audio content, event updates, and announcements stored or cached on each of the personal communication devices. The selection of available content, event updates, and announcements can be carried out in any number of different ways. For example, one of the ancillary services provided by the communication application can include categorizing content and/or stored on the personal communication device based upon various values of a particular music characteristic or content previously cached or individual identifications of participants. The communication application can create an alert to the presence of other participants selected on the basis of a specified criteria to facilitate ad hoc social networking connection. The criteria may be “fiends” or “contacts” within a certain distance. The criteria may also be based on common interests or other factors or information accessible to the system. The selected information may be prepared for private playing to a user of the personal communication device by way of a private listening accessory, such as headphones. In one embodiment, the music item(s) selected can be added to a playlist for private playing. The playlist can be presented for viewing on the personal communication device and in some cases, made available to the user for manual selection of specific content or connections. It should be noted that the individuals selected can be prequalified according to a specified criterion.

(218) These and other embodiments of an environment where the lighting subsystem may be deployed are discussed below with reference to FIGS. 36, 37 and 38. However, those skilled in the art will readily appreciate that the detailed description given herein with respect to these figures is for explanatory purposes only and should not be construed as limiting.

(219) FIG. 36 shows group 3600 participating in a multifaceted event. Along the lines of a music festival, group 3600 can congregate at the event. The congregating can occur in separated areas, for example, at a first stage 3620, a second stage 3622, a food court exhibition area, etc. The participants can each be apprised of event updates by, for example, SMS messaging, emails (similar to a silent disco), instant messages, or a dedicated communication app such as the aforementioned audio communication or preferred audio systems. An event update might be an announcement that a particular act is about to perform at an identified stage. Each personal communication device (PCD) can privately play content for the associated member of group 3600. The member can select the content it will receive. By privately playing it is meant that only the member in possession of the personal communication device can hear the privately played content. This audio privacy can be accomplished using private listening accessory 3602 along the lines of a head phone, ear bud, and so on. The members may be listening to the same content broadcast, or listening to customized and/or selected content. The lighting display may be correlated to the selected content.

(220) The members may be listening to the same content broadcast, or listening to customized and/or selected content.

(221) In order to participate in the multifaceted event communications, each of PCD 3614-PCD 3618 must include communications infrastructure and a control interface to select and play appropriate content. In order to assure that each of the personal communication devices in group 3600 has access to the content, a communication application (not shown) can be provided and stored on each of the personal communication devices. In one embodiment, the communication application can be part of an operating system provided upon the original purchase of a personal communication device. Alternatively, the communication application can be obtained after-market using, for example, remote media management services along the lines of iTunes. On the other hand, the communication application can be obtained in an ad hoc manner during, for example, an initial invitation session whereby part of an individual acceptance of an invitation to participate in the shared music session (using email, SMS messaging, Facebook, and so on) involves downloading and installing the communication application with a subsequent verification and acceptance.

(222) In some cases, the system may communicate over an ad hoc P2P network, or by direct by broadcast 3640 communications. It should be noted that broadcast 3640 can take the form of a wireless RF transmission using any number and combination of available wireless protocols. For example, broadcast 3640 can take the form of conventional over the air (OTA) AM or FM broadcast in which case the user can be instructed to manually input the appropriate tuning instruction to their respective personal communication device. Alternatively, broadcast 200 can take the form of a Wi-Fi or Bluetooth RF signal that the communication application can recognize as including the updated music characteristic information.

(223) If the system utilizes an ad hoc P2P network a limited number of members of group 3600 (referred to as initiators) can be identified to seed the P2P network with announcements or a menu of available content. For a more detailed description of the heuristics of distributing information in an ad hoc P2P network please refer to “On Disseminating Information Reliably Without Broadcasting”, Proc. 7th Int. Conf. on Distributed Computing Systems (ICDCS-7), pp. 74-81 Berlin, September 1987 by Alon, N., Barak, A. and Manber, U and “An Asynchronous Algorithm for Scattering Information Between the Active Nodes of a Multicomputer System”, Journal of Parallel and Distributed Computing, Vol. 3, No. 3, pp. 344-351, September 1986 by Drezner, Z. and Barak each incorporated by reference in their entireties. Assuming that member 3606 has been designated as an initiator, member 3606 can seed ad hoc P2P network with the event information. Member 3606 may be replaced by an initiation server acting as a control station.

(224) It is foreseeable that due to local conditions, it may not be possible to reliably send information from one node directly to another node in P2P network. For example, PCD 3614 belonging to member 3606 (initiator) can broadcast token T that can be received by PCD 3612 and PCD 3616 belonging to members 3604 and 3608, respectively. However, member 3610 may be too far away or may be in an area (such as behind a wall) where direct reception by PCD 3618 is unlikely. Therefore, each node of network can be instructed to retransmit the information wirelessly upon receiving information wirelessly. For example, when PCD 3616 (as well as PCD 3612) wirelessly receives the event information each can generate re-broadcast a signal that includes the event information received from member 3606. In this way, PCD 3618 can receive re-broadcast content information from PCD 3616 (as well as that from PCD 3612).

(225) In some cases, a multifaceted event can have session rules. The session rules can define various relationships and actions that can occur between the members of the group during a specific session. For example, the session rules can provide criteria for identifying networking proposals for individual members to connect during the session. In this way, by setting the session networking rules individual members can be identified to each other and establish social networking communications.

(226) FIG. 37 shows a block diagram of a representative personal communication device (PCD) 3700 in accordance with the described embodiments. PCD 3700 can be formed to include at least housing 3702 configured to enclose and support various operational circuits. In some cases, PCD 3700 can include controller 3704 used to control data storage device 206 that can be used for storing a plurality of data files that can take the form of, for example, audio data, textual data, graphical data, image data, video data and multimedia data. The stored data files can be encoded either before or after being stored using a variety of compression algorithms. It should be noted that a user can interact with manager 3712 through an interface. For example, audio content can be compressed using MP3, AAC and Apple Lossless compression protocols. Other data may be compressed using protocols appropriate to such data. The audio content can include, for example, auxiliary content files 3708 stored in memory 510 controlled by the content manager 3712. Content manager 3712 can be embodied as software executed by processor 3714 or as a separate hardware component. In any case, content manager 3712 can control the audio output of content files 3708 stored in memory 3710. The content may also include available content menus, in audio or graphic form as well as social networking criteria and/or identification.

(227) During operation, for example, content manager 3712 can select content item 3716 from auxiliary content 3708 which can be decoded using an appropriate codec. The decoded content file can then by output as audio signal 3718 to audio output interface 3720. In accordance with one embodiment, content manager 3712 can select content items 3716 identified by a user through a guide or by voice command. Furthermore content manager 3712 may receive transmission of content and play such content substantially in real time, subject to loading, buffering and decoding delays and subject to any user control such as pause or rewind or replay.

(228) Content may include a tag 3722 to identify content type or other characteristic of the auxiliary content. For example, in a music festival the tag may indicate that the content is a commercial advertisement or offer. The tag may indicate information regarding purchase of the content, or may identify the facet of the multifaceted event that the content relates to. For example, the tag may indicate that the content relates to a performance on stage.

(229) User input interface 3724 can assist a user of PCD 3700 in controlling various functions performed by PCD 3700. For example, user interface 3724 can include a touch sensitive layer (not shown) that can facilitate the use of a user touch event for inputting control instructions or the user interface may be an audio interface for voice commands. In the case where PCD 3700 includes speakers, then audio signal 3718 can be broadcast to the external environment via the speakers. However, in those situations where PCD 3700 does not include speakers, or the speakers can be bypassed, PCD 3700 can include private listening interface 3726 suitable for directing audio signal 3718 to an external transducer associated with a personal listening accessory, such as earphones, ear buds, and so on. The personal/listening device may also include a microphone for detecting and sensing audio. In this way, the user of PCD 3700 can privately listen to audio output by music manager 3712. PCD 3700 can also include wireless interface 3728 arranged to both receive and transmit information by way of any suitable wireless protocol such as, for example, Wi-Fi, Bluetooth, and so on capable of accessing various configurations of wireless networks, such as WLAN or peer to peer (P2P). It should be noted that even though only a limited set of components are shown this does not imply a limitation on the functional components that can be included in PCD 3700. For example, in addition to the components shown in FIG. 37, embodiments of PCD 3700 can also include a power connector, a data transfer component, voice recognition circuits, and so on.

(230) Content manager 3712 can customize the audio experience of the user. The audio may be processed to enhance and/or mask aspects of the audio to be delivered to the user, for example, in accordance with the techniques described herein.

(231) In another implementation, content manager 3712 can control social networking functionality. Selective networking may be provided by identifying participants in the event that satisfy a selection criteria. The system may allow a user the option of establishing networking communications with other participants who satisfy the selection criteria and designated by one or both users.

(232) A communication application 3728 can provide instructions executable by processor 3714 for controlling the operations of PCD 200. In the described embodiment, the communication application can be downloaded from an online data store automatically or as a result of a user selection at user interface 3724 from a central media management application (such as iTunes™) or from Apps Store maintained by Apple Inc. Alternatively, communication application 3728 can be present at the time of original purchase. In any case, communication application 3728 maintains a connection table to be periodically updated. The updating can occur, for example, during a synchronization operation performed between PCD 3700 and a central media management application (such as iTunes™). The updating can also occur on an ad hoc basis.

(233) Communication application 3728 can provide a mechanism by which a user of PCD 3700 can participate in a social networking experience provided that a connection between two users satisfies a criteria identifying a suggested connection. In addition to providing services required for participation in the social networking experience, communication application 3728 can provide PCD 3700 with at least the appropriate network protocols required to exchange information with other personal communication devices in a P2P network. In addition to providing the requisite communication protocols, communication application 3728 can provide services related to categorizing music items stored on PCD 3700 based upon various values of a particular music characteristic. The selection and networking function can be based in or distributed among PCDs or be server based. In a server-based system, the server may be local (logically) to the multifaceted event or remote such as a server connected through a wide area network including, without limitation, the Internet.

(234) In any case, PCD 3700 can obtain a connection token T by way of RF transmission 3730. It should be noted that if PCD 3700 is a node in a P2P network, RF transmission 3730 can originate from another personal communication device within the network. In this situation, upon receiving token T, PCD 3700 can generate re-broadcast signal 3732 that includes at least token T while storing only tokens designated for that user. In this way, other personal communication devices with the P2P network can receive connection tokens applicable to other devices. Tokens can be transmitted by way of RF transmission 3730 that originates from a central broadcaster unit. It is also possible that PCD 3700 does not have wireless capabilities, in which case the token T can be provided by the communication application 3728. In this way, a more limited session can be held since only those personal communication devices that have the same version of communication application 3728 can participate. For example, in order to participate, PCD 3700 may require the latest version of token T which can be obtained during, for example, a synchronization operation performed between the personal communication device and a central media management application.

(235) Once token T has been received, processor 3714 can determine if token T has an indication of supplemental content. For example, token T can indicate availability of content which might be background information, coupon or commercial offers, or schedules. In this case, the user may have the option to listen to the supplemental content which may be requested or accessed and can be privately played by PCD 3700. Accordingly content 3730, 3732, and 3734 each tagged as an ID that corresponds to token t1 may be accessed. In the described embodiment, a content venue 3736 can be visually displayed at interface 3724.

(236) FIG. 38 shows an event-centric networking matching system 3800. The system includes a connection server 3801 connected to a plurality of user personal communication devices 3802 by a network 3803. The personal communication devices 3802 may have an interface for users to control, provide instructions, and provide information to the system. Alternatively the instruction and information interface may be a separate terminal also connected to the network 3803. The network 3803 may be a wired or wireless local area network or wide area network. The connections may be by Bluetooth, peer-to-peer connections, small cell LTE or any other connection mechanism. The system is not specific to a particular network. The communication server 3801 may be connected to data store 3804.

(237) FIG. 38 illustrates a single data store 3804 in the form of a database management system however individual tables or distributed tables may be utilized. The data may be distributed among the users 3802 or centrally located. The data may include user profile data 3805 composed of a user ID 3809 associated with a profile 3810. The profile may include any information used by the system related to the user, for example, user name, password, gender, musical tastes, playlist, age, geographic location and any other demographic information. The system may also include a matching criteria table 3806. The criteria table may include a plurality of rules 3811, each associated with a rule number 3812. In addition, the system may include a participation table 3807 which includes a user ID 3813 as an index and a rule number 3814 correlating to rule numbers 3812 of the matching criteria table 3806. The participation table 3807 includes a list of user IDs correlated to the rule numbers and the matching criteria table 3806 includes those rule numbers correlated to matching criteria. Each user may be subscribed to one or more of the criteria as indicated by entries in the participation table 3807. The matching criteria may include one or more requirements such as an identification of an event, a location service matching criteria, demographic matching criteria, a flag indicating appearance in a contact or approved list, and other criteria. In the example of a multi-faceted event such as a concert festival, the system may first identify all users who are participating in the event, i.e. are attending the music festival. This may be accomplished by determining which users have purchased tickets or have a token on their PCD indicating they have been admitted to the event. Alternatively, participation may be determined by location services. Each user may establish or subscribe to criteria which, if satisfied, suggests a connection. A matched status connection table 3808 may be established in order to identify connections approved in accordance with the proper operation of the system. The system may go through each entry in participation table 3807. For each entry the rule corresponding to the user ID may be utilized to evaluate all of the entries in the user profile table. When an entry in the user profile table satisfies a user ID rule designation, an entry may be placed in the matched status connection table 3808 of the user ID in the user 1 field 3815. The ID of the user who satisfied the criteria may be placed in user 2 field 3816. The system may use different logic or sequences, but the idea is to create a table which has an entry for each pair of users who both satisfy the other's designated criteria. The designated criteria may be customized by each user and/or established by the system. An additional feature may permit each participant in a connection to approve or deny access even though the established criteria have been satisfied. Alternatively, one of the criteria may be approval of the matching user.

(238) The system may also be able to establish communication groups so that connections may be one-to-many or even one-to-all. This may be established by user ID corresponding to a group criteria and each individual user who matches the group criteria is connected in the group. The system may impose an artificial limitation of allowing participation in only a single group.

(239) FIG. 39 shows an audio play system 3901. The audio play system 3901 has an output representative of one or more aspects of the audio selection. A display attribute generation unit 3903 may be provided and is responsive to the signal representative of content 3902. The customized audio play system 3901 may be connected to personal audio speakers 3906. The personal audio speakers 3906 may be headphones, earphones, or any other device for converting electrical signals to audio.

(240) The display 3905 may constitute one or more light elements. The light elements may be LED light elements or any other light emitting element. The display 3905 may be monochrome or controllable to vary the color, intensity, and image of the lighting output. The display 3905 may have one or more color points such as the Pixmob or Xyloband displays. The display 3905 may be suitable to display image or video. The display 3905 may be mounted on a headphone or may be wearable in some other fashion, although it is not necessary for the display 3905 to be mounted on or even co-located with a user. The signal representative of content 3902 must be derived in part from the operational parameters of the customized audio play system. While the display 3905 may in part be controlled by audio intensity in the fashion of a light organ, the signal representative of content must include, in part, a signal representative of operating parameters. The operating parameters may include audio source selection, non-audio control signals, user-selected parameters, system-selected parameters, content-type parameters or other non-audio parameters.

(241) A display attribute generation unit 3903 may be provided to generate signals to be displayed. Those signals may be provided to the display driver 3904.

(242) As an example, the light display system might be utilized in connection with a system shown in FIGS. 36-38 for a multi-stage concert event. In such a multi-stage concert event, each user may customize the audio being provided to a headphone by selection of one stage to be included in the user's customized audio. The light attribute to be displayed will in some way correspond the selected stage. For example, a country music stage may be designated by the color red, a rock and roll stage may be designated by the color white, and a techno stage may be designated by blue. When a user selects which stage to include in a customized audio feed, the display 3905 may be illuminated with the corresponding color.

(243) The invention is described in detail with respect to preferred embodiments, and it will now be apparent from the foregoing to those skilled in the art that changes and modifications may be made without departing from the invention in its broader aspects, and the invention, therefore, as defined in the claims, is intended to cover all such changes and modifications that fall within the true spirit of the invention. For the sake of clarity, D/A and A/D conversions and specification of hardware or software driven processing may not be specified if it is well understood by those of ordinary skill in the art. The scope of the disclosures should be understood to include analog processing and/or digital processing and hardware and/or software driven components

(244) Thus, specific apparatus for and methods of a customized audio display system have been disclosed. It should be apparent, however, to those skilled in the art that many more modifications besides those already described are possible without departing from the inventive concepts herein. The inventive subject matter, therefore, is not to be restricted except in the spirit of the disclosure. Moreover, in interpreting the disclosure, all terms should be interpreted in the broadest possible manner consistent with the context. In particular, the terms “comprises” and “comprising” should be interpreted as referring to elements, components, or steps in a non-exclusive manner, indicating that the referenced elements, components, or steps may be present, or utilized, or combined with other elements, components, or steps that are not expressly referenced.

(245) FIG. 40 shows a schematic of a narrowcast messaging system. In particular, FIG. 40 illustrates the receiver side of the messaging system. A transducer 4001 is provided to convert acoustic signals to electrical signals. The transducer of FIG. 40 may be suitable for detecting acoustic waves in the ultrasound frequency band. The transducer may also be suitable to detect acoustic signals in the audible frequency range. The transducer 4001 may be connected to an ultrasound isolation unit 4002. The ultrasound isolation unit 4002 may be responsive to a channel control unit 4003. The channel control unit 4003 may be responsive to a permissioning subsystem 4004. A frequency transposition unit 4005 may be responsive to the ultrasound isolation unit 4002 and the channel control unit 4003. The frequency transposition unit 4005 may have an output of an electrical signal corresponding to audio information. The audio information may be provided to an audio signal processing unit 4006.

(246) The audio signal processing unit 4006 may be provided to output audio information to a user. In one embodiment the audio signal processing unit may be a preamp connected to a speaker such as an earphone or headphone. In another embodiment the audio signal processing may be an audio customization unit.

(247) In operation, an ultrasonic beacon system may be provided. An example of a beacon system is the iBeacon compatible transmitters. See https://developer.apple.com/iBeacon/. The Apple iBeacon system use Bluetooth LE. A beacon system may include an ultrasonic transmitter. Beacons, such as the iBeacon have localized transmission and are designed to assist in determining proximity of a receiving device to the beacon.

(248) A drawback to a proximity sensing system is that it can only determine proximity to a particular beacon and to some extent distance from a particular beacon. The beacon may be designed to work with a directional sensing audio receiver.

(249) An embodiment may include a microphone array having two or more spaced microphones. The microphones may receive the signal emitted by a beacon and determine the direction to that beacon. The direction may be represented in the form a vector. One or more additional beacons may be provided to facilitate the direction-sensing microphone to identify one or more vectors indicating the direction of the one or more additional beacons.

(250) FIG. 41 shows a location generation unit 4101 which may be used with the narrowcast messaging system and FIG. 42 shows an embodiment of a location generation system which may be utilized. A position map 4201 may be a digital representation of the absolute or relative locations of two or more beacons.

(251) A directionally discriminating acoustic sensor 4202 may be connected to a directional vector generation unit 4203. The directional vector generation unit 4203 may operate to determine the direction of a beacon 4204 relative to the acoustic sensor 4202. The directional vector generation unit 4203 may also determine a vector representing the direction of a second beacon 4205 relative to the acoustic sensor 4202 which may be a microphone array. A position processor 4206 may be responsive to the position map 4201 and the directional vector generation unit 4203. The position map is a digital representation of information sufficient to specify the relative positioning of beacons 4204 and 4205. The relative positioning of the beacons and directionality of the beacons relative to the directionally discriminating acoustic sensor 4202 is sufficient to determine the location of the array relative to the beacons. In addition if the absolute position of one or more of the beacons is known the relative location of the array is sufficient to determine the absolute location of the array. A rule set 4102 may be responsive to the location generation unit 4101 and a user ID 4103 corresponding to the sensor 4202. The location generation unit 4101 as described in connection with FIG. 42 may base the location, in part, on information reflecting the site location 4104 and a site identification 4105.

(252) The rule set 4102 includes logic that facilitates generation of a channel ID 4106. The channel ID represents content or instructions to be played or executed by a personal communication device on the basis of the location of sensor 4202 coinciding with a designated location subject to qualifications (contingencies) as applied by the rule sets 4102. The channel control unit 4003 may provide the channel ID 4106 to the ultrasound identification unit 4002 and the frequency transposition unit 4005.

(253) In operation, a user wearing or carrying a microphone array, may obtain transmissions of selected information based upon positioning in or traversal of a beacon field. One example of a beacon field may be installed in a retail department store. As the array moves through the department store the system facilitates determining the precise location of the array. iBeacon technology determines proximity and utilizes signal strength to infer some measure of confidence and distance. An iBeacon has no directional sensitivity. Thus if an iBeacon infers a distance of 3 meters, the sensor is inferred to lie on the circumference of a circle that is 6 meters in diameter. An iBeacon is unable to determine if the device is at an exact position of interest or up to six meters away. The location may be utilized along with other parameters such as user preferences and system preferences to determine what information to provide to a user. For example a user may select to enable messaging for special offers related to a particular type of product, for example, men's clothing. The retail outlet may establish a message that communicates a special offer for certain golf shirt. As the microphone array reaches a predetermined location, which may be a location immediately adjacent to the golf shirt, the system may communicate a special offer to the user triggered by being in that location. The message may be a promotional offer for the nearby golf shirt, for example, other types of offers may also be suitable such as a promotional offer for a golfing vacation package or a promotional offer for a different related or unrelated product. The position in this example is important as the message may not be relevant to a position up to 6 meters away.

(254) Having determined the position of an array and permissioning for a particular message, the message may be transmitted to the user. It is desirable to have the ability to restrict the message to the individual user. One embodiment is the transmission of an inaudible ultrasonic wave containing the message. Various mechanisms can be provided to allow the user to receive and isolate an ultrasonic transmission. For example the user system may be informed of the direction of the ultrasonic transmission source relative to the microphone array. The microphone array may use beamforming techniques to isolate that direction.

(255) Another embodiment may provide for multi-channel ultrasonic transmissions. The transmission information may be modulated at different frequencies or may be provided in a specified frequency band. The isolation system may be provided to isolate the modulated transmission on the basis of its modulation frequency or filter communications outside of the specified frequency band.

(256) Once the desired ultrasonic frequency is received and isolated, it remains an inaudible signal. The inaudible signal may be subject to frequency transposition converting the signal from an inaudible frequency to an audible frequency, for example, a frequency in the voice band. In this manner a personalized narrowcast message may be transmitted to a user on the basis of being in or having been in a particular location.

(257) FIG. 43 shows a multi-directional acoustic sensor integrated into a ski helmet 4300. Multi-directional acoustic sensors may be similarly integrated into other types of headgear, particularly protective headgear. For example, but without limitation, construction hardhats, bicycle helmets, football helmets, hockey helmets, skateboarding helmets, batting helmets, combat helmets, or any other kind of protective headgear. The elements described herein may be integrated directly into the outer surface of protective headgear integrated into a shell attached to the protective headgear.

(258) The headgear may include a plurality of microphones 4301 mounted onto a surface of the headgear 4300. Because of the typical dimensions of protective headgear it is possible to position microphone element 4301 at a greater distance from each other than microphone elements integrated into the headband of a pair of headphones. The accuracy of the sensing array is dependent in part upon the distance between the microphone elements, and as such implementation of a multi-directional acoustic sensor on protective headgear may enhance the accuracy of the directional location and isolation.

(259) One or more additional microphone elements 4302 may be attached to the protective headgear 4300 at a position that is not coplanar with microphone element 4301. Advantageously, microphone element 4301 may be positioned around the crown of the headgear and additional microphones 4302 may be positioned at a location corresponding to a wearer's ears or lower. The protective headgear 4300 may also be provided with a motion sensor 4303. The location of the motion sensor is not critical.

(260) The protective headgear 4300 may also be provided with an ultrasonic transmitter 4304. The ultrasonic transmitter 4304 is useful to generate an ultrasound signal operating as a beacon. The ultrasound signal may be inaudible and may also be coded for identification purposes. In an alternative configuration, an audible acoustic transmitter or radio frequency transmitter, such as an iBeacon or other BLE beacon may be used. The transmitter facilitates identification and location of the protective headgear.

(261) FIGS. 44A and 44B show a multi-directional acoustic sensor integrated into a ski jacket 4400. Multi-directional acoustic sensors may be similarly integrated into other types of outerwear, particularly activewear. For example, but without limitation, ski jackets, sports jerseys, jumpsuits, flack jackets, biker jackets, bomber jackets, dusters, water ski vests, live preservers, or any other garment to be worn on a torso. The acoustic sensor elements described herein may be integrated directly into the outer surface of the outerwear or integrated into a shell worn over the outerwear.

(262) The jacket may include a plurality of microphones 4401 mounted onto a surface of the jacket 4400. Because of the typical dimensions of outerwear it is possible to position microphone element 4401 at a greater distance from each other than microphone elements integrated into the headband of a pair of headphones. The accuracy of the sensing array is dependent in part upon the distance between the microphone elements, and as such implementation of a multi-directional acoustic sensor on outerwear may enhance the accuracy of the directional location and isolation. Microphone element 4401 may be positioned directly on the jacket 4400 or microphone elements 4401 may be positioned on a base 4405 attached by a fastener 4406. The fastener 4406 may be hook and loop buttons, snaps, or other fasteners.

(263) One or more additional microphone elements 4402 may be attached to the jacket 4400 at a position that is not coplanar with microphone element 4401. Advantageously, microphone element 4401 may be positioned on the shoulders or around the collar and neckline and additional microphones 4402 may be positioned at a location lower than the microphone elements 4401. The jacket 4400 may also be provided with a motion sensor 4403. The location of the motion sensor is not critical.

(264) The jacket 4400 may also be provided with an ultrasonic transmitter 4404. The ultrasonic transmitter 4404 is useful to generate an ultrasound signal operating as a beacon. The ultrasound signal may be inaudible and may also be coded for identification purposes. In an alternative configuration, an audible acoustic transmitter or radio frequency transmitter, such as an iBeacon or other BLE beacon may be used. The transmitter facilitates identification and location of the protective outerwear.

(265) The techniques, processes and apparatus described may be utilized to control operation of any device and conserve use of resources based on conditions detected or applicable to the device.

(266) FIG. 45 shows a smartphone with an integrated microphone. FIG. 46 shows a smartphone or smartphone case with an integrated microphone array. FIG. 47 shows a smartphone case with an integrated microphone array and an auxiliary power supply. FIG. 48 illustrates a smartphone case with a removable microphone array and battery module. FIG. 49 illustrates a smartphone or smartphone case with an integrated microphone array having pivot-mounted legs and aerial. FIG. 50 shows a smartphone 4901 or smartphone case 4701 according to FIG. 49 in a deployed configuration. A base 4902 may be free-standing or associated with a mobile computing device, or may be a smartphone case as shown in FIGS. 47 and 48, and may have three or more microphones 4903 mounted on the base 4901. The microphones 4903 may be mounted in a configuration with a first microphone mounted in a position that is not co-linear with a second microphone and a third microphone. A fourth microphone 4910 may be mounted in a location that is not co-planar with the first, second, and third microphones 4903. The base 4901 may be the outer housing of said mobile computing device as shown in FIGS. 49 and 50. The base 4701 may be a protective case configured to receive a mobile computing device 4702 as shown in FIGS. 47 and 48.

(267) The device may have an auxiliary power supply 4706 mounted in the protective case 4701. The microphones 4805 may be mounted in a detachable module 4806. The auxiliary power supply 4706 may be mounted in the detachable module 4806. The fourth microphone 4910 may be mounted on a boom 4905D that may be pivot mounted on the base 4902. The boom may be a telescoping boom. The device may have three or more legs 4905 that are pivot mounted on the base and may have resilient material 4906 mounted on the legs 4905. The resilient material 4906 may be a vibration-damping material.

(268) The techniques, processes and apparatus described may be utilized to control operation of any device and conserve use of resources based on conditions detected or applicable to the device. For the sake of clarity, D/A and A/D conversions and specification of hardware or software driven processing may not be specified if it is well understood by those of ordinary skill in the art. The scope of the disclosures should be understood to include analog processing and/or digital processing and hardware and/or software driven components.

(269) The invention is described in detail with respect to preferred embodiments, and it will now be apparent from the foregoing to those skilled in the art that changes and modifications may be made without departing from the invention in its broader aspects, and the invention, therefore, as defined in the claims, is intended to cover all such changes and modifications that fall within the true spirit of the invention.

(270) Thus, specific apparatus for and methods of audio signature generation and automatic content recognition have been disclosed. It should be apparent, however, to those skilled in the art that many more modifications besides those already described are possible without departing from the inventive concepts herein. The inventive subject matter, therefore, is not to be restricted except in the spirit of the disclosure. Moreover, in interpreting the disclosure, all terms should be interpreted in the broadest possible manner consistent with the context. In particular, the terms “comprises” and “comprising” should be interpreted as referring to elements, components, or steps in a non-exclusive manner, indicating that the referenced elements, components, or steps may be present, or utilized, or combined with other elements, components, or steps that are not expressly referenced.