A method (800) to detect a hotword in a spoken utterance (120) includes receiving a sequence of input frames (210) characterizing streaming multi-channel audio (118). Each channel (119) of the streaming multi-channel audio includes respective audio features (510) captured by a separate dedicated microphone (107). For each input frame, the method includes processing, using a three-dimensional (3D) single value decomposition filter (SVDF) input layer (302) of a memorized neural network (300), the respective audio features of each channel in parallel and generating a corresponding multi-channel audio feature representation (420) based on a concatenation of the respective audio features (344). The method also includes generating, using sequentially-stacked SVDF layers (350), a probability score (360) indicating a presence of a hotword in the audio. The method also includes determining whether the probability score satisfies a threshold and, when satisfied, initiating a wake-up process on a user device (102).

A wake word detector, at a server of a content delivery network (CDN) that provides audio (or other) content to a device, such as a voice-enabled device, detects false wake words in the audio content. The CDN wake word detector analyzes the audio stream to determine if the audio stream contains any audio that sounds like the wake word. If so, the CDN wake word detector can generate metadata that describes the time period, within the audio content, in which the false wake word was encountered. The metadata can include time offsets, from the start of the audio content, which can instruct a voice-enabled device to deactivate during the time period. This metadata is stored and then sent to the media-playback device requests the media content. The media-playback device can then instruct or inform the voice-enabled device of the presence of the false wake word. In this way, the wake word detector, at the voice-enabled device, is not activated to receive the false wake word.

A second wake word detector, at a media-playback device, that plays audio (or other) content to a device, such as a voice-enabled device, detects false wake words in the audio content. The second wake word detector analyzes the audio stream to determine if the audio stream contains any audio that sounds like the wake word. If so, the second wake word detector can generate one of a plurality of instructions that describes the time period, within the audio content, in which the false wake word was encountered. The instruction can cause a first wake word detector to assume one of a plurality of configurations. The media-playback device can then instruct or inform the voice-enabled device of the presence of the false wake word. In this way, the wake word detector, at the voice-enabled device, is not activated to receive the false wake word or ignores the wake word.

An audio services agent manager (ASAM) of a control device can provide an improved user experience by iterating through a list of audio services agents so as to verify that a requested audio service is provided. The ASAM can receive an audio input from an audio input device and direct the audio input as an audio command to an audio services agent based on an audio service rule. The ASAM can verify processing of the audio command by the audio service agent based on a response received by the ASAM from the audio service agent. If the response indicates success, then the audio service requested by the audio command is provided, otherwise, the ASAM can select another audio services agent based on the audio service rule and proceed to direct the audio command to the newly selected audio services agent so as to provide the requested audio service.

A device includes a memory configured to store instructions and one or more processors configured to execute the instructions. The one or more processors are configured to execute the instructions to receive audio data including a first audio frame corresponding to a first output of a first microphone and a second audio frame corresponding to a second output of a second microphone. The one or more processors are also configured to execute the instructions to provide the audio data to a first noise-suppression network and a second noise-suppression network. The first noise-suppression network is configured to generate a first noise-suppressed audio frame and the second noise-suppression network is configured to generate a second noise-suppressed audio frame. The one or more processors are further configured to execute the instructions to provide the noise-suppressed audio frames to an attention-pooling network. The attention-pooling network is configured to generate an output noise-suppressed audio frame.

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for implementing hotword recognition and passive assistance are disclosed. In one aspect, a method includes the actions of receiving, by a computing device that is operating in a low-power mode and that includes a display that displays a graphical interface while the computing device is in the low-power mode and that is configured to exit the low-power mode in response to detecting a first hotword, audio data corresponding to an utterance. The method further includes determining that the audio data includes a second, different hotword. The method further includes obtaining a transcription of the utterance by performing speech recognition on the audio data. The method further includes generating an additional user interface. The method further includes providing, for output on the display, the additional graphical interface.

Hot word free adaptation, of function(s) of an automated assistant, responsive to determining, based on gaze measure(s) and/or active speech measure(s), that a user is engaging with the automated assistant. Implementations relate to techniques for mitigating false positive occurrences of and/or false negative occurrences, of hot word free adaptation, through utilization of a permissive parameter set in some situation(s) and a restrictive parameter set in other situation(s). For example, utilizing the restrictive parameter set when it is determined that a user is engaged in conversation with additional user(s). The permissive parameter set includes permissive parameter(s) that are more permissive than counterpart(s) in the restrictive parameter set. A parameter set is utilized in determining whether condition(s) are satisfied, where those condition(s), if satisfied, indicate that the user is engaging in hot word free interaction with the automated assistant and result in adaptation of function(s) of the automated assistant

Hot word free adaptation, of one or more function(s) of an automated assistant, responsive to determining, based on gaze measure(s) and/or active speech measure(s), that a user is engaging with the automated assistant. Implementations relate to various techniques for mitigating false positive occurrences of and/or false negative occurrences, of hot word free adaptation, through utilization of personalized parameter(s) for at least some user(s) of an assistant device. The personalized parameter(s) are utilized in determining whether condition(s) are satisfied, where those condition(s), if satisfied, indicate that the user is engaging in hot word free interaction with the automated assistant and result in adaptation of function(s) of the automated assistant.

Implementations described herein relate to transitioning a computing device between operating modes according to whether the computing device is suitably oriented for received non-audio related gestures. For instance, the user can attach a portable computing device to a docking station of a vehicle and, while in transit, wave their hand near the portable computing device in order to invoke the automated assistant. Such action by the user can be detected by a proximity sensor and/or any other device capable of determining a context of the portable computing device and/or an interest of the user in invoking the automated assistant. In some implementations location, orientation, and/or motion of the portable computing device can be detected and used in combination with an output of the proximity sensor to determine whether to invoke the automated assistant in response to an input gesture from the user.

The present disclosure is generally related to a data processing system to selectively invoke applications for execution. A data processing system can receive an input audio signal and can parse the input audio signal to identify a command. The data processing system can identify a first functionality of a first digital assistant application hosted on the data processing system in the vehicle and a second functionality of a second digital assistant application accessible via a client device. The data processing system can determine that one of the first functionality or the second functionality supports the command. The data processing system can select one of the first digital assistant application or the second digital assistant application based on the determination. The data processing system invoke one of the first digital assistant application or the second digital assistant application based on the selection.