Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for hotword detection on multiple devices are disclosed. In one aspect, a method includes the actions of receiving, by a first computing device, audio data that corresponds to an utterance. The actions further include determining a first value corresponding to a likelihood that the utterance includes a hotword. The actions further include receiving a second value corresponding to a likelihood that the utterance includes the hotword, the second value being determined by a second computing device. The actions further include comparing the first value and the second value. The actions further include based on comparing the first value to the second value, initiating speech recognition processing on the audio data.

Disclosed herein are system, apparatus, article of manufacture, method and/or computer program product embodiments, and/or combinations and sub-combinations thereof, for providing voice control using multiple digital assistants. In some embodiments, a voice platform operates to receive a voice input from a user. The voice platform selects a digital assistant from a plurality of digital assistants based on a trigger word. The voice platform then generates an intent from the voice input using the selected digital assistant. The voice platform then transmits the intent to a media device for processing.

A speech interface device is configured to detect an interrupt event and process a voice command without detecting a wakeword. The device includes on-device interrupt architecture configured to detect when device-directed speech is present and send audio data to a remote system for speech processing. This architecture includes an interrupt detector that detects an interrupt event (e.g., device-directed speech) with low latency, enabling the device to quickly lower a volume of output audio and/or perform other actions in response to a potential voice command. In addition, the architecture includes a device directed classifier that processes an entire utterance and corresponding semantic information and detects device-directed speech with high accuracy. Using the device directed classifier, the device may reject the interrupt event and increase a volume of the output audio or may accept the interrupt event, causing the output audio to end and performing speech processing on the audio data.

Systems and methods detecting a spoken sentence in a speech recognition system are disclosed herein. Speech data is buffered based on an audio signal captured at a computing device operating in an active mode. The speech data is buffered irrespective of whether the speech data comprises a signature word. The buffered speech data is processed to detect a presence of the sentence comprising at least one command and a query for the computing device. Processing the buffered speech data includes detecting the signature word in the buffered speech data, and in response to detecting the signature word in the speech data, initiating detection of the sentence in the buffered speech data.

Systems, methods performed by data processing apparatus and computer storage media encoded with computer programs for receiving an utterance from a user in a multi-user environment, each user having an associated set of available resources, determining that the received utterance includes at least one predetermined word, comparing speaker identification features of the uttered predetermined word with speaker identification features of each of a plurality of previous utterances of the predetermined word, the plurality of previous predetermined word utterances corresponding to different known users in the multi-user environment, attempting to identify the user associated with the uttered predetermined word as matching one of the known users in the multi-user environment, and based on a result of the attempt to identify, selectively providing the user with access to one or more resources associated with a corresponding known user.

A method for optimizing speech recognition includes receiving a first acoustic segment characterizing a hotword detected by a hotword detector in streaming audio captured by a user device, extracting one or more hotword attributes from the first acoustic segment, and adjusting, based on the one or more hotword attributes extracted from the first acoustic segment, one or more speech recognition parameters of an automated speech recognition (ASR) model. After adjusting the speech recognition parameters of the ASR model, the method also includes processing, using the ASR model, a second acoustic segment to generate a speech recognition result. The second acoustic segment characterizes a spoken query/command that follows the first acoustic segment in the streaming audio captured by the user device.

A data processing apparatus includes storage circuitry to store a plurality of sound recordings, receiving circuitry to receive input data indicative of one or more sounds detected by a microphone, selection circuitry to select, from the plurality of sounds recordings, one or more candidate sound recordings in dependence upon the input data and output circuitry to output data in dependence upon one or more of the candidate sound recordings.

A robot according to the present disclosure comprises: a microphone; a camera disposed to face a predetermined direction; and a processor configured to: inactivate driving of the camera and activate driving of the microphone, if a driving mode of the robot is set to a user monitoring mode; acquire a sound signal through the microphone; activate the driving of the camera based on an event estimated from the acquired sound signal; confirm the event from the image acquired through the camera; and control at least one constituent included in the robot to perform an operation based on the confirmed event.

Systems and methods to implement commands based on selection sequences to a user interface are disclosed. Exemplary implementations may: store, electronic storage, a library of terms utterable by users that facilitate implementation of intended results; obtain audio information representing sounds captured by a client computing platform; detect the spoken terms uttered by the user present within the audio information; determine whether the spoken terms detected are included in the library of terms; responsive to determination that the spoken terms are not included in the library of terms, effectuate presentation of an error message via the user interface; record a selection sequence that the user performs subsequent to the presentation of the error message that causes a result; correlate the selection sequence with the spoken terms based on the selection sequence recorded subsequent to error message to generate correlation; and store the correlation to the electronic storage.

An electronic apparatus is provided. The electronic apparatus includes a communication interface with communication circuitry, a memory configured to store at least one instruction and a processor, and the processor is configured to receive a first audio recognized as a wake up word by an external device from the external device, determine whether the first audio corresponds to the wake up word by analyzing the first audio, based on determining that the first audio does not correspond to the wake up word, obtain a neural network model for detecting a wake up word misrecognition based on the first audio, and transmit information regarding the neural network model to the external device.