Described herein are systems and methods for compressing and speeding up dense matrix multiplications as found, for examples, in the fully connected and recurrent layers of neural networks for embedded large vocabulary continuous speech recognition (LVCSR). For compression, trace norm regularization technique embodiments were introduced and studied for training low rank factored versions of matrix multiplications. Compared to standard low rank training, the methods more consistently lead to good accuracy versus number of parameter trade-offs and can be used to speed-up training of large models. Faster inference may be further enabled on ARM processors through kernels optimized for small batch sizes, resulting in speed ups over the currently used library. Beyond LVCSR, the techniques are also generally applicable to embedded neural networks with large fully connected or recurrent layers.

Systems and methods detecting a spoken sentence in a speech recognition system are disclosed herein. Speech data is buffered based on an audio signal captured at a computing device operating in an active mode. The speech data is buffered irrespective of whether the speech data comprises a signature word. The buffered speech data is processed to detect a presence of the sentence comprising at least one command and a query for the computing device. Processing the buffered speech data includes detecting the signature word in the buffered speech data, and in response to detecting the signature word in the speech data, initiating detection of the sentence in the buffered speech data.

In one embodiment, a method includes receiving at a head-mounted device a speech input from a user and a visual input captured by cameras of the head-mounted device, wherein the visual input comprises subjects and attributes associated with the subjects, and wherein the speech input comprises a co-reference to one or more of the subjects, resolving entities corresponding to the subjects associated with the co-reference based on the attributes and the co-reference, and presenting a communication content responsive to the speech input and the visual input at the head-mounted device, wherein the communication content comprises information associated with executing results of tasks corresponding to the resolved entities.

An electronic apparatus is provided. The electronic apparatus includes a communication interface with communication circuitry, a memory configured to store at least one instruction and a processor, and the processor is configured to receive a first audio recognized as a wake up word by an external device from the external device, determine whether the first audio corresponds to the wake up word by analyzing the first audio, based on determining that the first audio does not correspond to the wake up word, obtain a neural network model for detecting a wake up word misrecognition based on the first audio, and transmit information regarding the neural network model to the external device.

Utilizing an adaptive multichannel technique to mitigate reverberation present in received audio signals, prior to providing corresponding audio data to one or more additional component(s), such as automatic speech recognition (ASR) components. Implementations disclosed herein are “adaptive”, in that they utilize a filter, in the reverberation mitigation, that is online, causal and varies depending on characteristics of the input. Implementations disclosed herein are “multichannel”, in that a corresponding audio signal is received from each of multiple audio transducers (also referred to herein as “microphones”) of a client device, and the multiple audio signals (e.g., frequency domain representations thereof) are utilized in updating of the filter—and dereverberation occurs for audio data corresponding to each of the audio signals (e.g., frequency domain representations thereof) prior to the audio data being provided to ASR component(s) and/or other component(s).

A probability matrix P is obtained on the basis of an acoustic feature amount sequence, the probability matrix P being the sum for all symbols c.sub.n of the product of an output probability distribution vector z.sub.n having an element corresponding to the appearance probability of each entry k of the n-th symbol c.sub.n for the acoustic feature amount sequence and an attention weight vector α.sub.n having an element corresponding to an attention weight representing the degree of relevance of each frame t of the acoustic feature amount sequence with respect to a timing at which the symbol c.sub.n appears; a label sequence corresponding to the acoustic feature amount sequence in a case where a model parameter is provided is obtained; a CTC loss of the label sequence for a symbol sequence corresponding to the acoustic feature amount sequence is obtained using the symbol sequence and the label sequence; a KLD loss of the label sequence for a matrix corresponding to the probability matrix P is obtained using the matrix corresponding to the probability matrix P and the label sequence; and the model parameter is updated on the basis of an integrated loss obtained by integrating the CTC loss and the KLD loss, and the processing is repeated until an end condition is satisfied.

Provided is a method including obtaining a corpus and an associated set of domain indicators. The method includes learning a set of vectors in an embedding space based on n-grams of the corpus. The method includes updating ontology graphs comprising a set of vertices and edges associating the set of vertices with each other. The method also includes determining a vector cluster using hierarchical clustering based on distances of the set of vectors with respect to each other in the embedding space and determining a hierarchy of the ontology graphs based on a set of domain indicators of a respective set of vertices corresponding to vectors of the vector cluster. The method also includes updating an index based on the ontology graphs.

Embodiments provide for improved data sequence validity processing, for example to determine validity of sentences or other language within a particular language domain. Such improved processing is useful at least for arranging data sequences based on determined validity, and/or making determinations and/or performing actions based on the determined validity. A determined probability (e.g., transformed into the perplexity space) of each token appearing in a data sequence is used in any of a myriad of manners to perform such data sequence validity processing. Example embodiments provide for generating a perplexity value set for each data sequence in a plurality of data sequences, generating a probabilistic ranking set for the plurality of data sequences based on the perplexity value sets and at least one sequence ranking metric, and generating an arrangement of the plurality of data sequences based on the probabilistic ranking set.

Disclosed herein is a method for data augmentation, which includes pretraining latent variables using first data corresponding to target speech and second data corresponding to general speech, training data augmentation parameters by receiving the first data and the second data as input, and augmenting target data using the first data and the second data through the pretrained latent variables and the trained parameters.

Methods and systems for performing at least one audio activity (e.g., conducting a phone call or playing music or other audio content) in an environment including by determining an estimated location of a user in the environment in response to sound uttered by the user (e.g., a voice command), and controlling the audio activity in response to determining the estimated user location. The environment may have zones which are indicated by a zone map and estimation of the user location may include estimating in which of the zones the user is located. The audio activity may be performed using microphones and loudspeakers which are implemented in or coupled to smart audio devices.