Systems and processes for operating an intelligent automated assistant are provided. In accordance with one example, a method includes, at an electronic device with one or more processors, memory, and a plurality of microphones, sampling, at each of the plurality of microphones of the electronic device, an audio signal to obtain a plurality of audio signals; processing the plurality of audio signals to obtain a plurality of audio streams; and determining, based on the plurality of audio streams, whether any of the plurality of audio signals corresponds to a spoken trigger. The method further includes, in accordance with a determination that the plurality of audio signals corresponds to the spoken trigger, initiating a session of the digital assistant; and in accordance with a determination that the plurality of audio signals does not correspond to the spoken trigger, foregoing initiating a session of the digital assistant.

A method, an electronic device and a readable storage medium for creating a label marking model are disclosed. The method for creating the label marking model includes: obtaining text data and determining a word or phrase to be marked in the text data; according to the word or phrase to be marked, constructing a first training sample of the text data corresponding to a word or phrase replacing task and a second training sample corresponding to a label marking task; training a neural network model with a plurality of the first training samples and a plurality of the second training samples, respectively, until a loss function of the word or phrase replacing task and a loss function of the label marking task satisfy a preset condition, to obtain the label marking model.

A method, an electronic device and a readable storage medium for creating a label marking model are disclosed. The method for creating the label marking model includes: obtaining text data and determining a word or phrase to be marked in the text data; according to the word or phrase to be marked, constructing a first training sample of the text data corresponding to a word or phrase replacing task and a second training sample corresponding to a label marking task; training a neural network model with a plurality of the first training samples and a plurality of the second training samples, respectively, until a loss function of the word or phrase replacing task and a loss function of the label marking task satisfy a preset condition, to obtain the label marking model.

A speech recognition system and method thereof are provided. The speech recognition system connects to an external general-purpose speech recognition system, and including a storage unit and a processing unit. The storage unit stores a specific application speech recognition module, a comparison module and an enhancement module. The specific application speech recognition module converts a speech signal into a first phonetic text. The general-purpose speech recognition system converts the speech signal into a written text. The comparison module receives the first phonetic text and the written text, converts the written text into a second phonetic text, and aligns the second phonetic text with the first phonetic text according to similarity of pronunciation to output a phonetic text alignment result. The enhancement module receives the phonetic text alignment result, and constructs with the written text and the first phonetic text after path weighting to form an outputting recognized text.

A speech recognition system and method thereof are provided. The speech recognition system connects to an external general-purpose speech recognition system, and including a storage unit and a processing unit. The storage unit stores a specific application speech recognition module, a comparison module and an enhancement module. The specific application speech recognition module converts a speech signal into a first phonetic text. The general-purpose speech recognition system converts the speech signal into a written text. The comparison module receives the first phonetic text and the written text, converts the written text into a second phonetic text, and aligns the second phonetic text with the first phonetic text according to similarity of pronunciation to output a phonetic text alignment result. The enhancement module receives the phonetic text alignment result, and constructs with the written text and the first phonetic text after path weighting to form an outputting recognized text.

A method for voice recognition includes: performing by an electronic device, voice recognition on voice information; and updating by the electronic device, a waiting duration for EPD from a first preset duration to a second preset duration in response to recognizing a preset keyword from the voice information, where the first preset duration is less than the second preset duration.

A method for training a speech recognition model with a loss function includes receiving an audio signal including a first segment corresponding to audio spoken by a first speaker, a second segment corresponding to audio spoken by a second speaker, and an overlapping region where the first segment overlaps the second segment. The overlapping region includes a known start time and a known end time. The method also includes generating a respective masked audio embedding for each of the first and second speakers. The method also includes applying a masking loss after the known end time to the respective masked audio embedding for the first speaker when the first speaker was speaking prior to the known start time, or applying the masking loss prior to the known start time when the first speaker was speaking after the known end time.

A method for training a speech recognition model with a loss function includes receiving an audio signal including a first segment corresponding to audio spoken by a first speaker, a second segment corresponding to audio spoken by a second speaker, and an overlapping region where the first segment overlaps the second segment. The overlapping region includes a known start time and a known end time. The method also includes generating a respective masked audio embedding for each of the first and second speakers. The method also includes applying a masking loss after the known end time to the respective masked audio embedding for the first speaker when the first speaker was speaking prior to the known start time, or applying the masking loss prior to the known start time when the first speaker was speaking after the known end time.

The present disclosure in some embodiments provides a dialogue state tracking method and a dialogue state tracking apparatus in a goal-oriented dialogue system, which track a dialogue state through training based on an attention mechanism between utterances and contextual semantic vectors corresponding respectively to domain-slot types and based on a distance metric-based non-parametric method and thereby facilitate service extension to a multi-domain scenario in the dialogue system.

A virtual object lip driving method performed by an electronic device includes: obtaining a speech segment and target face image data about a virtual object; and inputting the speech segment and the target face image data into a first target model to perform a first lip driving operation, so as to obtain first lip image data about the virtual object driven by the speech segment. The first target model is trained in accordance with a first model and a second model, the first model is a lip-speech synchronization discriminative model with respect to lip image data, and the second model is a lip-speech synchronization discriminative model with respect to a lip region in the lip image data.