A method of determining influence of language elements in script to an overall classification of the script by perturbing the dataset representing a conversation. In some instances, for example, in the conversation, the language elements and turns within the conversation (e.g., in a chat bot) are analyzed for their influence in escalation or non-escalation of the conversation to a higher level of resolution, e.g., to a human representative or manager.

An automated natural dialogue system provides a combination of structure and flexibility to allow for ease of annotation of dialogues as well as learning and expanding the capabilities of the dialogue system based on natural language interactions.

It is an aspect of the present disclosure to provide a dialogue system capable of providing an extended function to the user by registering a new vocabulary that matches the user's preference and by changing the pre-stored conversation pattern.

A personal assistant operation is provided for teaching a personal assistant device names preferred by the user for sensor activated devices. For this purpose, a method includes the personal assistant device receiving a request from a user to activate a requested device which the user has identified with a requested name which is unrecognized by the personal assistant device, determining a most likely candidate device from a list of candidate devices to activate in response to the request, activating the determined most likely candidate device, and identifying and saving the requested name as the name of the most likely candidate device in response to receiving confirmation from the user that the determined most likely candidate device is the requested device.

Examples of multi-user configuration are disclosed. An example method includes, at an electronic device: receiving a request; and in response to the request: if the voice input does not match a voice profile associated with an account associated with the electronic device: causing output of first information based on the request using a first account associated with the electronic device; if a setting of the electronic device has a first state, causing update of account data of the first account based on the request; and if the setting has a second state, forgoing causing update of the account data; and if the voice input matches a voice profile associated with an account associated with the electronic device: causing output of the first information using the account associated with the matching voice profile; and causing update of account data of the account based on the request.

Systems described herein may use machine classifiers to perform a variety of natural language understanding tasks including, but not limited to multi-turn dialogue generation. Machine classifiers in accordance with aspects of the disclosure may model multi-turn dialogue as a one-to-many prediction task. The machine classifier may be trained using adversarial bootstrapping between a generator and a discriminator with multi-turn capabilities. The machine classifiers may be trained in both auto-regressive and traditional teacher-forcing modes, with the maximum likelihood loss of the auto-regressive outputs being weighted by the score from a metric-based discriminator model. The discriminators input may include a mixture of ground truth labels, the teacher-forcing outputs of the generator, and/or negative examples from the dataset. This mixture of input may allow for richer feedback on the autoregressive outputs of the generator. Additionally, dual sampling may improve response relevance and coherence by overcoming the problem of exposure bias.

A method and apparatus for controlling learning of a model for estimating an intention of an input utterance is disclosed. A method of controlling learning of a model for estimating an intention of an input utterance among a plurality of intentions includes providing a first index corresponding to the number of registered utterances for each intention, providing a second index corresponding to a learning level for each intention, providing a learning target setting interface such that at least one intention that is to be a learning target is selected from among the intentions based on the first index and the second index, and training the model based on the registered utterances for each intention and setting of the learning target for each intention.

An automated natural dialogue system provides a combination of structure and flexibility to allow for ease of annotation of dialogues as well as learning and expanding the capabilities of the dialogue system based on natural language interactions.

Techniques disclosed herein relate to generating a hierarchical classification model that includes a plurality of classification models. The hierarchical classification model is configured to classify an input into a class in a plurality of classes and includes a tree structure. The tree structure includes leaf nodes and non-leaf nodes. Each non-leaf node has two child nodes associated with two respective sets of classes in the plurality of classes, where a difference between numbers of classes in the two sets of classes is zero or one. Each leaf node is associated with at least two but fewer than a first threshold number of classes. Each of the leaf nodes and non-leaf nodes is associated with a classification model in the plurality of classification models of the hierarchical classification model. The classification model associated with each respective node in the tree structure can be trained independently.

It is an aspect of the present disclosure to provide a dialogue system capable of providing an extended function to the user by registering a new vocabulary that matches the user's preference and by changing the pre-stored conversation pattern.