Provided are embodiments for a computer-implemented method for interacting with a user by an automated response system supporting topic switching and information collection. The computer-implemented method includes receiving a plurality of utterances from the user by the automated response system, and analyzing the utterances to form a first topic thread and an information collection objective. The computer-implemented method also includes utilizing an information collection user interface to gather data to support the information collection objective, and providing responses to the user after the gathered data related to the first topic thread. Also provided are embodiments for a system and computer program product for implementing the techniques described herein.

A computer process provide for users employing a conversation program to dynamically program automated assistants with information and processes that can later be invoked to accomplish task(s) on one or more of the users' devices. The conversation program might generate an automated assistant from a text or graphic source. The conversation program might access multiple automated assistants and determine which is most appropriate to use in order to address a user's request. The user can generate avatars for the conversation program that parallel human emotion, facial expression and body gestures that can be displayed in visual context. A resulting automated assistant can be used for software system testing.

Structured conversation enhancement can include determining an anticipated ebb point of a current conversation. The determination can be made in response to a predetermined triggering event indicating a start of the current conversation. Structured conversation enhancement also can include monitoring the current conversation using pattern recognition. A probable change in the anticipated ebb point can be determined in response to recognizing a predetermined word pattern indicating a change in the conversation. A response action can be initiated in response to the probable change in the anticipated ebb point.

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for speech recognition. One of the methods includes receiving first audio data corresponding to an utterance; obtaining a first transcription of the first audio data; receiving data indicating (i) a selection of one or more terms of the first transcription and (ii) one or more of replacement terms; determining that one or more of the replacement terms are classified as a correction of one or more of the selected terms; in response to determining that the one or more of the replacement terms are classified as a correction of the one or more of the selected terms, obtaining a first portion of the first audio data that corresponds to one or more terms of the first transcription; and using the first portion of the first audio data that is associated with the one or more terms of the first transcription to train an acoustic model for recognizing the one or more of the replacement terms.

A communication method for hearing impaired communication comprising: providing a speech training device to a hearing impaired user, the speech training device configured to teach the hearing impaired user how to determine non-speech sounds. The method further includes providing a haptic output device to a hearing impaired user where the haptic output device is configured to be relasably coupled to the hearing impaired user. The haptic output device receives, a sound input signal comprising a non-speech sound and provides the haptic output signal to an actuator which is in electrical communication with the haptic output device. The actuator actuates in response to the haptic output signal and provides a haptic sensation to the hearing impaired user.

Voice coaching system, voice coaching device, and related methods, in particular a method of operating a voice coaching system comprising a voice coaching device is disclosed, the method comprising obtaining audio data representative of one or more voices, the audio data including first audio data of a first voice; obtaining first voice data based on the first audio data; determining whether the first voice data satisfies a first training criterion; in accordance with determining that the first voice data satisfies the first training criterion, determining a first training session; outputting, via the interface of the voice coaching device, first training information indicative of the first training session.

An automatic agent may be improved through feedback. A user input may be received through a user interface. A plurality of current utterance variables may be obtained by tokenizing the user input. The automatic agent may execute a machine learning policy to generate a reply to the user input based on the plurality of current utterance variables. A different reply may be obtained in response to an indication that the reply will lead to a breakdown, wherein the breakdown comprises an unhuman response from the automatic agent according to the machine learning policy. The machine learning policy may be adjusted based on the plurality of current utterance variables and the different reply.

An electronic device is provided. The electronic device includes a processor and a memory operatively connected to the processor. The memory may store instructions that, when executed, cause the processor to receive a voice input of a user, to extract a feature from the voice input of the user, to select an acoustic model through comparison with the extracted feature, and to learn the feature of the voice input by performing fine-tuning on the selected acoustic model.

A method may include obtaining a dialogue of a user and a pre-trained language model. The method may include obtaining a corpus of dialogues and a corpus of response materials. The method may include modifying the pre-trained language model. The method may include identifying a dialogue topic of the dialogue of the user and identifying a set of response topics. The method may include selecting a set of response materials from the corpus of response materials. The method may include determining a first plurality of probabilities and, for each response material of the set of response materials, a respective second plurality of probabilities. The method may include comparing the first plurality of words with each respective second plurality of words associated with each respective response material of the set of response materials. The method may include selecting a response material of the set of response materials based on the comparison.

Methods, systems, and computer program products for training an artificial intelligence (AI) of a voice response system. Aspects include receiving, by the voice response system from a user, a voice command to perform a requested action and interpreting, by an AI model, the voice command. Aspects also include performing an action based on the interpretation of the voice command and receiving non-verbal feedback from the user. Aspects further include updating the AI model based on a determination that the non-verbal feedback indicates that the user is not satisfied with the action performed.