A method of controlling a home appliance which operates in an Internet of Things environment through a 5G communication network and which is performed using a neural network model generated by machine learning, including determining whether there is a user in the vicinity of the home appliance, capturing a motion of the user using a vision sensor based on a determination that there is a user in the vicinity of the home appliance, identifying an intention of the user based on the captured motion, and activating a speech module of the home appliance based on the intention of the user.

Systems and methods for integration of speech processing functionality with organization systems are disclosed. For example, a voice interface application may be created to enable a voice interface functionality for devices associated with an organization. Space identifiers of spaces of the organization may be created and associated with the voice interface application. Devices associated with the space identifiers may be enabled for utilizing the voice interface application and may be set up utilizing wireless network identifiers associated with the spaces and/or the organization.

A computer-implemented input-method editor process includes receiving a request from a user for an application-independent input method editor having written and spoken input capabilities, identifying that the user is about to provide spoken input to the application-independent input method editor, and receiving a spoken input from the user. The spoken input corresponds to input to an application and is converted to text that represents the spoken input. The text is provided as input to the application.

An electronic device includes a processor configured to: receive a user voice input, identify a state of the electronic device corresponding to at least one item related to the electronic device, select a voice recognition engine corresponding to the identified state, from among a plurality of voice recognition engines, based on correlations between the plurality of voice recognition engines and a plurality of states, and perform an operation corresponding to the user voice input based on the selected voice recognition engine.

A method for recording a plurality of augmented reality (AR) sessions between a set of user(s) and an AR computer system, receiving first user input, through the AR computer system and from a first user, identifying a first AR session of the plurality of AR sessions, and presenting at least a portion of the recording of the first AR session on the AR computer system for the first user.

Arrangements for voice enabled event processing are provided. In some aspects, a self-service kiosk may detect a mobile device of a user and a connection may be established between the self-service kiosk and the mobile device. The user may request, via natural language data input, processing of an event, such as a transaction. The natural language data input may be captured by the mobile device of the user and transmitted to the self-service kiosk or other processing device. The natural language input may be processed to identify the requested event. Based on the processed natural language data, an event processing request may be generated. Based on processing the event, one or more event processing commands may be generated. The event processing commands may be executed to perform one or more functions associated with completion of the event processing (e.g., distributing funds, activating a deposit receptacle, or the like).

A remote control device for controlling devices in an environment can utilize an environment map and location information to accurately determine an intended device to provide control for multiple devices in an environment. The environment mapping can be performed using the remote control device including a plurality of sensors. A spatial map can be generated for an environment along with location information for controllable devices within the environment. The spatial map and location information can be stored on the remote control device. The mapping can allow the remote control device to quickly group devices or drag and drop content from one type of device to another type of device. The remote control device can perform search queries based on combinations of image and audio data in some examples.

Systems and processes for operating an intelligent automated assistant are provided. In accordance with one example, a method includes, at an electronic device with one or more processors, memory, and a plurality of microphones, sampling, at each of the plurality of microphones of the electronic device, an audio signal to obtain a plurality of audio signals; processing the plurality of audio signals to obtain a plurality of audio streams; and determining, based on the plurality of audio streams, whether any of the plurality of audio signals corresponds to a spoken trigger. The method further includes, in accordance with a determination that the plurality of audio signals corresponds to the spoken trigger, initiating a session of the digital assistant; and in accordance with a determination that the plurality of audio signals does not correspond to the spoken trigger, foregoing initiating a session of the digital assistant.

An electronic apparatus is provided. The electronic apparatus includes: a memory configured to store at least one instruction; and a processor configured to execute the at least one instruction to: obtain usage information on an application installed in the electronic apparatus, obtain a natural language understanding model, among a plurality of natural language understanding models, corresponding to the application based on the usage information, perform natural language understanding of a user voice input related to the application based on the natural language understanding model corresponding to the application, and perform an operation of the application based on the preformed natural language understanding.

An agent device includes an acquirer configured to acquire an utterance of a user of a first vehicle, and a first agent controller configured to perform processing for providing a service including causing an output device to output a response of voice in response to an utterance of the user of the first vehicle acquired by the acquirer. When there is a difference between a service which is utilized in the first vehicle and is available from one or more agent controllers including at least the first agent controller and a service which is utilized in a second vehicle and is available from one or more agent controllers, the first agent controller provides information on the difference.