Systems and methods for language capability evaluation of large language models are provided. A communication platform accesses a pair of parallel inputs, including a reference input in a reference language corresponding to a target input in a target language different from the reference language. The communication platform executes a reference large language model for the generative task to obtain a reference output in the reference language based on the reference input. The communication platform executes a target large language model for the generative task to obtain a target output in the target language based on the target input. The communication platform evaluates a cross-lingual similarity between the target output and the reference output to obtain an evaluation score. The communication platform fine-tunes the target large language model based on the evaluation score using a reinforcement learning algorithm.

One example method includes receiving, by an artificial intelligence (AI) assistant, a user query comprising one or more tasks; determining one or more services based on the user query; obtaining, for each of the one or more services, a plurality of examples, each example providing an example command suitable for execution by the respective service; generating one or more instructions based on the user query, the one or more services, and the one or more pluralities of examples; providing the one or more instructions to a trained large language model (LLM); receiving, from the LLM, one or more commands corresponding to the user query; for each command of the plurality of commands, issuing the respective command to a corresponding service of the one or more services; generating a response to the user query based on results of the plurality of commands; and outputting the response

A determination device according to one aspect according to the present disclosure includes an input unit that receives an input of reaction information indicating a reaction of a user, a generation unit that generates first persona information that is information generated on the basis of the reaction information received by the input unit, the information indicating a characteristic of the user, a determination unit that determines consistency between the first persona information generated by the generation unit and second persona information based on past reaction information of the user, and an update unit that updates the persona information regarding the user on the basis of the consistency determined by the determination unit.

Systems and processes for delegating tasks to electronic devices based on intents and associated applications are provided. For example, receiving an input from a user at an electronic device, wherein the input is associated with an intent and determining, based on the intent, an intent type. In accordance with a determination that the intent type is a first type, causing a digital assistant to perform an action associated with the intent, and in accordance with a determination that the intent type is a second type, different than the first type, transmitting instructions to a model remote to the electronic device, wherein the instructions cause the model to perform an action associated with the intent.

Embodiments of the disclosure provide a method, apparatus, device and storage medium for querying. The method includes: determining, in a conversational interaction application, whether a query request from a user relates to a search requirement in response to receiving the query request; obtaining at least one search result matching the query request with a search system in response to determining that the query request relates to a search requirement; and presenting a response for the query request, the response being generated based on the at least one search result. In this way, in accordance with a determination that the query request relates to the search requirement, the search result is determined by the search system, and then the response to the query request is determined based on the search result. This helps to improve the accuracy and efficiency of the query.

Embodiments of the disclosure provide a method, an apparatus, a device and a storage medium for information extraction. The method includes: determining, based on a user input indicating information extraction, a target content and a target structured data object; obtaining structured information of the target structured data object, the structured information indicating at least one field comprised in the target structured data object; determining, based on the target content and the structured information, at least one data item from the target content, the data item corresponding to one or more fields in the at least one field; and adding the at least one data item to corresponding one or more fields in the target structured data object, respectively. Thereby, it is possible to help a user in more efficiently organizing the information in the target content into various carriers.

Systems and methods are disclosed herein for compressing a prompt. In an example system, an importance score listing is obtained that includes a score indicative of an importance of a plurality of dataset keywords. From the importance score listing, a keyword importance score is identified for a plurality of keywords in a current text fragment, such as a text fragment to be compressed. A set of placeholders in an abstract prompt template is populated based on the current text fragment. The current text fragment is compressed based on the importance of the plurality of keywords in the current text fragment to generate a compressed text fragment. In an example, the compressed text fragment is included in the prompt for transmission to a computing entity, such as a large language model of a generative question-answering system.

The technical solutions described herein present a computing action search using natural language processing. A system can identify a request containing an executable action associated with a first account identifier of a client system and select a prompt that corresponds to the action, is structured as text including fields, and identifies compatible actions corresponding to the client system or the first account identifier. The system can embed content, including text or metadata, of the first account identifier into the fields of the prompt. The system can provide the prompt to a model and obtain a response from the model indicating a recommended action and a second account identifier associated with the recommended action. The system can validate that the recommended action corresponds to the compatible actions, and the second account identifier corresponds to the first account identifier and execute, responsive to validation, the recommended action for the first account identifier.

Techniques for grounding automatically-generated responses produced by a question-and-answer system are provided. In one technique, a list of items and introductory text that is associated with the list of items are identified within text data. For each item in the list of items, a claim that is based on the introductory text and said each item is generated and the claim is added to a set of claims that is associated with the text data. For each claim in the set of claims, a score that reflects a level of support of said each claim in a set of documents is generated and the score is added to a set of scores for the set of claims. Data that is based on the set of scores is presented on a screen of a computing device.

A terminal includes an acquiring means and a control means. The acquiring means acquires an operation history of each of a plurality of applications installed in own apparatus. The control means generates a prompt to be input into a language model using the operation history of an application selected by a user from among the plurality of applications as the application for which the user requests an explanation of a method of using. The control means presents to the user an explanatory text related to a method of using the selected application, the explanatory text being acquired from the language model using the generated prompt.