Methods and systems for providing computer-implemented services using generative AI models are disclosed. To do so, a prompt may be obtained for a generative AI model of the generative AI models. The prompt and the generative AI model may be used to obtain a first output. An output explainability process may be performed using at least the prompt, the first output, and the generative AI model to obtain an explainability report for the first output. The explainability report may indicate relationships between first information elements of the prompt and second information elements of the first output. The relationships may be based, at least in part, on a second output generated by the generative AI model using a modified prompt based on the prompt. The first output and the explainability report may be provided to a downstream consumer as part of providing computer-implemented services to the downstream consumer.

System and method for query augmentation for generating responses is disclosed. The method includes, receiving an input data from a user device, determining a context of the received input data, determining a domain specific graphical knowledge schema corresponding to the received input data, and identifying a plurality of missing entities by analyzing the determined context and at least one graphical instance corresponding to the determined appropriate domain specific graphical knowledge schema. The method further includes, prioritizing the identified plurality of missing entities, generating at least one sub-query for each of the identified plurality of missing entities and retrieving a relevant content corresponding to the generated at least one sub-query using a RAG-based system. The method further includes, generating at least one response to the received input data by augmenting the received input data with the retrieved relevant content, and outputting the generated at least one response on a user interface of the user device and updating the at least one graphical instance with the retrieved relevant content.

Query response generation using a large language model based on structured data and unstructured data (e.g., using a computerized tool), is enabled. For example, a system can comprise at least one processor, and at least one memory that stores executable instructions that, when executed by the at least one processor, facilitate performance of operations. The operations can comprise updating a metadata repository, wherein the metadata repository comprises first metadata representative of structured data of a data system and second metadata representative of unstructured data of the data system, based on the metadata repository, updating a large language model (LLM), wherein updating the LLM comprises retraining the LLM, in response to receiving a query, determining, using the first metadata representative of structured data and the second metadata representative of unstructured data of the data system, data, from the data system, applicable to the query, and generating a response to the query.

A large language model (LLM) interaction method and system are provided. A conversation input by a user is acquired. Retrieval and summarization are performed to obtain an answer retrieval result. Based on a conversation and a current user intent, marketing placement information is retrieved from a marketing database and summarized to obtain a marketing placement retrieval result. Comprehensive sorting is performed on cited sources of the answer retrieval result and the marketing placement retrieval result to obtain a sorted result, which is returned to the user as a cited source result. Summarization and integration are performed on the cited source result and the marketing placement retrieval result to obtain an answer result. The answer result is returned to the user. The user request, the answer result and the marketing retrieval result are integrated to obtain a recommended related question, which is returned to the user to guide further multi-turn dialogues.

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for digital content creation within website environments. In some implementations, a server receives request data from a client device, which specifies an action corresponding to a text segment displayed on the client device. The server identifies a content item that (i) corresponds to the text segment, and (ii) is structured according to a collection schema. The server generates prompt data for trained machine learning (ML) models, the prompt data includes a text generation instruction based on the content item. The server provides the prompt data to the trained ML models and obtains a candidate output in response. The server determines that the candidate output is valid. In response, the server generates graphical user interface (GUI) data for the client device to display the candidate output. The server transmits the GUI data to the client device.

The disclosure relates to methods of providing a response to a user query. A query is derived from the user query. An embedded query is obtained by passing the query through a first portion of a trained large language model. A semantically relevant element is obtained from an embedded database. The embedded database was obtained by embedding an initial database using the first portion of the trained large language model. The semantically relevant element is combined with the embedded query to form an augmented query. A response is provided to the user query by passing the augmented query through a second portion of the trained large language model.

Embodiments described herein provide a RAG framework including a question decomposition module to decompose an open-ended question and a classification module to evaluate whether a RAG LLM-generated answer accurately address each decomposed sub-question. Specifically, a question received from a user may be decomposed into subquestions using a neural network based language model. Then, each subquestion may be classified, e.g., as core, background, or follow-up. Text chunks may be retrieved based on the classifications and the subquestions, and a neural network based language model may generate a response to the user question based on the retrieved text chunks. Finally, a rating may be determined, where the rating is indicative of whether the response answers a subquestion in the plurality of subquestions. The rating may thus be used as feedback for a RAG LLM to revise and/or re-generate the answer to the user question.

An information processing apparatus for processing information on a dialog between a plurality of users, the apparatus includes processing circuitry configured to: acquire analysis data obtained by analyzing the dialog; and create input data to be input to a generative AI, based on the acquired analysis data.

Machine learning-based evaluation of recorded interactions is disclosed, including: obtaining an evaluation plan to correspond to a new question; retrieving a representative interaction based at least in part on the new question; using a reasoning and answer language model to evaluate the representative interaction against the new question based at least in part on the evaluation plan and to provide a preview evaluation result; outputting, at a user interface, the new question and the preview evaluation result of the representative interaction; receiving, via the user interface, user feedback to the preview evaluation result; updating the reasoning and answer language model based at least in part on the user feedback to the preview evaluation result; and storing a feedback data set including the user feedback.

Described herein are systems and methods for monitoring and evaluating language performance according to real-time data in a distributed networking environment. A system can receive, from a client device, a prompt for a communication session. The system can determine, based on the prompt, a classification of an intent corresponding to a first output type of multiple output types. A first language model of a plurality of language models can be selected based on the classification of the intent, the first language model being associated with a first intent type and selected in response to the classification matching the first intent type. The system can generate an output message using the first language model and the prompt, the output message comprising text data that is responsive to the prompt.