Evaluating and improving information retrieval in question-answering systems is an area of importance in machine learning growth. Retrieval components in a retrieval-augmented generation (RAG) question answering system enable machine learning models to provide more accurate and reliable answers to questions. Systems for retriever evaluation involve processing queries in comparison to reference documents. The system first retrieves documents deemed relevant, then generates a first answer based on them. A second answer is generated using a set of documents that includes ground truth documents known to be relevant to the query. By analyzing semantic overlap between these responses, a quantitative evaluation of the retrieval component is obtained. This evaluation then informs automatic modifications to retrieval parameters, enhancing future document selection and response accuracy.

A conversation content generation method and apparatus, a storage medium and a terminal are provided. The method includes: acquiring a current utterance entered by a user; reading a preset topic transfer graph and target topic, wherein the topic transfer graph includes nodes and connecting lines between the nodes, the nodes correspond to topics in one-to-one correspondence, each connecting line points from a first node to a second node, a weight of the connecting line indicates probability of transferring from a topic corresponding to the first node to a topic corresponding to the second node, and the topic transfer graph includes a node corresponding to the target topic; determining a topic of reply content of the current utterance at least based on the current utterance, the topic transfer graph and the target topic, and recording it as a reply topic; generating the reply content at least based on the reply topic.

An intelligent flow agent system comprising: a processor and a memory element, the memory element comprising a non-transitory computer-readable medium storing instructions that, when executed by the processor, cause the system to: receive input from an actor, the input comprising at least one of an event, a task, or a mission; embed contextual information into the input via a contextual unit, the contextual information including at least one of a system state, environmental conditions, user behavior, or historical interactions; construct the mission based on the received input and the embedded contextual information; evaluate the mission using the intelligent flow agent to determine one or more workflows or actions suitable for execution, wherein the evaluation includes a context-aware decision process to select, sequence, or delegate actions based on at least one of the contextual relevance, system policies, or optimization criteria; and initiate an intelligent workflow comprising dynamically adaptive and coordinated actions performed by one or more intelligent flow agents to fulfill the mission.

A method for AI-driven natural language search includes receiving a user query for one or more items from a user, processing the user query by searching against at least one relational database associated with the query, where the relational database is generated by extracting features from electronic documents of a plurality of items associated with the one or more items and by identifying specifications or respective values corresponding to the extracted features of the plurality of items, generating one or more query results based on the processing of the user query, where the one or more results include at least one item identified from the plurality of items and a justification for explaining an irrelevance of the at least one item, and transmitting the one or more query results to a user device for presentation to the user.

Methods and systems are disclosed for implementing a Large Language Model utilizing a prompt attention-processing subsystem and a generation attention-processing subsystem. A sequence of tokens is first processed by a prompt attention-processing subsystem, which utilizes an associated prompt KV-cache to store matrix values generated during prompt attention-processing. Upon the completion of prompt attention-processing, the populated prompt KV-cache is transferred to a generation KV-cache for processing by the generation attention-processing subsystem. The prompt and generation attention-processing subsystem can be multi-headed. The separate processing of the prompt facilitates efficient computations. Further, the prompt can be processed in segments that match available memory and computational resources. The generation attention-processing subsystem then produces an output token sequence based on the prompt KV-cache values transferred to the generation attention-processing system. The described system ensures optimized processor and memory usage and streamlined processing for large language model systems.

Examples relate to systems and methods for restoring threads including context of the threads outside of a chat interface. During a thread including multiple queries and responses, one or more of the responses may include links to web pages and/or to other applications (e.g., presentation applications, word-processing applications). During interactions with the thread, one or more of the links may be selected. The selection of the links causes the corresponding web pages to be loaded and/or the corresponding applications to be launched. The web pages that are opened and/or the applications that are launched during an ongoing thread are stored as thread data for the ongoing thread. Then, when the thread is resumed at a later time, not only is the chat interface populated with the prior queries and responses of the thread, but the web pages and/or applications are also restored.

A system performs assessment of users based on a simulated meeting. The system stores video segments in a database. The system retrieves an execution plan for a simulated interaction with a user. The execution plan comprises instructions for a plurality of video interactions. Each video interaction comprises either displaying one or more pre-recorded video segments selected from a plurality of pre-recorded video segments or a live video stream of the user. The system repeatedly performs the following steps according to the execution plan of the simulated interaction. The system performs a sequence of video interactions. A video interaction may comprise sending a set of pre-recorded video segments for display via a user interface. In response to the sequence of video interactions, the system performs a second video interaction by recording a live video stream of the user. The system analyzes the simulated interaction to evaluate the user.

A system uses a machine learning based language model for performing assessments of users. The system stores media objects comprising text data, video data, or audio data. The system retrieves an execution plan for a simulated interaction. The execution plan identifies a sequence of stored media objects for presentation to a user for performing the simulated interaction with the user. The system performs interactions with a user via one or more channels in accordance with the execution. The system generates prompts for a trained neural network, for example, a machine learning based language model to evaluate responses received from the user. The system sends the prompts to a trained neural network and receives responses generated by executing the trained neural network. The system determines metrics for evaluating the user based on the response received from the trained neural network and takes actions based on the metrics.

Methods, systems, and storage media for refining search queries through interactive conversational exchange are disclosed. Exemplary implementations may receive a first search input from a user describing a desired asset such as an image. Aspects implementations may also interact with the user via a chat interface to solicit additional details about the desired asset in response to the first search input, construct a search query based on the first search input and the additional details solicited from the user, and submit the search query to a search service to retrieve relevant asset data for the user.

A computer-implemented method of automatically generating interactive compliance controls by a server computer system to a client computing system is provided. The method includes receiving, by the server computer system, a first input from the client computing system. The first input provides an electronic rules document including a plurality of compliance rules or identifying information for the electronic rules document, and information related to an asset. The method also includes outputting, by the server computer system to the client computing system and in response to the first input, controls corresponding to the compliance rules. The controls being rephrasings of the compliance rules and generated by inputting the electronic document into a first large language model (LLM). The first LLM being pretrained by examples specifying acceptable and unacceptable control outputs for a plurality of compliance rule inputs.