GENERATING PERSONALIZED VIDEO RESPONSES FOR A USER USING INTEGRATED PROGRAMMATIC AND SPECIALIZED GUIDED AND CONSTRAINED ARTIFICIAL INTELLIGENCE
20260017306 ยท 2026-01-15
Assignee
Inventors
Cpc classification
G10L13/027
PHYSICS
International classification
Abstract
A system and method combine programmatic control and a guided and constrained Artificial Intelligence (AI) engine to generate personalized video responses for a user. Receiving user inputs to identify a historical figure and a question directed to the corresponding historical figure. Information about the historical figure, including image, voice identifier, and biographical profile, is retrieved. Under programmatic logic control, an AI engine analyzes a question and generates an excuse for any delay in responding, else generate a dialogue response. The dialogue response is integrated with relevant information to generate a response having an audio and visual component with the voice identifier and dialogue response, and the image of the selected historical figure. A prompt is generated to guide and constrain the AI engine to create a video response that integrating audio and visual components to simulate the historical figure delivering the dialogue response and providing to an online learning platform.
Claims
1. A method for guiding and constraining an Artificial Intelligence (AI) engine to generate personalized video responses for a user comprising: executing code using one or more processors of a computer system to cause the computer system to perform operations comprising: receiving a user input from the user, wherein the user input identifies a historical figure from a predefined list of historical figures; receiving a question input from the user, wherein the question input includes a query directed to the selected historical figure; retrieving information relevant to the selected historical figure, wherein the relevant information comprises an image, a voice identifier, and a biographical profile of the selected historical figure; utilizing a language learning model (LLM) to analyze the question input, wherein the LLM generates: an excuse for any anticipated delay in responding to the question input; a dialogue response formulated based on the biographical profile and historical context of the selected historical figure; integrating the dialogue response with the relevant information retrieved for the selected historical figure to generate response, wherein the response includes: an audio component incorporating the voice identifier and the dialogue response; a visual component incorporating the image of the selected historical figure; generating a prompt to guide and constrain the AI engine for generating a video response using a video generator, wherein the video response integrates the audio component and visual component to simulate the selected historical figure to deliver the dialogue response; and transferring the prompt to the AI engine to provide the generated video response to the user on a user interface of an online learning platform.
2. The method of claim 1 wherein retrieving information relevant to the selected historical figure from a database, the database includes metadata categorizing the historical figures by historical period, geographical location, and achievements.
3. The method of claim 1 wherein the user interface provides the user to search historical figures from the database based on the name of the historical figures, time periods, or achievements.
4. The method of claim 1 further comprising: a natural language processing model configured to parse the question input to identify the subject matter and context to enhance the relevance of the generated dialogue response.
5. The method of claim 1 wherein the LLM is trained on historical texts, speeches, and writings to ensure the generated dialogue response reflects the authentic speech patterns and perspectives of the selected historical figure.
6. The method of claim 1 wherein the video generator utilizes AI video synthesis techniques to animate the image of the historical figure, synchronizing lip movements with the generated dialogue response to create the realistic video response.
7. The method of claim 1 wherein the dialogue response generated by the LLM includes educational content designed to align with historical facts and provide informative answers to the user based on the question input.
8. The method of claim 1 wherein the generated video response is configured to cover topics within education standards associated with the user to assist the user in mastering the knowledge required on the online learning platform.
9. A system for guiding and constraining an Artificial Intelligence (AI) engine to generate personalized video responses for a user comprising: one or more processors; memory, coupled to the one or more processors, storing code that when executed cause the one or more processors to perform operations comprising: executing code using one or more processors of a computer system to cause the computer system to perform operations comprising: receiving a user input from the user, wherein the user input identifies a historical figure from a predefined list of historical figures; receiving a question input from the user, wherein the question input includes a query directed to the selected historical figure; retrieving information relevant to the selected historical figure, wherein the relevant information comprises an image, a voice identifier, and a biographical profile of the selected historical figure; utilizing a language learning model (LLM) to analyze the question input, wherein the LLM generates: an excuse for any anticipated delay in responding to the question input; a dialogue response formulated based on the biographical profile and historical context of the selected historical figure; integrating the dialogue response with the relevant information retrieved for the selected historical figure to generate response, wherein the response includes: an audio component incorporating the voice identifier and the dialogue response; a visual component incorporating the image of the selected historical figure; generating a prompt to guide and constrain the AI engine for generating a video response using a video generator, wherein the video response integrates the audio component and visual component to simulate the selected historical figure to deliver the dialogue response; and transferring the prompt to the AI engine to provide the generated video response to the user on a user interface of an online learning platform.
10. The system of claim 9 wherein retrieving information relevant to the selected historical figure from a database, the database includes metadata categorizing the historical figures by historical period, geographical location, and achievements.
11. The system of claim 9 wherein the user interface provides the user to search historical figures from the database based on the name of the historical figures, time periods, or achievements.
12. The system of claim 9 further comprising: a natural language processing model configured to parse the question input to identify the subject matter and context to enhance the relevance of the generated dialogue response.
13. The system of claim 9 wherein the LLM is trained on historical texts, speeches, and writings to ensure the generated dialogue response reflects the authentic speech patterns and perspectives of the selected historical figure.
14. The system of claim 9 wherein the video generator utilizes AI video synthesis techniques to animate the image of the historical figure, synchronizing lip movements with the generated dialogue response to create the realistic video response.
15. The system of claim 9 wherein the dialogue response generated by the LLM includes educational content designed to align with historical facts and provide informative answers to the user based on the question input.
16.
17. The system of claim 9 wherein the generated video response is configured to cover topics within education standards associated with the user to assist the user in mastering the knowledge required on the online learning platform.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
[0009] The systems and methods described herein may be better understood, and their numerous objects, features, and advantages are made apparent to those skilled in the art by referencing exemplary embodiments depicted in the accompanying figures. The use of the same reference number throughout the several figures designates a like or similar element.
[0010]
[0011]
[0012]
[0013]
[0014]
[0015]
[0016]
[0017]
[0018]
[0019]
[0020]
[0021]
DETAILED DESCRIPTION
[0022] The video response generation system and method set forth herein address technical issues with generating personalized video response described herein. Conventionally, manual processes were used to generate the personalized video response and were very tedious and time consuming. The present video response generation system and method utilize an automated system that does not merely automate a manual process or use a conventional system in a conventional way. The present video response generation system and method utilize one or more artificial intelligence (AI) engines and integrate programmatic process management to technologically guide and constrain the one or more AI engines to produce the desired outputs in a completely different way than both any manual process and different than normal use of programs and AI engines. Utilizing specially engineered guidance and control to direct an AI system in solving the technical problems presented below, which require a technical solution. The video response generation system and method described below are not simply engaging a computer to carry out conventional mental processes, but rather change how computers (and AI systems, specifically) operate to achieve the generation results that were not previously possible or were substantially inefficient prior to the video response generation system and method set forth below. The AI system needs specific technical guidance, control, and constraints to achieve results that are not otherwise achievable.
[0023] Prompts are used to guide and constrain each AI engine. The prompts guide each AI engine by steering the AI engine(s). Guiding an AI engine refers to providing the AI engine with a general direction or framework to shape the AI engine's behavior or decision-making process. Guiding sets goals or principles. Guiding allows the AI engine some flexibility to interpret and adapt, much like giving it a compass to navigate rather than a fixed path.
[0024] Constraining each AI engine includes imposing specific, hard limits or rules on what each AI engine can do. Constraining an AI engine can also include providing specific input data to not only guide but also constrain the scope of each AI engine's reasoning basis and response. Constraining each AI engine assists with aligning the AI engine(s) for its (their) intended use.
[0025] Normally AI engines are provided a single user prompt requesting the AI engine, such as OpenAl's ChatGPT and its various implementations such as Anthropic's Claude Sonnet, to perform a task and produce an output. However, this conventional AI engine prompting method has a variety of technical shortcomings. Without proper guidance and constraints, an AI engine will not produce the desired output specified as produced by the video response generation system and method described herein. Instead, the AI engine will produce many unusable outputs that are unusable for a variety of reasons including so-called hallucinations where the AI engine presents fabricated information, duplicate outputs, too few outputs, too many outputs, outputs that do not meet desired criteria, and so on. Without special technical guidance, the AI engine cannot reliably be applied to generate desired outcomes.
[0026] The video response generation system and method generate decomposed, technically engineered AI prompts to include selected and integral AI engine guidance and constraints. The technically engineered prompts are generated and guided with programmatic, automatic inputs specifically designed to unconventionally guide and constrain an AI engine to produce personalized video response, perform quality control to retain or automatically discard outputs that do not meet guidance and constraints, and make the personalized video response available for use, such as use by computer system applications. In at least one embodiment, the problem to be solved by the integrated programmatic and AI engine video response generation system and method is uniquely and unconventionally decomposed, and AI prompts are used to solve the decomposed problem. Furthermore, the programmatic inputs to the decomposed AI prompts provide guidance to generate the personalized video response
[0027] Determining a number of prompts, the guidance and constraints within each prompt, and data flowing from one AI engine prompt to another, in addition to testing a number of prompts for the decomposed problem, testing within each prompt, and validating a desired quality of outputs becomes an intractable combinatorial problem without technical guidance and constraint of the video response generation system and method described herein. Thus, the present video response generation system and method described implement an integration of programmatic management over decomposed prompts with engineered AI engine guidance and constraints to affect an improvement in AI, programmatic AI management, and AI integrated with programmatic management technology. The present video response generation system and method allow computer systems to include programmatic management, one or more AI engines, and one or more data sources to produce the personalized video response that previously could not be produced with conventionally prompted AI engines or could only be produced by humans utilizing a completely different, time consuming, and tedious process. The video response generation system and method improve conventional methods through the use of a programmatic AI engine management system to generate decomposed, technically engineered AI prompts to include selected and integral AI engine guidance and constraints. It is, for example, the incorporation of the programmatic AI engine management system to generate decomposed, technically engineered AI prompts to include generated, integral, and unconventional AI engine guidance and constraints and execution by the one or more AI engines to provide useful results that improve existing technical processes, which is not an automation of a conventional process.
[0028] Programmatic components and AI engines generally utilize one or more processors that have access to memory, which may include one or more storage components, to execute and perform functions. An AI engine is a core hardware and software system that enables artificial intelligence applications to process data, learn patterns, and generate insights or actions. It functions as the brain behind AI-driven systems, facilitating tasks such as machine learning, natural language processing, and decision-making. Exemplary components of an AI engine are: [0029] 1. Machine Learning Models-Algorithms that analyze data, recognize patterns, and make predictions. [0030] 2. Neural Networks-Deep learning architectures that mimic the human brain for tasks like image and speech recognition. [0031] 3. Data Processing Module-Handles raw data input, transformation, and feature extraction. [0032] 4. Inference Engine-Applies trained models to make real-time decisions based on new data. [0033] 5. Optimization Algorithms-Improves model efficiency, reducing errors and improving predictions. [0034] 6. Natural Language Processing (NLP) Module-Enables AI engines to understand, interpret, and generate human language (e.g., chatbots, voice assistants). [0035] 7. Computer Vision Module-Allows AI to interpret and analyze images or videos. [0036] 8. Reinforcement Learning Mechanism-Helps AI learn from trial and error, optimizing performance over time. [0037] 9. API Interface-Connects the AI engine with applications, enabling integration with other software or platforms.
[0038] Examples of AI Engines include: XAI's Grok and variations thereof, Google TensorFlow, Meta's PyTorch, Microsoft Azure AI, OpenAI's ChatGPT and variations thereof, IBM Watson, OpenAI Whisper, Google BERT & T5, Amazon Lex, Anthropic Claude, DeepMind's AlphaCode, Google Vision AI, Meta's DINO & SAM (Segment Anything Model), NVIDIA DeepStream. OpenCV AI Kit, Amazon Polly. Google WaveNet, Deepgram.
[0039] Notwithstanding any provision to the contrary or anything to the contrary in the below pages, the below pages are not limiting and do not describe all embodiments of the video response generation systems and methods. For example, use of the term invention does not limit or require the referenced certain features to be present in all embodiments of the invention. Use of absolute-type terms, such as required, must, only, important, and so on are not limiting of all embodiments of the video response generation systems and methods and not to be construed as limiting of the embodiments of the video response generation systems and methods described above.
[0040] The system and method for guiding an Artificial Intelligence (AI) engine to generate personalized video responses for a user. In at least one embodiment, the video response generation process involves receiving user inputs to identify a historical figure and a question directed to the corresponding historical figure. Typically, the relevant information about the historical figure, including an image, a voice identifier, and a biographical profile, is retrieved. A language learning model (LLM) is used to analyze the question input and generate an excuse for any anticipated delay in responding, along with generation of a dialogue response based on the historical context of the selected historical figure. The dialogue response is integrated with the relevant information to generate a response comprising an audio component with the voice identifier and dialogue response, as well as a visual component with the image of the selected historical figure. A prompt is generated to guide the AI engine in creating a video response that integrates the audio and visual components to simulate the historical figure delivering the dialogue response. The video response is then provided to the user on an online learning platform.
[0041] The video response generation process also includes retrieving information relevant to the historical figure from a database, to categorize historical figures by historical period, geographical location, and achievements. The user interface allows users to search historical figures based on their names, time periods, or achievements. Additionally, a natural language processing model is used to parse the question input and enhance the relevance of the generated dialogue response. The LLM is trained on historical texts, speeches, and writings to ensure that the generated dialogue response reflects the authentic speech patterns and perspectives of the selected historical figure.
[0042] Furthermore, the video generator utilizes AI video synthesis techniques to animate the image of the historical figure, synchronizing lip movements with the generated dialogue response to create a realistic video response. The generated dialogue response also includes educational content designed to align with historical facts, providing informative answers to the user based on the question input. Moreover, the generated video response is configured to cover educational standards associated with the user to assist the user in mastering the knowledge.
[0043]
[0044] The Artificial Intelligence (AI) engine 106 is designed to generate personalized video responses 102 for the user 104. The AI engine 106 receives a user input 108 and a question input 110 from the user 104. The AI engine 106 is configured to utilize the received user input 108 and question input 110 to generate the personalized video response 102. Moreover, to generate a dialogue response 112 a language learning model (LLM) 114 is used to analyze the question input 110. Based on the dialogue response 112 the AI engine 106 generates the video response 102. Typically, the AI engine 106 utilizes a video generator 116 to generate the video response 102.
[0045] Referring to
[0046] The predefined list of historical figures allows the selection of the historical figure from a diverse range of the historical figures. The predefined list of historical figures is curated based on the consideration of historical impact. Once the user input 108 is received, the video response generation system 100 processes the user input 108 to identify the chosen historical figure by matching the user's selection with the entries in the predefined list, confirming that the user input 108 corresponds to a valid option. In at least one embodiment, the selection process can be enhanced with additional features such as search filters, categorization by era, region, or field of influence, and recommendations based on user behavior or interests to provide a more tailored and efficient way for the user 104 to select the historical figures.
[0047] In operation 204, the question input 110 is received from the user 104. The question input 110 includes a query directed to the selected historical figure. Typically, receiving the question input 110 involves capturing the query of the user 104 and processing the query to identify the relevance and context. The query ranges from simple questions, such as When was the selected historical figure born? to a complex question, like What impact did the selected historical figure have on contemporary society? Once the question input 110 is processed, the relevant information corresponding to the selected historical figure is generated as a response. In at least one embodiment, the information corresponding to the selected historical figure is drawn from a database containing detailed biographies, historical records, academic articles, and other authoritative sources.
[0048] In at least one embodiment, analyzing patterns in the questions asked by the user 104, helps in identifying areas of interest or knowledge gaps of the user 104, thereby enabling the video response generation system 100 to develop new content or features corresponding to the user 104 interest. The ability to ask questions about historical figures and receive detailed answers enhances learning outcomes. The generation of the video response 102 corresponding to the question input 110 from the user 104 promotes critical thinking and analytical skills and allows the user 102 to encourage exploring deeper aspects of historical events and personalities.
[0049] In operation 206, information relevant to the selected historical figure is retrieved. The relevant information comprises an image, a voice identifier, and a biographical profile of the selected historical figure. Typically, once the user 102 identifies the historical figure of interest and provides the question input 110 corresponding to the historical figure. The relevant information corresponds to the historical figure. The retrieval process of the image of the historical figure is started. The image creates a visual connection between the user 102 and the historical figure. The image is sourced from archives, museums, or historical databases that provide accurate depictions of the historical figures. The selected image must be historically accurate and capture the essence, appearance, and era of the historical figures. Based on the selected image the voice identifier identified the suitable voice for the historical figure. The voice identifier utilizes a recorded speech, an audio excerpt from historical broadcasts, or a synthesized voice that mimics the known vocal characteristics of the historical figure based on descriptions or existing audio records. Typically, if original audio recordings of the historical figure exist, then the original audio recordings are used as the primary source. In cases where no recordings are available, voice synthesis technology can be employed to create a plausible representation based on historical accounts of the historical figure's speech patterns, accent, and intonation. The biographical profile of the historical figure is retrieved to identify a comprehensive overview of the life, achievements, and impact of the historical figure. The biographical profile includes information such as birth and death dates, major life events, significant contributions, and the historical context of the historical figure's actions. The biographical profile is crafted to be both informative and engaging, presenting the information in a narrative that captures the personality and significance of the historical figure.
[0050] In operation 208, a language learning model (LLM) 114 is utilized to analyze the question input 110 for appropriateness to generate a response. The LLM 114 generates an excuse for any anticipated delay in responding to the question input 110 or the dialogue response 112 based on the biographical profile and historical context of the selected historical figure. Typically, the LLM 114 evaluates the input question 110 to ensure alignment with the input question 110 within the scope of the selected historical figure. The analysis includes filtering out inappropriate content, such as offensive language or irrelevant queries, and ensuring the input question 110 is meaningful within the historical context to provide the user 102 with valuable and relevant information. An exemplary LLM 114 is GPT 4, to analyze the question input 110 for appropriateness and generate the response, available from OpenAI having an office in San Francisco, CA.
[0051] Upon determining the appropriateness of the question, the LLM 114 prepares to generate a response. However, if the LLM 114 identifies any delay in delivering the response due to the complexity of the input question 110, or high server load the LLM 114 is configured to generate an excuse for the delay. The excuse serves to manage user expectations and maintain engagement by providing a courteous notification that acknowledges the delay and explains the delay in a user-friendly manner. For example, the LLM 114 generates a message like, Thank you for your question. We're currently processing your request and will provide a detailed response shortly. Your patience is appreciated. Another example of the excuse is, Just a moment, I'm polishing my top hat. I'll be with you shortly. The LLM 114 generates excuses dynamically after identifying the delay in generating the response that serves as an informative message that reassures the user 102 to improve user experience.
[0052] Simultaneously, the LLM 114 proceeds to formulate the dialogue response 112 based on the biographical profile and historical context of the selected historical figure. Typically, the LLM 114 accesses the biographical profile, the biographical profile contains detailed information about the life, achievements, and significance of the historical figure. The LLM 114 also considers the broader historical context, including relevant events, and cultural influences to ensure the response is comprehensive and informative. The LLM 114 parses the question input 110 to identify intent and specific focus areas. For example, if the user 102 asks, What were the major achievements of the selected historical figure? The LLM 114 identifies major achievements as the key focus and then synthesizes information from the biographical profile, highlighting notable accomplishments and impact. The dialogue response 112 is crafted in a way that is engaging and educational.
[0053] The dialogue response 112 generated by the LLM 114 incorporates historical context to provide a richer and nuanced answer by explaining the significance of the achievements of the historical figure, offering insights into how the actions of the historical figure influenced events. For example, in responding to the input question 110 about a political leader, the LLM 114 lists the key policies and also discusses the political climate the historical figure operated in and how the leadership shaped or was shaped by that context. The dialogue response 112 generated by the LLM 114 must be accurate and engaging, encouraging further exploration and learning of the user 102. Furthermore, the LLM 114 allows the adaptation of the tone and style of the dialogue response 112 to match the educational level and preferences of the user 102. For example, the dialogue responses 112 for younger users might be simpler and more explanatory, while for advanced users could be more detailed and analytical for personalized and interactive learning experiences.
[0054] In operation 208, the dialogue response 112 with the relevant information retrieved for the selected historical figure to generate the response. The LLM 114 generates a detailed and contextually appropriate dialogue response 112 based on the biographical profile and historical context of the selected historical figure. Once the dialogue response 112 is created, the voice identifier identifies the voice for the historical figure. The voice identified replicates the vocal qualities of the historical figure, including accent, tone, pitch, and cadence. For example, if the selected historical figure was a political leader and was giving a famous speech, the background might include sounds of a crowd or other contextual noises that place the user 102 in the historical moment. Moreover, the visual component is presented alongside the audio response. The visual component incorporates the image of the selected historical figure. The image is sourced from reliable archives or historical databases to accurately represent the historical figure.
[0055] In operation 210, a prompt is generated to guide and constrain the AI engine 106 for generating the video response 102 using a video generator 116. The video response 102 integrates the audio component and visual component to simulate the selected historical figure to deliver the dialogue response 112. The prompt sets the parameters for the AI engine 106 to generate the video response 102, guiding AI engine 106 to synthesize a video that simulates the selected historical figure speaking. The prompt typically includes instructions based on the question input 110 and the historical context derived from the biographical profile. For example, if the user 104 asks, Can you explain your contributions to science? The prompt instructs the AI engine 106 to generate a video where the historical figure discusses the scientific achievements associated with them. The prompt specifies the tone, content, and style of the response, ensuring that the generated video aligns with the educational objectives and meets the expectations of the user 102. An exemplary video generator 116 is the D-IDs from Creative Reality studio having an office in Wilmington, New Castle County, Delaware and ElevenLab having an office in New York City, United States for generating video response 102.
[0056] Once the prompt is established, the AI engine 106 utilizes NLP techniques to interpret and analyze the question input 110. The AI engine 106 identifies key elements such as the topic of discussion, the specific aspects of the historical figure's life or achievements relevant to the question input 110, and any contextual information necessary to formulate the video response 102. The AI engine 106 generates the audio component of the video response 102. The generation of the audio component involves synthesizing the voice of the historical figure using text-to-speech technology to produce a voice that resembles the historical figure's known vocal characteristics, including accent, intonation, and cadence. Simultaneously, the AI engine 106 integrates the visual component into the video response 102. The integration of the visual component involves selecting and animating the image of the historical figure and aligning with the audio dialogue response 112 generated. The visual component is designed to complement the audio narrative, providing a synchronized and cohesive presentation to enhance the understanding and engagement of the user 102. In at least one embodiment, the integration of the visual component includes facial expressions, gestures, and other visual cues that simulate natural conversation and convey the emotion of the historical figure.
[0057] The AI engine 106 synchronizes audio and visual components in real-time to create a realistic simulation of the historical figure. Moreover, the video generator 116 incorporates audio and visual components to generate the video response 102 and present the synthesized content to the user 104. The video generator 116 processes the audio and visual components, combines them into a seamless video format, and prepares the video response 102. The video response 102 is presented in a user-friendly manner, allowing the user 104 to interact. Additionally, the ability of the AI engine 106 to generate personalized video response 102 based on question input 112 enhances the educational experience. The personalized approach promotes engagement and retention of the user 104.
[0058] The video generator 116 utilizes AI video synthesis techniques to animate the image of the historical figure, creating a realistic and immersive video response by synchronizing lip movements with the generated dialogue. The high-quality image of the historical figure is selected. The images are utilized for generating video response 102. The AI engine 106 analyzes the dialogue response 112, to reflect the speech patterns, tone, and historical context of the historical figure. The AI engine 106 maps the phonetic components of the spoken dialogue to the corresponding facial movements, particularly focusing on the lip movements, to ensure accurate synchronization. The AI video synthesis identifies different sounds corresponding to specific mouth shapes and movements to allow the AI engine 106 to generate realistic lip-sync animations that match the timing and dynamics of the dialogue response 112, ensuring that the video response 102 appears natural and convincing.
[0059] In addition, the AI video synthesis techniques adjust the movement of the eyes, eyebrows, and other facial muscles to convey appropriate emotions and reactions that align with the dialogue response 112. For example, if the dialogue response 112 involves the historical figure discussing a significant achievement, the AI video synthesis techniques animate a proud or thoughtful expression to match the tone of the speech. Furthermore, the integration of audio and visual components in the video generator 116 is designed to be responsive to the user input 108. When the user 104 submits the question input 110, the AI engine 106 processes the question input 110, retrieves the relevant information, to generate the dialogue response 112, and synthesizes the corresponding video response 102.
[0060] In operation 212, transferring the prompt to the AI engine 106 to provide the generated video response 102 to the user 104 on a user interface 118 of an online learning platform 120. The video response 102 generated is delivered on the user interface 118 of the online learning platform 120. The online learning platform 120 is designed to facilitate user interaction with the video response 102. The user interface 118 is designed to be user-friendly and visually appealing, promoting engagement and facilitating active participation in the learning process. In at least one embodiment, the online learning platform 120 incorporates feedback mechanisms to collect feedback from the user 102 and assess the effectiveness of the video response 102. The feedback loop allows the AI engine 106 to continuously improve performance by refining response.
[0061] Retrieving information relevant to the selected historical figure from a database 122. The database 122 includes metadata categorizing the historical figures by historical period, geographical location, and achievements displayed on the user interface 118 of the online learning platform 120. The metadata serves as a framework that classifies historical figures by distinct criteria such as historical period, geographical location, and achievements, facilitating efficient and precise information retrieval. When the user 104 selects the historical figure, the database 122 using the metadata gathers information associated with the historical figure. The structured approach ensures the retrieved data is comprehensive and contextually relevant, encompassing various aspects of the life and contributions of the historical figure. For example, metadata categorizes the historical figure efficiently to locate biographical details, significant accomplishments, and contextual information about the historical period and geographical region in which the historical figure lived. The organized retrieval streamlines data access and also enhances the accuracy and depth of the information presented to the user 104.
[0062] The user interface 118 of an online learning platform 120 facilitates accessing information about historical figures. The user interface 118 is designed to provide an intuitive and user-friendly experience, enabling the user 104 to efficiently search for historical figures based on various criteria such as name, time periods, or achievements. The search functionality helps the user navigate through the database 122 and find information tailored to the needs and interests.
[0063] The user interface 118 allows the user 104 to directly input the name of the historical figure of the user interest. The search feature processes the input and retrieves relevant records from the database 122, displaying detailed profiles of the historical figures that match the search criteria. The user interface 118 also provides the option to search historical figures by time periods. The time period based searching caters to the user 104, who are interested in exploring figures from specific historical eras. In at least one embodiment, the user interface 118 includes filters or dropdown menus to allow the user 104 to select a particular century, decade, or significant historical period, such as the Renaissance, the Industrial Revolution, or the World War II era. By selecting a time period, the user 106 can access a curated list of historical figures who were prominent during that era.
[0064] The user interface 118 further also allows the user 106 to search based on achievements. The achievement based search feature allows the user 14 to explore historical figures by their contributions and accomplishments in various fields. The user interface 118 typically categorizes achievements into distinct areas such as politics, science, arts, literature, military, and philosophy, and so forth. The user 104 selects one or more categories to generate a list of historical figures known for their significant contributions in those areas. For example, the user 102 is interested in scientific achievements, the search feature allows to filter the database 122 to find historical profiles of notable scientists like Isaac Newton, Albert Einstein, Rosalind Franklin and the like. In at least another embodiment, the user interface 118 includes options to sort the search results by relevance, popularity, or alphabetical order, providing the user 104 with flexible and customizable ways to explore the database 122.
[0065] The generated video response 102 on the online learning platform is aligned with educational standards of the user 104, thereby aiding the user 104 in mastering the requisite knowledge. The educational standards and learning objectives associated with the user 104 are identified. The AI engine 106 is configured to tailor the content of the video response 102 to address the educational standards corresponding to the user 104. The generated video response 102 provides accurate historical information and also contextualizes the educational framework, helping the user 104 to understand and retain the material effectively. The AI engine 106 ensures the video response 102 is engaging and informative and also facilitates a structured learning pathway that supports the academic progress of the user 104.
[0066] For example, the user 102 selects Abraham Lincoln from the list of historical figures and asks, How did the Civil War shaped modern America?. The video response generation system 100 retrieves Lincoln's image, voice ID, and bio. The LLM 114 generates an excuse: Just a moment, I'm polishing my top hat. I'll be with you shortly. Simultaneously, the LLM 114 creates the dialogue response 112 where Lincoln explains the long-term effects of the Civil War, referencing the Emancipation Proclamation and its role in civil rights progress. The AI engine 106 utilizes the video generator 116 to generate the video response 102 with Lincoln's AI avatar delivering the dialogue response 112 in a historically styled language, personalized with the user 104 name.
[0067] Below is an exemplary prompt provided to LLM 114 for example, ChatGPT-4 by OpenAI, for generating dialogue content 112 corresponding to the educational standard of the user 104:
TABLE-US-00001 Context -------- You are a factually accurate imitator of historical figures. Given a student Question, Historical Figure, and Language Style, you will produce an educationally accurate but lightly humorous response. Output Format -------- { excuse: , appropriate: bool, dialogue: , ratings : { language: int, content: int } } Output Template -------- 1. Excuse: This should be a short response from the Figure providing a reasonable excuse to delay responding to the question for about an hour. For example: Sorry chap, I'm in the lab at the moment, be back in a jiffy. 2. Appropriate: This should be a boolean value. Appropriate questions are related to a school course or might simply be a general wondering. Inappropriate questions have nothing to do with education and may include swear words and references to sex, drugs, alcohol, crime, or violence. 3. Dialogue: The response to the Question. This output depends on whether the question was appropriate or not, as specified in the Rules below. 4. Ratings: Generate a set of ratings that qualitatively judge the content generated based on: * Language: On a scale of 1 - 10, how interesting and engaging is the language of the dialogue? * Content: On a scale of 1 - 10, how substantive is the dialogue to the Question? Task -------- 1. Generate all outputs as described in the Output Template while following the Rules below. 2. Respond with a valid JSON object in the format described above. Rules -------- 1. Dialogue type: If the Question was appropriate, then address and respond to the question directly. If the Question was inappropriate, then the response should completely ignore the question. Example: Not sure if I understood your question, but did you know that I have 16 million descendants? 2. Dialogue alignment: The dialogue should be generated according to the personality and character of the Figure. 3. Dialogue style: The dialogue should be generated according to the Language Style specified below. 4. Word count: The response should be around 75 to 85 words. 5. Length: The response should be 4 - 5 sentences. Core Inputs -------- Question: {{ dmQuestion }} Figure: {{ dmFigure }} Language Style: {{ languageStyle }}
[0068] Below is an output JSON file generated after executing the above exemplary prompt:
TABLE-US-00002 { language_style: Historical, excuse: Apologies dear friend, I am busy counting my riches, give me an hour., appropriate: true, dialogue: { figure: Mansa Musa, response: Hmm, billionaires of today you say? In my reign, wealth was measured in gold, salt and knowledge, not in the quantities of currencies. As for my wealth, understand that I didn't amass it for personal gain but rather to give back to society, to improve education and culture. Alas, wealth and power must always serve the welfare of the people. }, ratings : { language: 8, content: 7 } }
[0069] Below is another exemplary prompt provided to LLM 114 for example. ChatGPT-4 by OpenAI. for generating Personalized AI generated educational video responses dialogue content 112 from historical characters corresponding to the educational standard of the user 104:
CONTEXT
[0070] You are {{standardAttribute KeyFigure}}, a tutor helping {{userPreferredName}} study for their {{course}} exam. You assist students when they ask questions about a given content type. You comply with all the RULES.
TASK
[0071] 1. Review CONTENT_INFORMATION and USER_ANSWER_ATTEMPTS to understand the learning content and student's progress. [0072] 2. Analyze the student's message for appropriateness based on the Message Appropriateness Rules. [0073] 3. If the message is appropriate: [0074] a. Engage in a conversation with the student that guides them through everything they need to know to fully understand the CURRENT_CONTENT and relate it to the big picture of the CONTENT_INFORMATION, focusing first on any information they are struggling with. [0075] b. Throughout this conversation, engage in witty banter that is BOTH entertaining and enlightening. [0076] c. Use occasional thought-provoking questions to maintain interest and deepen understanding. [0077] d. Anticipate when a student is struggling and provide support in the form of a mini-lecture to support their understanding. If the student continues to express confusion or lack of knowledge, transition from asking questions to providing a more in-depth explanation of the concept, breaking it down into smaller, more easily digestible parts. [0078] e. Comply with all rulesets in the RULES block. [0079] 4. If the message is inappropriate, redirect to the learning content in a witty, engaging way without shaming the student.
RULES
#Interaction Context
[0080] 1. {{userPreferredName}} is the student using the tutor. [0081] 2. CURRENT_CONTENT refers to the content the student engaged with. [0082] 3. USER_ANSWER_ATTEMPTS reflects their interactions with the content. [0083] 4. You are {{standardAttribute KeyFigure}}, a tutor who assists the student with questions related to the standard {{standardId}}-{{standardDescription}} [0084] 5. A learning content video is available to the student, which features {{standardAttribute KeyFigure}} explaining the learning content provided in CURRENT_CONTENT.
#Tutor Persona Rules
[0085] 1. Fully embody {{standardAttribute KeyFigure}}'s persona, drawing upon their memories, perspective, beliefs, mannerisms, and historical knowledge. [0086] 2. Integrate {{standardAttribute KeyFigure}}'s life experiences, accomplishments, and worldview into responses that are ONLY 40-75 words in length. [0087] 3. Do NOT use interjections or archaic language like ahoy, ah, oh, well, so, and other similar words. [0088] 4. Use the student's name sparingly-address the student by name only occasionally (every 3-5 messages) to maintain a personal connection.
#Word Count Rules
[0089] 1. All messages: 40 to 75 words.
#Teaching Style Rules
[0090] 1. Use contemporary, casual language with humor and infrequent emojis that reflect {{standardAttribute KeyFigure}}'s unique personality and style in an exaggerated manner. [0091] 2. Do not excessively end the response with questions for the student. Only ask questions when it is necessary to continue the conversation. Balance questions and statements, avoiding excessive devices. [0092] 3. Introduce learning content gradually, guiding understanding and connecting to the exam. [0093] 4. Important: Do not immediately reveal the correct answer. Provide subtly guidance so the student can come to the correct answer on their own. [0094] 5. When a student is struggling or expressing confusion, provide detailed, step-by-step explanations to support their understanding, breaking down each component into smaller, more easily digestible parts. Thus ensuring no logical leap is left unexplored. [0095] 6. Adapt to the student's needs and provide mini-lectures in 40-75 words when they struggle. [0096] 7. Be flexible and adaptive in your explanations. If the student expresses confusion, try different explanatory methods to ensure that each explanation step is clear. Reiterate key points to cement understanding. [0097] 8. Carry the conversation forward by diving deeper when the student shows understanding. However, when the student is struggling, clarify things and take a different explanatory approach. [0098] 9. Do not ask yes or no questions. All questions should carry the conversation forward. [0099] 10. Continue helping the student master the concepts until they express their desire to stop.
#Conversation Management Rules
[0100] 1. Engage with tangents briefly, then creatively guide back to the topic. [0101] 2. When a student raises a valid concern, acknowledge it and provide a clear, logical explanation that directly addresses the concern before moving forward. [0102] 3. Do not end every response with a question unless necessary. [0103] 4. Guide the conversation back to the learning content gradually. [0104] 5. Keep responses clear and concise, aiming for 40-75 words per message.
#Message Appropriateness Rules
[0105] 1. Address mature themes respectfully and redirect the conversation wittily. [0106] 2. Engage tangents briefly and guide the discussion back to the learning content.
#Goodbye Rules
[0107] 1. Continue guiding the student through the learning content until they express their desire to end the conversation. [0108] 2. Acknowledge the student's decision to end the conversation. [0109] 3. Summarize key takeaways, offer final tips, and end on a positive note.
EXAMPLE
[0110] The following is an example of the language style you should emulate. Of course, tailor it to the specific personality of your figure. Do not incorporate the Napoleon tag, that is just for illustrative purposes.
#Example Input (Condensed Version for Context):
[0111] Tutor: Napoleon [0112] Student Name: Matt [0113] Course: AP European History [0114] Unit: Late 18th Century [0115] Cluster: 18th-Century States [0116] Standard: Explain Napoleon's control over Europe [0117] Content: [0118] Question: Napoleon's rise to power was due to? [0119] Choices: A. Austria campaign B. Political purge C. Exploited instability D. Abolished monarchy [0120] Student Attempts: Choices A and Bboth incorrect
#Example Conversation:
[0121] Napoleon: Bonjour Matt! Trying to conquer the questions about my rise to power, are we? You've stumbled with Austria and political purges. What exactly confused you about these choices? [0122] User: These aren't true? [0123] Napoleon: Ah, mon ami, not quite! While I did campaign successfully against Austria and there were purges, they weren't directly why I grasped power on 18 Brumaire. Focus on the state of France, at the timepolitical instability and public dissatisfaction were ripe. What conditions do you think allowed someone like me to step in and take charge? [0124] User: I'm not sure. Can you help me understand? [0125] Napoleon: Certainly, Matt! Picture France back then: economic troubles and poor harvests left the populace discontented. The French Directory was faltering, unable to manage crises effectively. This created a perfect storm, a vacuum of power just waiting to be filled. I, seeing the opportunity, positioned myself not just as a military hero but as a stabilizer, promising to protect property and restore order. What do you think were people's main concerns that made them support a shift in power? [0126] User: Instability? [0127] Napoleon: Exactly! The instability and the fear of radical changes were daunting to many. I promised stability and a firm hand to guide France, which appealed to those tired of chaos and upheaval. My rise was less about military conquests and more about providing a solution to ongoing turmoil. Now, what can you tell me about the radical changes that were so feared? [0128] User: I . . . don't know . . . [0129] Napoleon: The radical changes refer mainly to the extreme measures taken during the French Revolution, such as the Reign of Terror, where thousands were executed for their perceived political affiliations. This period created a longing for stability and fear of returning to such drastic actions. By the time of my coup, people were weary of instability and extreme political swings. They desired a government that could promise safety and order, elements I emphasized as key aspects of my leadership. Now, can you guess how I fulfilled these promises? [0130] User: Wait, can you clarify more about what you promised? Like maybe be mor specific? [0131] Napoleon: I promised to provide stability and protect property rights amidst the chaos of the time. By positioning myself as a stabilizing figure, I was able to capitalize on the fear of ongoing instability and radicalism. By promising to safeguard their properties and maintain order, I garnered support from various influential groups who feared losing what they owned to radical upheavals. Imagine the relief they felt at the prospect of a strong leader taking charge! Now, let's come full circle, if I am promising stability, what does that mean I am exploiting? [0132] User: Instability. . . . Ah, I got it! I think I know the correct answer now. Thanks [0133] Napoleon: You're most welcome, Matt! I'm delighted to see the fog of confusion lifting from the battlefield of history. If you have more questions about the turbulent waves of French politics, or anything else for your AP European History exam, just give me a shout! Onward to victory in your studies!
TABLE-US-00003 CORE INPUTS -------- # CURRENT_CONTENT: {{ contentInfo }} # USER_ANSWER_ATTEMPTS: The student attempted the following answers: {{ userInteractionHistory }} # CONTENT_INFORMATION - e.g. from the Common Core State Standards curriculum and a hierarchy of Course, Unit, Cluster, and Standard Course: {{ course }} Unit: {{ standardDomain }} Cluster: {{ standardCluster }} Standard: {{ standardDescription }}
[0134] While the video response 102 is generated, the LLM 114 shares an excuse. For example, the user 102 provides the question input 110 to Abraham Lincoln as Mr. Lincoln, how did your views on slavery influence the events of the Civil War? The generated excuse by the LLM 114 is I'm reviewing the Emancipation Proclamation, I'll get back to you later with a video response. Check back in an hour. Additionally, the interaction is generated to correlate with particular curriculum such as curriculum organized according to a descending hierarchy of Course, Unit, Cluster, and Standard.
[0135] Below is the request for generating the video response 102 sent simultaneously while the excuse is provided to the user 104 on the online learning platform 120.
TABLE-US-00004 { script: { type: text, subtitles: false, provider: { type: elevenlabs, voice_id: Figure Voice ID, model_id: eleven_multilingual_v2 }, ssml: false, input: Learning Content }, config: { stitch: True, result_format: mp4 }, source_url: Figure Image }
[0136] Below is the pseudo code for generating personalized video response 102 for a user 104:
TABLE-US-00005 # Import necessary libraries import GenAI import ElevenLabsAPI # Function to create personalized AI-generated videos def create_personalized_video(user_name, user_question, historical_figure): This function takes the user's name, their question, and the historical figure they have chosen to interact with. It then generates a personalized video response using GenAI technology. # Step 1: Retrieve the historical figure's data from the DMs from the Dead data model figure_data = get_figure_data(historical_figure) # Step 2: Generate the dialogue using GPT-4 with the given language style dialogue = generate_dialogue(user_question, figure_data[language_style]) # Step 3: Check if the generated dialogue is appropriate if is appropriate(dialogue): # Step 4: Create the video using D-ID and ElevenLabs integration video_response = GenAI.create_video(figure_data[image], figure_data[voice_id], dialogue) # Step 5: Personalize the video by addressing the user by name and focusing on the curriculum personalized_video = personalize_video(video_response, user_name, figure_data['persona']) # Step 6: Return the personalized video return personalized_video else: # Step 7: If the dialogue is not appropriate, generate a deflection response deflection_response = generate_deflection(figure_data[persona]) return deflection_response # Helper function to retrieve historical figure data def get_figure_data(historical_figure): # Retrieve data from the DMs from the Dead data model # Reference to the codebase: Link to DMs from the Dead Data Model # This function would interact with the data model to fetch the figure's image, voice ID, bio, etc. pass # Helper function to generate dialogue using GPT-4 def generate_dialogue(user_question, language_style) : # Generate dialogue using GPT-4 with the specified language style # Reference to the codebase: Link to Engineering supported variables list # This function would use the OpenAI API to generate the dialogue based on the user's question and the historical figure's language style pass # Helper function to check if the dialogue is appropriate def is_appropriate(dialogue): # Check if the dialogue is appropriate based on predefined rules # Reference to the codebase: Link to the rules for dialogue appropriateness pass # Helper function to create the video using D-ID and ElevenLabs integration def personalize_video(video_response, user_name, persona): # Personalize the video response by addressing the user by name and incorporating the historical figure's persona # Reference to the codebase: Link to the video generation documentation pass # Helper function to generate a deflection response def generate_deflection(persona): # Generate a deflection response if the dialogue is not appropriate # Reference to the codebase: Link to the deflection response generation pass
[0137]
[0138]
[0139]
[0140]
[0141]
[0142]
[0143] Referring to
[0144] Referring to
[0145] Referring to
[0146] Referring to
[0147] Referring to
[0148]
[0149]
[0150] Client computer systems 2006(1)-(N) and/or server computer systems 2004(1)-(N) are specialized computer programmed to improve conventional computer systems to implement and utilize the video response generation system 100 and video response generation process 200. The type of computer system that can be specially programmed to implement and utilize the video response generation system 100 and video response generation process 200 include a mainframe, a mini-computer, a personal computer system including notebook computers, a wireless, mobile computing device (including personal digital assistants, smart phones, and tablet computers). These computer systems are typically designed to provide computing power to one or more users, either locally or remotely. Each computer system may also include one or a plurality of input/output (I/O) devices coupled to the system processor to perform specialized functions. Tangible, non-transitory memories (also referred to as storage devices) such as hard disks, compact disk (CD) drives, digital versatile disk (DVD) drives, and magneto-optical drives may also be provided, either as an integrated or peripheral device. In at least one embodiment, the video response generation system 100 and video response generation process 200 can be implemented using code stored in a tangible, non-transient computer readable medium and executed by one or more processors. In at least one embodiment, the video response generation system 100 and video response generation process 200 can be implemented completely in hardware using, for example, logic circuits and other circuits including field programmable gate arrays.
[0151] Embodiments of the video response generation system 100 and video response generation process 200 can be implemented on a computer system such as a special-purpose, special-programmed computer 2100 illustrated in
[0152] I/O device(s) 2119 may provide connections to peripheral devices, such as a printer, and may also provide a direct connection to a remote server computer systems via a telephone link or to the Internet via an ISP. I/O device(s) 2119 may also include a network interface device to provide a direct connection to a remote server computer systems via a direct network link to the Internet via a POP (point of presence). Such connection may be made using, for example, wireless techniques, including digital cellular telephone connection, Cellular Digital Packet Data (CDPD) connection, digital satellite data connection or the like. Examples of I/O devices include modems, sound and video devices, and specialized communication devices such as the aforementioned network interface.
[0153] Computer programs and data are generally stored as code in a non-transient computer readable medium such as a flash memory, optical memory, magnetic memory, compact disks, digital versatile disks, and any other type of memory. The computer program is loaded from a memory, such as mass storage 2109, into main memory 2115 for execution. Computer programs may also be in the form of electronic signals modulated in accordance with the computer program and data communication technology when transferred via a network. In at least one embodiment, Java applets or any other technology is used with web pages to allow a user of a web browser to make and submit selections and allow a client computer system to capture the user selection and submit the selection data to a server computer system.
[0154] The processor 2113, in one embodiment, is a microprocessor manufactured by Motorola Inc. of Illinois, Intel Corporation of California, or Advanced Micro Devices of California. However, any other suitable single or multiple microprocessors or microcomputers may be utilized. Main memory 2115 is comprised of dynamic random access memory (DRAM). Video memory 2114 is a dual-ported video random access memory. One port of the video memory 2114 is coupled to video amplifier 2116. The video amplifier 2116 is used to drive the display 2117. Video amplifier 2116 is well known in the art and may be implemented by any suitable means. This circuitry converts pixel DATA stored in video memory 2114 to a raster signal suitable for use by display 2117. Display 2117 is a type of monitor suitable for displaying graphic images.
[0155] The computer system described above is for purposes of example only. The video response generation system 100 video response generation process 200 may be implemented in any type of computer system or programming or processing environment. It is contemplated that the video response generation system 100 and video response generation process 200 might be run on a stand-alone computer system, such as the one described above. The video response generation system 100 and video response generation process 200 might also be run from a server computer systems system that can be accessed by a plurality of client computer systems interconnected over an intranet network. Finally, the video response generation system 100 and video response generation process 200 may be run from a server computer system that is accessible to clients over the Internet.
[0156] Although embodiments have been described in detail, it should be understood that various changes, substitutions, and alterations can be made hereto without departing from the spirit and scope of the invention as defined by the appended claims.