Systems and methods are provided for facilitating the building and use of natural language understanding models. The systems and methods identify a plurality of tokens and use them to generate one or more pre-trained natural language models using a transformer. The transformer disentangles the content embedding and positional embedding in the computation of its attention matrix. Systems and methods are also provided to facilitate self-training of the pre-trained natural language model by utilizing multi-step decoding to better reconstruct masked tokens and improve pre-training convergence.

A computing device receives from a user a partial natural language input related to a data source. The computing device receives an additional keystroke corresponding to the partial natural language input. The partial natural language input and the additional keystroke comprise a character string. In response to the additional keystroke, the computing device generates one or more interpretations corresponding to entities in the data source. The computing device displays the interpretations. In some implementation, the character string comprises a sequence of terms, and the device displays the interpretations in a dropdown menu adjacent to the most recently entered term in the sequence. In some implementations, the dropdown menu includes a plurality of rows, each row displaying a respective data value and a respective data field corresponding to the respective data value. Some implementations display a statistical distribution of data values for a data field (displayed adjacent to the first interpretation).

A computing device receives from a user a partial natural language input related to a data source. The computing device receives an additional keystroke corresponding to the partial natural language input. The partial natural language input and the additional keystroke comprise a character string. In response to the additional keystroke, the computing device generates one or more interpretations corresponding to entities in the data source. The computing device displays the interpretations. In some implementation, the character string comprises a sequence of terms, and the device displays the interpretations in a dropdown menu adjacent to the most recently entered term in the sequence. In some implementations, the dropdown menu includes a plurality of rows, each row displaying a respective data value and a respective data field corresponding to the respective data value. Some implementations display a statistical distribution of data values for a data field (displayed adjacent to the first interpretation).

Identifying documents that contain potential code words using a machine learning model. In some embodiments, a method may include receiving documents, identifying a first corpus and a second corpus in the documents, extracting a first set of word embeddings from the first corpus and a second set of word embeddings from the second corpus, generating a first vector space for the first set of word embeddings and a second vector space for the second set of word embeddings using a machine learning model, performing a vector rotation to improve alignment of the first set of word embeddings with the second set of word embeddings, identifying a word embedding in the first vector space that is not aligned with a corresponding word embedding in the second vector space as a potential code word, and identifying one or more documents that contain the potential code word in the first corpus.

Methods and systems for generating an end-to-end neural text-to-speech (TTS) model to process an input text to generate speech representations. An annotated set of text documents including annotations inserted therein to indicate prosodic features are input into the TTS model. The TTS model is trained using the annotated dataset and a corresponding dataset of speech representations of the text documents that include prosody associated with the indicated prosodic features. The trained TTS model learns to associate the prosody with the annotations.

Methods and systems for generating an end-to-end neural text-to-speech (TTS) model to process an input text to generate speech representations. An annotated set of text documents including annotations inserted therein to indicate prosodic features are input into the TTS model. The TTS model is trained using the annotated dataset and a corresponding dataset of speech representations of the text documents that include prosody associated with the indicated prosodic features. The trained TTS model learns to associate the prosody with the annotations.

The embodiments herein disclose a system and a method for providing an online web-based interactive audio-visual platform for note creation, presentation, sharing, organizing, and analysis. The system provides a conceptual and interactive interface to content; analyses a student's notes and instantly determines the accuracy of the conceptual connections made and a student's understanding of a topic. The system enables the student to add and use audio, visual, drawing, text notes, and mathematical equations in addition to those suggested by the note taking solution; to collate notes from various sources in a meaningful manner by grouping concepts using colors, images, and text; and to personalize other maps developed within the same environment while maintaining links back to the original source from which the notes are derived. The system highlights keywords in conjunction with spoken text to complement the advantages of using visual maps to improve learning outcomes.

A method, system, and computer program product for data reorganization and logs reorganization. The method includes receiving, by one or more processing units, original data. The method also includes classifying, by the one or more processing units, the original data into different types based on a trained type classification model. The method also includes generating, by the one or more processing units, at least one severity for at least part of the original data based on a trained severity classification model, the at least part of the original data corresponding to at least one type. The method also includes outputting, by the one or more processing units, at least one message, the at least one message indicating the severity of the at least part of the original data.

In one embodiment, a system includes an automatic speech recognition (ASR) module, a natural-language understanding (NLU) module, a dialog manager, one or more agents, an arbitrator, a delivery system, one or more processors, and a non-transitory memory coupled to the processors comprising instructions executable by the processors, the processors operable when executing the instructions to receive a user input, process the user input using the ASR module, the NLU module, the dialog manager, one or more of the agents, the arbitrator, and the delivery system, and provide a response to the user input.

To arrange all words so that the distance of a given word pair will be appropriate. Using as input a concept base 22 which is a set of pairs of a word and a vector representing a concept of the word, and a dictionary 24 which is a set of semantically distant or close word pairs, when a word pair C being a pair of given words A, B in the concept base 22 is present in the dictionary 24, conversion means 30 associates with the word pair C a magnitude D of a difference vector between a difference vector V′ between a converted vector of the word A and a converted vector of the word B, and a vector kV determined by multiplying a difference vector V between the vector of the word A in the concept base 22 and the vector of the word B in the concept base 22 by a scalar value k. When the word pair C is not present in the dictionary 24, the conversion means 30 associates the magnitude D of the difference vector between the difference vector V′ and the difference vector V with the word pair C. The conversion means 30 converts the vector of a given word in the concept base 22 such that a total sum of the magnitude D corresponding to every word pair C is minimized.