Systems and methods are provided for facilitating the building and use of natural language understanding models. The systems and methods identify a plurality of tokens and use them to generate one or more pre-trained natural language models using a transformer. The transformer disentangles the content embedding and positional embedding in the computation of its attention matrix. Systems and methods are also provided to facilitate self-training of the pre-trained natural language model by utilizing multi-step decoding to better reconstruct masked tokens and improve pre-training convergence.

In some examples, a word embedding is received. A word embedding includes a plurality of vectors in vector space. Each vector of the plurality of vectors represents a natural language word or other character sequence. Each vector is oriented in vector space based on a semantic similarity between each of the natural language word or other character sequence. A first distance is determined between a first vector and a second vector. A second distance is determined between a third vector and the second vector. Based at least in part on the first distance between the first vector and the second vector and the second distance between the third vector and the second vector, the third vector is moved closer or further away from the second vector. The moving is indicative of introducing a bias or removing a bias between the third vector and the second vector. This introducing or removing of bias supports more accurate and inclusive results from an application such as search of healthcare data, among other things.

Root cause estimation for a data set corresponding to customer returns of a product may use a probabilistic model to associate customer-entered product return data with probability distributions relating to possible root causes for the returns. A particular application relates to applying a Bayesian network to customer-selected return reason codes and customer-entered return reason comments to estimate a probability distribution for root causes of a plurality of returns and uncertainties relating to the probability distribution estimation. A bag-of-n-grams can be used to enable the Bayesian network to process natural language portions of the customer-entered product return data. The output of the model and other data relating to the root cause estimation can be conveyed to a seller of the returned products via a user interface.

Methods and systems for determining relevance for a new document are described. Existing documents that have a high probability of relevance can be chosen. A vocabulary of words in the existing documents can be built. Each word can be mapped into a vector such that each existing document can be represented by a sequence of vectors and each sentence and/or paragraph in each existing document can be represented by a subsequence of vectors including a subset of the sequence of vectors. Data augmentation can be applied changing an order of the subsequences in order to create additional documents represented by the subsequences. A deep neural network can be trained using the subsequences that represent the existing documents and the subsequences that represent additional documents. The new documents can be trained using a trained deep neural network. A relevant document can be output using the trained deep neural network.

A semantic vector generation device (100) obtains vectors of a plurality of words included in text data. The semantic vector generation device (100) extracts a word included in any group. The semantic vector generation device (100) generates a vector in accordance with the any group on the basis of a vector of the word extracted among the obtained vectors of the words. The semantic vector generation device (100) identifies a vector of a word included in an explanation of any semantics of the word extracted among the obtained vectors of the words. The semantic vector generation device (100) generates a vector in accordance with the any semantics on the basis of the vector identified and the vector generated.

A semantic vector generation device (100) obtains vectors of a plurality of words included in text data. The semantic vector generation device (100) extracts a word included in any group. The semantic vector generation device (100) generates a vector in accordance with the any group on the basis of a vector of the word extracted among the obtained vectors of the words. The semantic vector generation device (100) identifies a vector of a word included in an explanation of any semantics of the word extracted among the obtained vectors of the words. The semantic vector generation device (100) generates a vector in accordance with the any semantics on the basis of the vector identified and the vector generated.

A system for computer-aided escalation can include and/or interface with any or all of: a set of user interfaces (equivalently referred to herein as dashboards and/or hubs), a computing system, and a set of models. A method for computer-aided escalation includes any or all of: receiving a set of inputs; and processing the set of inputs to determine a set of outputs; triggering an action based on the set of outputs; and/or any other processes.

A method and system for analyzing and detecting a human tension based conflict between one or more parties using electronic communication data generated during communication between those parties. The method includes receiving electronic communication data, analyzing the electronic communication data by applying a predetermined method and generating conflict analysis score, the conflict analysis score providing a trigger or a seed for the conflict form the analyzed electronic communication data. In one or more embodiments, electronic communication data may be one or more of electronic-mail data, web content data, text message data, voice data, video content data, social media data.

A method and system for analyzing and detecting a human tension based conflict between one or more parties using electronic communication data generated during communication between those parties. The method includes receiving electronic communication data, analyzing the electronic communication data by applying a predetermined method and generating conflict analysis score, the conflict analysis score providing a trigger or a seed for the conflict form the analyzed electronic communication data. In one or more embodiments, electronic communication data may be one or more of electronic-mail data, web content data, text message data, voice data, video content data, social media data.

An information processing apparatus includes a lexical analysis unit that generates a training word string, a pair generation unit that generates a plurality of training word pairs, a matrix generation unit that generates, for each training word pair, a training matrix in which a plurality of words and respective semantic vectors of the words are associated, a classification unit that calculates, for a word of each position of the training word string, a probability of the word corresponding to a specific word, using the training matrices generated by the matrix generation unit and a determination model that uses a convolutional neural network, and an optimization processing unit that updates parameters of the determination model, such that the probability of the word labeled as corresponding to the specific word is high, among the probabilities of the words of the respective positions of the training word string calculated by the classification unit.