A system for providing intelligent part of speech processing of complex natural language is disclosed. The system identifies a multiword concept from an input and replaces the multiword concept with a token to be tagged as a desired part of speech. The system passes the modified text including the token to a part-of-speech tagger to tag each word in the text with the appropriate part-of-speech. The system may replace the token with the original text that the token was utilized to replace so that the original intent of the text is evident. The system may analyze the tagged text to generate analyses and interpretations associated with the input. When multiple multiword concepts are identified, the system may evaluate them by computing scores for each of the multiword concepts that may be replaced with tokens, for each of the modified texts including the tokens, or for any interpretations and analyses thereof.

A learning device includes one or more processors. The processors input, to an input layer of a neural network including hidden layers defined for respective first arrangement patterns indicating arrangement of one or more words, and output layers connected with some of hidden layers, one or more first morphemes conforming to any of first arrangement patterns, among morphemes included in a document, and learn the neural network to minimize a difference between one or more second morphemes conforming to any of second arrangement patterns indicating arrangement of one or more words, among morphemes included in the document, and output morphemes from the neural network for the input first morphemes. The processors output an embedding vector of the first morphemes that is obtained based on a weight of the learned neural network.

A learning device includes one or more processors. The processors input, to an input layer of a neural network including hidden layers defined for respective first arrangement patterns indicating arrangement of one or more words, and output layers connected with some of hidden layers, one or more first morphemes conforming to any of first arrangement patterns, among morphemes included in a document, and learn the neural network to minimize a difference between one or more second morphemes conforming to any of second arrangement patterns indicating arrangement of one or more words, among morphemes included in the document, and output morphemes from the neural network for the input first morphemes. The processors output an embedding vector of the first morphemes that is obtained based on a weight of the learned neural network.

A document processing device includes: an input unit for receiving an input question sentence; a text analysis unit for performing a morphological analysis on a sentence received by the input unit; a hypothetical question storage unit for storing a hypothetical question sentence and an answer sentence in association with each other; a search unit for searching for the hypothetical question sentence similar to the input question sentence from the hypothetical question storage unit and obtaining an answer sentence; an output unit for outputting the answer sentence; and a normalization processing unit for performing a normalization on each word of the input question sentence and the hypothetical question sentence converted into word strings by the text analysis unit. The search unit is configured to perform a similarity determination on the input question sentence and the hypothetical question sentence whose words are normalized by the normalization processing unit.

A document processing device includes: an input unit for receiving an input question sentence; a text analysis unit for performing a morphological analysis on a sentence received by the input unit; a hypothetical question storage unit for storing a hypothetical question sentence and an answer sentence in association with each other; a search unit for searching for the hypothetical question sentence similar to the input question sentence from the hypothetical question storage unit and obtaining an answer sentence; an output unit for outputting the answer sentence; and a normalization processing unit for performing a normalization on each word of the input question sentence and the hypothetical question sentence converted into word strings by the text analysis unit. The search unit is configured to perform a similarity determination on the input question sentence and the hypothetical question sentence whose words are normalized by the normalization processing unit.

Disclosed herein are methods, systems, processes, and machine learning paradigms to implement a password semantic analysis pipeline. A password semantic analysis pipeline model is trained according to one or more machine learning techniques to at least (a) determine, based on given characteristics data of a given network environment, whether each of several tokens that are chunked portions of a data structure input as a password in an application is a known syntax type or a recognized entity, (b) generate, using the password semantic analysis pipeline model, a password strength score that is a combination of a confidence score determined for each of the plurality of tokens and a weight factor assigned to the known syntax type or the recognized entity, (c) apply the password strength score to the data structure input as the password in the application, and (d) provide an output to the application indicating whether the data structure input as the password is acceptable or unacceptable for continued access to the application.

A multi-feature log anomaly detection method includes steps of: preliminarily processing a log data set to obtain a log entry word group corresponding to all semantics of a log sequence in the log data set, and using the log entry word group as a semantic feature of the log sequence; extracting a type feature, a time feature and a quantity feature of the log sequence, and encoding the semantic feature, the type feature, the time feature and the quantity feature into a log feature vector set of the log sequence; training a BiGRU neural network model with all log feature vector sets to obtain a trained BiGRU neural network mode; and inputting the log data set to be detected into the trained BiGRU neural network model for prediction, and determining whether the log sequence is a normal or abnormal log sequence according to a prediction result.

Provided is a technique for generating a tagging model for attaching a tag in consideration of a phrase based on dependency between words. A tagging model generation apparatus includes a learning section 2 which generates, by using inputted learning data, a tagging model including probability-related information serving as information related to the probability that each tag is associated with each word-related information, and joint probability-related information serving as information related to a joint probability which serves as the probability of appearance of each tag in which appearance frequencies of a plurality of consecutive tags associated with pieces of word-related information of a plurality of consecutive words in each text are taken into consideration, and a storage section 3 which stores the generated tagging model.

An information processing apparatus includes a reception unit that receives information from a user in an interactive form and a selection unit that selects a first service for analyzing contents of natural language as a transmission destination of the information in a case where information from the user is input in natural language and selects a second service as an analysis destination of the received information in a case where information from the user is input as an image or a sound.

An information processing apparatus includes a reception unit that receives information from a user in an interactive form and a selection unit that selects a first service for analyzing contents of natural language as a transmission destination of the information in a case where information from the user is input in natural language and selects a second service as an analysis destination of the received information in a case where information from the user is input as an image or a sound.