A machine reading comprehension (MRC) question and answer providing method includes receiving a user question; analyzing the user question; selecting at least one document from at least one domain corresponding to an analyzed user question and searching for a passage, which is a candidate answer determined as being suitable for the user question, in the selected at least one document; obtaining at least one correct answer candidate value by inputting the user question and a corresponding passage into each of at least one MRC question and answer unit; and determining whether the at least one correct answer candidate value is a best answer.

A machine reading comprehension (MRC) question and answer providing method includes receiving a user question; analyzing the user question; selecting at least one document from at least one domain corresponding to an analyzed user question and searching for a passage, which is a candidate answer determined as being suitable for the user question, in the selected at least one document; obtaining at least one correct answer candidate value by inputting the user question and a corresponding passage into each of at least one MRC question and answer unit; and determining whether the at least one correct answer candidate value is a best answer.

A data extraction apparatus includes a parameter analysis unit that performs analysis of learning text information, extracts words that serve as machine learning parameters, and classifies the words into types of parameters; a grouping settings display unit that finalizes search-target data and clustering conditions based on the parameters; at least one clustering training data extraction unit that extracts training data from a database based on the search-target data and the clustering conditions; at least one clustering unit that performs clustering based on the clustering condition on the training data; an applicable-clustering determination unit that performs analysis of search text information and identifies search-target data serving as a narrowing-down condition and which clustering unit is to be operated; and a search range specification unit that causes the clustering unit to operate and extracts a narrowed range of search-target data from the database based on an operation result.

A data extraction apparatus includes a parameter analysis unit that performs analysis of learning text information, extracts words that serve as machine learning parameters, and classifies the words into types of parameters; a grouping settings display unit that finalizes search-target data and clustering conditions based on the parameters; at least one clustering training data extraction unit that extracts training data from a database based on the search-target data and the clustering conditions; at least one clustering unit that performs clustering based on the clustering condition on the training data; an applicable-clustering determination unit that performs analysis of search text information and identifies search-target data serving as a narrowing-down condition and which clustering unit is to be operated; and a search range specification unit that causes the clustering unit to operate and extracts a narrowed range of search-target data from the database based on an operation result.

A system for providing intelligent part of speech processing of complex natural language is disclosed. The system identifies a multiword concept from an input and replaces the multiword concept with a token to be tagged as a desired part of speech. The system passes the modified text including the token to a part-of-speech tagger to tag each word in the text with the appropriate part-of-speech. The system may replace the token with the original text that the token was utilized to replace so that the original intent of the text is evident. The system may analyze the tagged text to generate analyses and interpretations associated with the input. When multiple multiword concepts are identified, the system may evaluate them by computing scores for each of the multiword concepts that may be replaced with tokens, for each of the modified texts including the tokens, or for any interpretations and analyses thereof.

Approaches for natural language processing include a multi-layer encoder for encoding words from a context and words from a question in parallel, a multi-layer decoder for decoding the encoded context and the encoded question, a pointer generator for generating distributions over the words from the context, the words from the question, and words in a vocabulary based on an output from the decoder, and a switch. The switch generates a weighting of the distributions over the words from the context, the words from the question, and the words in the vocabulary, generates a composite distribution based on the weighting of the distribution over the first words from the context, the distribution over the second words from the question, and the distribution over the words in the vocabulary, and selects words for inclusion in an answer using the composite distribution.

In a prediction model generating apparatus 1, data obtaining unit 11 obtains text data; variable group classifying unit 12 classifies the data into a plurality of variable groups; variable scoring unit 13 scores the data of at least one of the plurality of variable groups by associating that data with the data of another group; variable input unit 14 takes the data of the scored group as a response variable, and the data of the other group associated with the scored group as an explaining variable, and inputs those data to machine learning unit 15. The machine learning unit 15 generates, through machine learning, a prediction model predicting the response variable from the explaining variable.

In a prediction model generating apparatus 1, data obtaining unit 11 obtains text data; variable group classifying unit 12 classifies the data into a plurality of variable groups; variable scoring unit 13 scores the data of at least one of the plurality of variable groups by associating that data with the data of another group; variable input unit 14 takes the data of the scored group as a response variable, and the data of the other group associated with the scored group as an explaining variable, and inputs those data to machine learning unit 15. The machine learning unit 15 generates, through machine learning, a prediction model predicting the response variable from the explaining variable.

A system described herein may provide a technique for the compression of query terms and search data against which the query terms may be evaluated. The compression may be dynamic, in that a quantity of bits used to compress the search data and query terms may be based on a quantity of unique characters included in a given query term. The compression may further include reducing the volume of search data by compressing entire words, that do not include any of the unique characters of the query term, to one particular code.

The acquiring unit which acquires for each item name, one or more words composing an item name from the item name belonging to a group including a plurality of item names, respectively. The computing unit which computes for each item name, relevance that is a degree to which the acquired word is related to the item name, respectively. The determination unit which determines words among the acquired words as candidates for a classification name of each item represented by the plurality of item names. The sum over the plurality of item names of the computed relevance of the determined word is up to the top Mth (M is a natural number).