STORAGE MEDIUM, OUTPUT METHOD, AND OUTPUT DEVICE
20230076884 · 2023-03-09
Assignee
Inventors
Cpc classification
G06V20/70
PHYSICS
International classification
Abstract
A non-transitory computer-readable storage medium storing an output program that causes at least one computer to execute a process, the process includes converting input data into a semantic representation; and outputting a validity score based on a matching degree between a first relationship between a noun and a verb in the semantic representation and a second relationship between the noun and the verb in a database.
Claims
1. A non-transitory computer-readable storage medium storing an output program that causes at least one computer to execute a process, the process comprising: converting input data into a semantic representation; and outputting a validity score based on a matching degree between a first relationship between a noun and a verb in the semantic representation and a second relationship between the noun and the verb in a database.
2. The output program according to claim 1, wherein the outputting includes: counting a number of matches between the first relationship and the second relationship; and outputting the number of matches as the validity score.
3. The output program according to claim 1, wherein the process further comprising: when the validity score is lower than a certain threshold value, searching the database for the second relationship analogous to the first relationship; and outputting the second relationship as a correction candidate.
4. The output program according to claim 1, wherein the outputting includes outputting the validity score based on a weight preset in the second relationship that matches the first relationship.
5. The output program according to claim 1, wherein the outputting includes outputting the validity score based on a matching degree of a relationship type between the first relationship and the second relationship.
6. An output method for a computer to execute a process comprising: converting input data into a semantic representation; and outputting a validity score based on a matching degree between a first relationship between a noun and a verb in the semantic representation and a second relationship between the noun and the verb in a database.
7. The output method according to claim 6, wherein the outputting includes: counting a number of matches between the first relationship and the second relationship; and outputting the number of matches as the validity score.
8. The output method according to claim 6, wherein the process further comprising: when the validity score is lower than a certain threshold value, searching the database for the second relationship analogous to the first relationship; and outputting the second relationship as a correction candidate.
9. The output method according to claim 6, wherein the outputting includes outputting the validity score based on a weight preset in the second relationship that matches the first relationship.
10. The output method according to claim 6, wherein the outputting includes outputting the validity score based on a matching degree of a relationship type between the first relationship and the second relationship.
11. An output device comprising: one or more memories; and one or more processors coupled to the one or more memories and the one or more processors configured to: convert input data into a semantic representation, and output a validity score based on a matching degree between a first relationship between a noun and a verb in the semantic representation and a second relationship between the noun and the verb in a database.
12. The output device according to claim 11, wherein the one or more processors are further configured to: count a number of matches between the first relationship and the second relationship, and output the number of matches as the validity score.
13. The output device according to claim 11, wherein the one or more processors are further configured to: when the validity score is lower than a certain threshold value, search the database for the second relationship analogous to the first relationship, and output the second relationship as a correction candidate.
14. The output device according to claim 11, wherein the one or more processors are further configured to output the validity score based on a weight preset in the second relationship that matches the first relationship.
15. The output device according to claim 11, wherein the one or more processors are further configured to output the validity score based on a matching degree of a relationship type between the first relationship and the second relationship.
Description
BRIEF DESCRIPTION OF DRAWINGS
[0010]
[0011]
[0012]
[0013]
[0014]
[0015]
[0016]
[0017]
[0018]
[0019]
[0020]
[0021]
[0022]
DESCRIPTION OF EMBODIMENTS
[0023] In a case of using the HV, there are disadvantages in terms of use such as difficulty in interpreting common sense, and difficulty in linking with a common sense DB.
[0024] In one aspect, it is aimed to provide an output program, an output method, and an output device capable of making it easier to use a common sense utilization technique in machine learning.
[0025] In one aspect, it becomes possible to make it easier to use the common sense utilization technique in the machine learning.
[0026] Hereinafter, embodiments of an output program, an output method, and an output device will be described in detail with reference to the drawings. Note that the embodiments do not limit the present disclosure. Furthermore, the individual embodiments may be appropriately combined with each other within a range without inconsistency.
First Embodiment
[0027] First, an existing technique of common sense inference to be executed by an information processing apparatus will be described with reference to
[0028] Then, in an inference phase of the machine learning, the information processing apparatus inputs a query to the NN11, and extracts a feature of the query. Then, the information processing apparatus generates an HV on the basis of the extracted feature, specifies a label recalled from the generated HV using the HV memory 15, and outputs the specified label as an inference result.
[0029]
[0030] As illustrated in
[0031] However, in a case of using the HV, there are disadvantages in terms of use such as difficulty in interpreting common sense, and difficulty in linking with a common sense DB. In view of the above, the present embodiment aims to provide an output program, an output method, and an output device capable of making it easier to use a common sense utilization technique in the machine learning.
[0032] [Functional Configuration of Output Device 10]
[0033] Next, a functional configuration of an output device 10, which is an execution subject of the present embodiment, will be described.
[0034] The communication unit 20 is, for example, a processing unit that controls communication with another information processing apparatus to/from which various kinds of data, such as input data of images, texts, and the like, and determination results of validity scores are transmitted/received, and is, for example, a communication interface such as a network interface card.
[0035] The storage unit 30 is an exemplary storage device that stores various kinds of data and a program to be executed by the control unit 40, and is, for example, a memory, a hard disk, or the like. The storage unit 30 stores input data 31, a common sense DB 32, and the like.
[0036] The input data 31 stores data to be input to the output device 10 for the purpose of utilizing common sense. The data may be an image, or may be text. Furthermore, the data may be uploaded from the another information processing apparatus to the output device 10 via the communication unit 20, or may be read by the output device 10 via any computer-readable recording medium.
[0037] The common sense DB 32 stores, for example, a combination of a noun and a verb determined to be valid and a relationship type of the combination in association with each other. For example, the common sense DB 32 stores, for example, a combination of “human” (noun) and “draw” (verb), and “capable of”, which is a relationship type of the combination, in association with each other. Furthermore, as another example, the common sense DB 32 stores a combination of “draw” (verb) and “picture” (noun), and “related to”, which is a relationship type of the combination, in association with each other. Note that the nouns and verbs, and the relationship types are not limited to the examples described above.
[0038] Note that the information described above stored in the storage unit 30 is merely an example, and the storage unit 30 may store various types of information other than the information described above.
[0039] The control unit 40 is a processing unit that controls the entire output device 10, and is, for example, a processor or the like. The control unit 40 includes a conversion unit 41 and an output unit 42. Note that each processing unit is an exemplary electronic circuit included in a processor or an exemplary process to be executed by the processor.
[0040] The conversion unit 41 analyzes the input image or text, and converts it into a semantic representation. For conversion of text into a semantic representation, the conversion unit 41 converts a text meaning into a semantic representation expressed by a directed acyclic graph using an abstract meaning representation (AMR) parser of an existing technique, for example.
[0041]
[0042] Meanwhile, for conversion of an image into a semantic representation, the conversion unit 41 generates a scene graph that describes a relationship between objects contained in the image using a scene graph generator of an existing technique, and converts the image into a semantic representation on the basis of the scene graph, for example.
[0043]
[0044] The output unit 42 outputs a validity score on the basis of a matching degree between a first relationship between the noun and the verb in the semantic representation and a second relationship between the noun and the verb in a database stored in advance. For example, the output unit 42 searches the common sense DB 32 for the combination of individual nodes in the subgraph converted from the image or text data by the conversion unit 41, counts the number of matches, and outputs it as a validity score. Note that the combination of individual nodes stored in the common sense DB 32 may be weighted, and the validity score may be calculated on the basis of the weighting.
[0045] Furthermore, the validity score is an exemplary index indicating that the combination of nodes is valid, which is a commonsensical combination, and may be used at a time of determining validity of a sentence, for example. However, with the validity score used as an index indicating specificity of a sentence, for example, it becomes possible to select, from collected ideas and the like, sentences with untrammeled and unconventional contents, novel contents, conspicuous opinions, and the like.
[0046] Furthermore, in a case where the validity score is lower than a predetermined threshold value, the output unit 42 searches the common sense DB 32 for a second relationship similar to the first relationship in the semantic representation that has not matched, and outputs it as a correction candidate.
[0047] [Function Details]
[0048] The output process of the validity score, which makes it easier to use the common sense utilization technique in the machine learning, will be described in more detail with reference to
[0049] As illustrated in
[0050] Then, the output unit 42 searches the common sense DB 32 on the basis of the subgraph extracted from the semantic graph 81, and outputs a validity score on the basis of a matching degree between the relationship between the noun and the verb in the subgraph and the relationship between the noun and the verb in the common sense DB 32.
[0051] The output process of the validity score will be described with a specific example. While the following output process of the validity score will be explained using a case where text data is input as an example, in a case where image data is input as well, the knowledge encoder to be used at the time of semantic graph generation is different but the subsequent process is similar to that in the case where the text data is input.
[0052]
[0053] Then, the output device 10 searches the common sense DB 32 for each combination of a noun and a verb included in the extracted subgraph 92, and calculates a validity score on the basis of a matching degree of the combination. In the example of
[0054]
[0055] The exemplary case where the validity score is high, which is a case where all combinations included in the subgraph 92 match the data in the common sense DB 32, has been described with reference to
[0056]
[0057] Then, the output device 10 searches the common sense DB 32 for each combination of a noun and a verb included in the extracted subgraph 91, and calculates a validity score on the basis of a matching degree of the combination. The example of
[0058] Next, an example in which, in a case where the validity score is low, a combination similar to the combination of the subgraph that has not matched is searched from the common sense DB 32 to output it as a correction candidate will be described.
[0059]
[0060] The example of
[0061] Furthermore, the validity score may be weighted and output.
[0062] In the example of
[0063] The weight in the output process of the validity score may be regarded as strength and reliability of a combination of a noun and a verb. For example, in the example of
[0064] [Process Flow]
[0065] Next, a flow of the validity score output process performed by the output device 10 will be described with reference to
[0066] First, the output device 10 obtains, from the input data 31, the input image or text to be processed, and converts it into a semantic representation (step S101).
[0067] Next, the output device 10 searches the common sense DB 32 for a relationship between a noun and a verb in the semantic representation converted in step S101 (step S102). For example, the output device 10 generates a semantic graph from the semantic representation converted in step S101 to extract a subgraph, and searches the common sense DB 32 for a combination of a noun and a verb in the subgraph. Note that, when searching the common sense DB 32, the searching may be carried out by including, in addition to the combination of the noun and the verb, a relationship type thereof.
[0068] Next, the output device 10 calculates a validity score on the basis of a result of the searching in the common sense DB 32 (step S103). For example, the output device 10 counts the number of matches of the combination of the noun and the verb in the searching in the common sense DB 32 in step S102, and outputs it as a validity score. Note that the validity score may be calculated while being weighted on the basis of a weight preset for each combination of a noun and a verb in the common sense DB 32, for example.
[0069] If the validity score output in step S103 is equal to or higher than a predetermined threshold value (No in step S104), it is determined that the combination of the noun and the verb in the semantic representation converted in step S101 is valid, and the validity score output process illustrated in
[0070] On the other hand, if the validity score output in step S103 is lower than the predetermined threshold value (Yes in step S104), the output device 10 determines that an inappropriate combination is included in the combinations of nouns and verbs in the semantic representation, and outputs a correction candidate (step S105). For example, the output device 10 searches the common sense DB 32 for a combination similar to the combination of the noun and the verb in the semantic representation that does not match the data in the common sense DB 32, and outputs it as a correction candidate. After the execution of step S105, the validity score output process illustrated in
[0071] [Effects]
[0072] As described above, the output device 10 analyzes the input image or text, converts it into a semantic representation, and outputs a validity score on the basis of the matching degree between the first relationship between the noun and the verb in the semantic representation and the second relationship between the noun and the verb in the common sense DB 32 stored in advance.
[0073] This makes it possible to determine the validity of the semantic representation in the input image or text without using the HV, which has disadvantages in terms of use such as difficulty in interpreting common sense, and difficulty in linking with the common sense DB. In this manner, the output device 10 may make it easier to use the common sense utilization technique in the machine learning.
[0074] Furthermore, the process of outputting the validity score executed by the output device 10 includes a process of counting the number of matches between the first relationship and the second relationship and outputting the number of matches as a validity score.
[0075] As a result, the output device 10 may make it easier to use the common sense utilization technique in the machine learning.
[0076] Furthermore, in a case where the validity score is lower than a predetermined threshold value, the output device 10 searches the common sense DB 32 for a second relationship similar to the first relationship that has not matched, and outputs it as a correction candidate.
[0077] As a result, the output device 10 is enabled to determine that an inappropriate combination is included in the combinations of nouns and verbs in the semantic representation of the input image or text to output a correction candidate.
[0078] Furthermore, the process of outputting the validity score executed by the output device 10 includes a process of outputting the validity score further on the basis of the weight preset in the second relationship that matches the first relationship.
[0079] As a result, the output device 10 is enabled to output a more appropriate validity score.
[0080] Furthermore, the process of outputting the validity score executed by the output device 10 includes a process of outputting the validity score further on the basis of the matching degree of the relationship type between the matching first relationship and second relationship.
[0081] As a result, the output device 10 is enabled to output a more appropriate validity score.
[0082] [System]
[0083] A processing procedure, a control procedure, a specific name, and information including various kinds of data and parameters indicated in the descriptions above or in the drawings may be changed in any ways unless otherwise specified. Furthermore, the specific examples, distributions, numerical values, and the like described in the embodiment are merely examples, and may be changed in any ways.
[0084] Furthermore, each component of each device illustrated in the drawings is functionally conceptual, and is not necessarily physically configured as illustrated in the drawings. For example, specific forms of distribution and integration of each device are not limited to those illustrated in the drawings. For example, all or a part thereof may be configured by being functionally or physically distributed or integrated in any units depending on various types of loads, usage situations, or the like. Moreover, all or any part of individual processing functions performed by each device may be implemented by a central processing unit (CPU), a graphics processing unit (GPU), and a program analyzed and executed by the CPU and the GPU, or may be implemented as hardware by wired logic.
[0085] [Hardware]
[0086]
[0087] The communication interface 10a is a network interface card or the like, and communicates with another server. The HDD 10b stores programs and DBs for operating the functions illustrated in
[0088] The processor 10d is a hardware circuit that reads, from the HDD 10b or the like, a program that executes processing similar to that of each processing unit illustrated in
[0089] In this manner, the output device 10 operates as an information processing apparatus that executes operation control processing by reading and executing the program that executes processing similar to that of each processing unit illustrated in
[0090] Furthermore, the program that executes processing similar to that of each processing unit illustrated in
Second Embodiment
[0091] While the embodiment has been described above, the embodiment may be implemented in various different modes in addition to the embodiment described above.
[0092] All examples and conditional language provided herein are intended for the pedagogical purposes of aiding the reader in understanding the invention and the concepts contributed by the inventor to further the art, and are not to be construed as limitations to such specifically recited examples and conditions, nor does the organization of such examples in the specification relate to a showing of the superiority and inferiority of the invention. Although one or more embodiments of the present invention have been described in detail, it should be understood that the various changes, substitutions, and alterations could be made hereto without departing from the spirit and scope of the invention.