DATA MODIFICATION METHOD AND INFORMATION PROCESSING APPARATUS

Abstract

A computer-readable recording medium having stored therein a data modification program executable by one or more computers, the data modification program includes: an instruction for specifying, from a plurality of attributes included in training data, a first attribute having a causal relation with a second attribute included in the plurality of attributes; and an instruction for modifying values of the first attribute in the training data in accordance with a condition for reducing a difference between distributions of the values of the first attribute corresponding to each value of the second attribute.

Claims

1. A non-transitory computer-readable recording medium having stored therein a data modification program executable by one or more computers, the data modification program comprising an instruction for specifying, from a plurality of attributes included in training data, a first attribute having a causal relation with a second attribute included in the plurality of attributes, and an instruction for modifying values of the first attribute in the training data in accordance with a condition for reducing a difference between distributions of the values of the first attribute corresponding to each value of the second attribute.

2. The non-transitory computer-readable recording medium according to claim 1, wherein the condition includes a condition that more intensively reduces a difference between distributions of values of a third attribute, the third attribute having a stronger causal relation with the second attribute than the causal relation between the first attribute and the second attribute, and the modifying comprises reducing the difference between distributions of values of the third attribute in accordance with the condition.

3. The non-transitory computer-readable recording medium according to claim 2, wherein the data modifying program further comprises an instruction for calculating an index representing strength of the causal relation between the first attribute and the second attribute, the index being based on the training data and an initial value of the modifying, and the modifying comprises reducing the values of the first attribute serving as the third attribute in accordance with a reduction ratio based on the initial value and the index.

4. The non-transitory computer-readable recording medium according to claim 1, wherein the data modifying program further comprises an instruction for suppressing modification of values of a fourth attribute included in the plurality of attributes included in the training data, the fourth attribute being correlated with the second attribute, and the fourth attribute having no causal relation with the second attribute.

5. The non-transitory computer-readable recording medium according to claim 1, wherein the second attribute is a protected attribute.

6. A computer-implemented data modification method comprising: specifying, from a plurality of attributes included in training data, a first attribute having a causal relation with a second attribute included in the plurality of attributes; modifying values of the first attribute in the training data in accordance with a condition for reducing a difference between distributions of the values of the first attribute corresponding to each value of the second attribute.

7. The data modification method according to claim 6, wherein the condition includes a condition that more intensively reduces a difference between distributions of values of a third attribute, the third attribute having a stronger causal relation with the second attribute than the causal relation between the first attribute and the second attribute, and the modifying comprises reducing the difference between distributions of values of the third attribute in accordance with the condition.

8. The data modification method according to claim 7, wherein the data modification method further comprises calculating an index representing strength of the causal relation between the first attribute and the second attribute, the index being based on the training data and an initial value of the modifying, and the modifying comprises reducing the values of the first attribute serving as the third attribute in accordance with a reduction ratio based on the initial value and the index.

9. The data modification method according to claim 6, wherein the method further comprises suppressing modification of values of a fourth attribute included in the plurality of attributes included in the training data, the fourth attribute being correlated with the second attribute, and the fourth attribute having no causal relation with the second attribute.

10. The data modification method according to claim 6, wherein the second attribute is a protected attribute.

11. An information processing apparatus comprising: a memory; and a processor coupled to the memory, the processor being configured to: perform specification of, from a plurality of attributes included in training data, a first attribute having a causal relation with a second attribute included in the plurality of attributes, and perform modification of values of the first attribute in the training data in accordance with a condition for reducing a difference between distributions of the values of the first attribute corresponding to each value of the second attribute.

12. The information processing apparatus according to claim 11, wherein the condition includes a condition that more intensively reduces a difference between distributions of values of a third attribute, the third attribute having a stronger causal relation with the second attribute than the causal relation between the first attribute and the second attribute, and the modification comprises reduction of the difference between distributions of values of the third attribute in accordance with the condition.

13. The information processing apparatus according to claim 12, wherein the processor is further configured to perform calculation of an index representing strength of the causal relation between the first attribute and the second attribute, the index being based on the training data and an initial value of the modifying, and the modification comprises reducing of the values of the first attribute serving as the third attribute in accordance with a reduction ratio based on the initial value and the index.

14. The information processing apparatus according to claim 11, wherein the processor is further configured to perform suppressing of modification of values of a fourth attribute included in the plurality of attributes included in the training data, the fourth attribute being correlated with the second attribute, and the fourth attribute having no causal relation with the second attribute.

15. The information processing apparatus according to claim 11, wherein the second attribute is a protected attribute.

Description

BRIEF DESCRIPTION OF DRAWINGS

[0009] FIG. 1 is a block diagram illustrating an example of the hardware (HW) configuration of a computer that achieves the function of a data modification apparatus according to one embodiment;

[0010] FIG. 2 is a block diagram schematically illustrating an example of the functional configuration of the data modification apparatus of the one embodiment;

[0011] FIG. 3 is a diagram illustrating an example of data;

[0012] FIG. 4 is a diagram illustrating an example of a reducing correlation by using a Disparate Impact Remover (DIR);

[0013] FIG. 5 is a diagram illustrating an example of a reduction ratio of correlation when a causal graph is not used;

[0014] FIG. 6 is a diagram illustrating an example of a causal graph;

[0015] FIG. 7 a diagram illustrating an example of a reduction ratio of correlation on the basis of a causal graph;

[0016] FIG. 8 is a flow diagram schematically illustrating an example of operation of the data modification apparatus of the one embodiment; and

[0017] FIG. 9 is a diagram illustrating an example of an inference result obtained with a machine learning model trained by modified data according to the one embodiment.

DESCRIPTION OF EMBODIMENT(S)

[0018] Since the above-described method does not use the exact causal relation (causal-effect relation) between a protected attribute and a non-protected attribute, data may be changed even for a non-protected attribute that accidentally has correlation with the protected attribute.

[0019] Further, the degree of rewriting of the values of a non-protected attribute is uniformly determined based on the specified (e.g., a single) parameter.

[0020] For the above, in the above-described method, data is changed even for a non-protected attribute that accidentally has correlation with the protected attribute, which may consequently degrade the accuracy of an inference result made by a machine learning model.

[0021] Hereinafter, an embodiment of the present invention will now be described with reference to the accompanying drawings. However, the embodiment described below is merely illustrative and is not intended to exclude the application of various modifications and techniques not explicitly described below. For example, the present embodiment can be variously modified and implemented without departing from the scope thereof. In the drawings to be used in the following description, the same reference numbers denote the same or similar parts, unless otherwise specified.

[0022] Hereinafter, description will now be made in relation to a data modification apparatus 1 (see FIG. 2) according to the one embodiment as a method for suppressing degradation in accuracy of an inference result by a machine learning model.

[0023] The data modification apparatus 1 according to the embodiment may be a virtual server (Virtual Machine (VM)) or a physical server. The functions of the data modification apparatus 1 may be achieved by one computer or by two or more computers. Further, at least some of the functions of the data modification apparatus 1 may be implemented using Hardware (HW) resources and Network (NW) resources provided by cloud environment.

[0024] FIG. 1 is a block diagram illustrating an example of the hardware (HW) configuration of a computer 10 that achieves the functions of the data modification apparatus 1. If multiple computers are used as the HW resources for achieving the functions of the data modification apparatus 1, each of the computers may include the HW configuration illustrated in FIG. 1.

[0025] As illustrated in FIG. 1, the computer 10 may illustratively include a HW configuration formed of a processor 10a, a memory 10b, a storing device 10c, an IF (Interface) device 10d, an I/O (Input/Output) device 10e, and a reader 10f.

[0026] The processor 10a is an example of an arithmetic operation processing device that performs various controls and calculations. The processor 10a may be communicably connected to the blocks in the computer 10 via a bus 10i. The processor 10a may be a multiprocessor including multiple processors, may be a multicore processor having multiple processor cores, or may have a configuration having multiple multicore processors.

[0027] The processor 10a may be any one of integrated circuits (ICs) such as Central Processing Units (CPUs), Micro Processing Units (MPUs), Graphics Processing Units (GPUs), Accelerated Processing Units (APUs), Digital Signal Processors (DSPs), Application Specific ICs (ASICs) and Field Programmable Gate Arrays (FPGAs), or combinations of two or more of these ICs.

[0028] For example, when the data modification apparatus 1 executes a machine learning process in addition to a data modification process according to the one embodiment, the processor 10a may be a combination of a processing device such as a CPU that executes the data modification process and an accelerator that executes the machine learning process. Examples of the accelerator include the GPUs, APUs, DSPs, ASICs, and FPGAs described above.

[0029] The memory 10b is an example of a HW device that stores various types of data and information such as a program. Examples of the memory 10b include one or both of a volatile memory such as a Dynamic Random Access Memory (DRAM) and a non-volatile memory such as Persistent Memory (PM).

[0030] The storing device 10c is an example of a HW device that stores various types of data and information such as program. Examples of the storing device 10c include a magnetic disk device such as a Hard Disk Drive (HDD), a semiconductor drive device such as a Solid State Drive (SSD), and various storing devices such as a nonvolatile memory. Examples of the nonvolatile memory include a flash memory, a Storage Class Memory (SCM), and a Read Only Memory (ROM).

[0031] The storing device 10c may store a program 10g (data modification program) that implements all or part of various functions of the computer 10.

[0032] For example, the processor 10a of the data modification apparatus 1 can achieve the functions of the data modification apparatus 1 (for example, the controlling unit 18 illustrated in FIG. 2) described below by expanding the program 10g stored in the storing device 10c onto the memory 10b and executing the expanded program 10g.

[0033] The IF device 10d is an example of a communication IF that controls connection and communication among various networks including a network between the data modification apparatus 1 and a non-illustrated apparatus. An example of the non-illustrated apparatus is a computer such as a user terminal or a server that provides data to the data modification apparatus 1, or a computer such as a server that carries out a machine learning process based on data outputted from the data modification apparatus 1.

[0034] For example, the IF device 10d may include an applying adapter conforming to Local Area Network (LAN) such as Ethernet (registered trademark) or optical communication such as Fibre Channel (FC). The applying adapter may be compatible with one of or both wireless and wired communication schemes.

[0035] Furthermore, the program 10g may be downloaded from the network to the computer through the communication IF and be stored in the storing device 10c.

[0036] The I/O device 10e may include one or both of an input device and an output device. Examples of the input device include a keyboard, a mouse, and a touch panel. Examples of the output device include a monitor, a projector, and a printer. Alternatively, the I/O device 10e may include, for example, a touch panel that integrates an input device with the output device.

[0037] The reader 10f is an example of a reader that reads data and programs recorded on a recording medium 10h. The reader 10f may include a connecting terminal or device to which the recording medium 10h can be connected or inserted. Examples of the reader 10f include an applying adapter conforming to, for example, Universal Serial Bus (USB), a drive apparatus that accesses a recording disk, and a card reader that accesses a flash memory such as an SD card. The program 10g may be stored in the recording medium 10h. The reader 10f may read the program 10g from the recording medium 1h and store the read program 10g into the storing device 10c.

[0038] The recording medium 10h is an example of a non-transitory computer-readable recording medium such as a magnetic/optical disk, and a flash memory. Examples of the magnetic/optical disk include a flexible disk, a Compact Disc (CD), a Digital Versatile Disc (DVD), a Blu-ray disk, and a Holographic Versatile Disc (HVD). Examples of the flash memory include a semiconductor memory such as a USB memory and an SD card.

[0039] The HW configuration of the computer 10 described above is exemplary. Accordingly, the computer 10 may appropriately undergo increase or decrease of HW devices (e.g., addition or deletion of arbitrary blocks), division, integration in an arbitrary combination, and addition or deletion of the bus.

[0040] FIG. 2 is a block diagram schematically illustrating an example of the functional configuration of the data modification apparatus 1 of the one embodiment. The data modification apparatus 1 is an exemplary information processing apparatus or computer that modifies data used to train a machine learning model. For example, the data modification apparatus 1 may modify data used to train a machine learning model by employing a method to suppress an unfair inference by a machine learning model.

[0041] The one embodiment may use a technique of a Disparate Impact Remover (DIR) as an exemplary method. The data modification apparatus 1 of the one embodiment suppresses the degradation in accuracy of an inference result caused by application of the DIR by, for example, individually changing a parameter used when rewriting values of a non-protected attribute for each attribute.

[0042] As illustrated in FIG. 2, the data modification apparatus 1 may illustratively include a memory unit 11, an obtaining unit 12, a causal graph generating unit 13, a data rewriting unit 14, and an outputting unit 15. The data modification apparatus 1 may include a machine learning unit 16, and may further include an inference processing unit 17. The obtaining unit 12, the causal graph generating unit 13, the data rewriting unit 14, the outputting unit 15 (and the machine learning unit 16 and the inference processing unit 17) are examples of a controlling unit 18.

[0043] The memory unit 11 is an example of a storing region and stores various data used by the data modification apparatus 1. The memory unit 11 may be achieved by, for example, a storing region that one or both of the memory 10b and the storing device 10c illustrated in FIG. 1.

[0044] As illustrated in FIG. 2, the memory unit 11 may illustratively be capable of storing data 11a, a protected attribute 11b, a parameter 11c, a causal graph lid, and modified data 11e. In addition, if the data modification apparatus 1 includes the machine learning unit 16, the memory unit 11 may be capable of storing a machine learning model 11f. Further, if the data modification apparatus 1 includes the inference processing unit 17, the memory unit 11 may be capable of storing an inference result 11g.

[0045] Hereinafter, for the sake of convenience, the information that the memory unit 11 stores is expressed in a table format, but the form of the information is not limited to this. At least one type of the information that the memory unit 11 stores may be in various formats such as a database (Database: DB) or an array.

[0046] The obtaining unit 12 obtains various types of information used in the data modification apparatus 1. For example, the obtaining unit 12 may obtain the data 11a, the protected attribute 11b, and the parameter 11c from a device (not illustrated) that provides data, and store them into the memory unit 11.

[0047] The data 11a is data containing multiple attributes, and is an example of training data used to train a machine learning model. Each of the multiple attributes may be a protected attribute or a non-protected attribute.

[0048] FIG. 3 is a diagram illustrating an example of data 11a. As illustrated in FIG. 3, the one embodiment assumes that the data 11a is adult data. Adult data is public data prepared on the basis of census data in the United States, and is data representing adult income. In the following description, it is assumed that the data 11a is used in a machine learning process for achieving a predetermined Artificial Intelligence (AI) task such as income prediction (prediction of whether “income” is “>=50 k”).

[0049] The protected attribute 11b is information for specifying (e.g., assigning) a second attribute among multiple attributes included in the data 11a. For example, the protected attribute 11b may include at least one of gender, age, race, nationality, and the like. In the example of FIG. 3, “sex”, which represents gender, is one of the protected attributes 11b.

[0050] The parameter 11c is information used when the values of a non-protected attribute except for the protected attribute 11b included in the data 11a are rewritten, and indicates the degree of rewriting the values of the non-protected attribute. For example, the parameter 11c may be one or more values. A non-protected attribute 11b is an example of a first attribute among multiple attributes included in the data 11a.

[0051] The parameter 11c may be, for example, similar to a parameter used to reduce correlation between a protected attribute and a non-protected attribute in a method for suppressing unfair inference made by a machine learning model. In one embodiment, the parameter 11c is an example of an initial value for modifying the values of a non-protected attribute.

[0052] FIG. 4 is a diagram illustrating an example of reducing correlation by using a Disparate Impact Remover (DIR). The horizontal axis of FIG. 4 indicates a value of a non-protected attribute, and the vertical axis indicates a probability distribution. The reference signs X (dashed line) and Y (dashed-dotted line) illustrated in FIG. 4 are probability density functions of a non-protected attribute for each value (e.g., gender: “male” and “female”) of a protected attribute 11b. Since the graphs represented by the reference signs X and Y indicates that distributions of the values of the non-protected attribute are deviated in accordance with the values of the protected attribute 11b, it can be said that the non-protected attribute has correlation with the protected attribute 11b.

[0053] The probability density function indicated by the reference sign Z (solid line) is a graph when the values of a non-protected attribute is uniformly rewritten using a single parameter 11c in a process using a normal DIR. The probability density function indicated by the reference sign Z is a function in which the correlation between a protected attribute 11b and a non-protected attribute are reduced as compared with the probability density functions indicated by the reference sign X and the reference sign Y.

[0054] FIG. 5 is a diagram illustrating an example of a reduction ratio of correlation when a causal graph is not used. FIG. 5 illustrates a case where a normal DIR is used as a case where a causal relation is not used. In FIG. 5, the parameter 11c is assumed to be “0.8”.

[0055] As illustrated in FIG. 5, when a non-protected attribute is modified on the basis of a single parameter 11c in the DIR, the correlation between each non-protected attribute and the protected attribute 11b is reduced at a uniform ratio. In this case, as described above, data may be changed for a non-protected attribute which has accidentally correlation with the protected attribute 11b, so that the accuracy in the inference result by a machine learning model trained with the data in question may be degraded.

[0056] As a solution to the above, the data modification apparatus 1 according to the one embodiment modifies the values of each non-protected attribute on the basis of the causal relation between the protected attribute 11b and the non-protected attribute that are correlated with each other. Accordingly, it is possible to suppress degradation in accuracy of an inference result by the machine learning model 11f trained with the data 11a (modified data 11e described below) including the modified values.

[0057] The causal relation between the protected attribute 11b and the non-protected attribute may mean a relationship between the cause and the result between these attributes. For example, having a casual relation may mean that the value of one attribute (the result) is caused by the value of the other attribute (the cause). In addition, the strength of the causal relation may mean one or the both of a possibility that these attributes have a causal relation and a degree of contribution of the value of one attribute to the other attribute. The strength of the causal relation may be referred to as the extent or the degree of the causal relation.

[0058] The causal graph generating unit 13 generates a causal graph (causal-effect graph) 11d, using the protected attribute 11b in the data 11a as an explanatory variable and the class to be classified as the response variable.

[0059] As an example, the causal graph generating unit 13 may execute causal estimation that estimates a matrix A representing causal relations between attributes included in the data 11a, using a trained machine learning model (not illustrated) for performing a causal search.

[0060] The causal graph 11d may be expressed, for example based on the matrix A estimated by the causal estimation. For example, the causal graph generating unit 13 may store the estimated matrix A, as the causal graph 11d, into the memory unit 11.

[0061] An example of a trained machine learning model for performing a causal search is a Linear Non-Gaussian Acyclic Model (LiNGAM). The causal estimation using a LiNGAM is formulated by the following Equations (1) to (3).

x=Ax+ε  (1)

x=(x.sub.1, x.sub.2, . . . ,x.sub.n).sup.T   (2)

ε=(ε.sub.1, ε.sub.2, . . . , ε.sub.n).sup.T   (3)

[0062] In Equations (2) and (3), the symbol “n” denotes the number of attributes (the attribute number) included in the data 11a. As an example, “n=11” is assumed to be satisfied. In the above Equation (2), “x.sub.i” (where, i is an integer between “1” and “n” both inclusive) indicates each attributes included in the data 11a. In the above Equation (3), the “ε.sub.i” denotes the noise of the non-Gaussian distribution.

[0063] FIG. 6 is a diagram illustrating an example of a causal graph 11d. The causal graph 11d is information in which the protected attribute 11b and non-protected attribute 11d1 are regarded as nodes, and an index 11d2, which indicates the strength of the causal relation between attributes, is associated with an edge (side) that connects the nodes (attributes). The causal graph 11d may be illustrated as a directed graph as exemplified in FIG. 6 and, in other instances, may be illustrated as the matrix A as described above.

[0064] In LiNGAM, an extrinsic variable and the response variable can be set in advance. The extrinsic variable corresponds to the root node of the causal graph 11d, and in the example of FIG. 6, is a protected attribute 11b “sex”. The response variable is a variable of which a causal relation with an extrinsic variable is to be estimated, and corresponds to a node at the end of the causal graph 11d. In the example of FIG. 6, the response variable is the “income” among the non-protected attributes 11d1.

[0065] The causal graph generating unit 13 may calculate the index 11d2 indicating the strength of the causal relation between the protected attribute 11b and each non-protected attribute 11d1 included in the data 11a on the basis of the data 11a and the protected attribute 11b, using the above Equations (1) to (3).

[0066] In the example of FIG. 6, the index 11d2 is illustrated on an edge connecting nodes. For example, the index 11d2 between “sex” and “edu_level” is “0.1”.

[0067] The data rewriting unit 14 adjusts the ratio of the parameter 11c to be applied to each non-protected attribute 11d1 on the basis of the causal graph 11d. The data rewriting unit 14 rewrites the values of the non-protected attribute 11d1 included in the data 11a at the adjusted ratio, and stores data 11a after the rewriting of the values into the memory unit 11 as modified data 11e.

[0068] This allows the data rewriting unit 14 to modify the values of the respective non-protected attributes 11d1, using an appropriate ratio depending on the causal relation between each non-protected attribute 11d1 and the protected attribute 11b. An exemplary process performed by the data rewriting unit 14 will now be described below.

[0069] For example, the data rewriting unit 14, based on the causal graph 11d, specifies, from the multiple attributes included in data 11a, a non-protected attribute 11d1 (hereinafter sometimes referred to as “modification-target non-protected attribute 11d1”) that has a causal relation with the protected attribute 11b among the multiple attributes.

[0070] A modification-target non-protected attribute 11d1 may be, for example, a non-protected attribute 11d1 for which an index 11d2 is set (calculated) with respect to the protected attribute 11b in the causal graph 11d.

[0071] In the example of FIG. 6, the modification-target non-protected attributes 11d are “marital-status”, “edu_level”, “occupation”, “relationship”, “hours-per-week”, and “income”.

[0072] On the other hand, no edge exists between the protected attribute 11b “sex” and the non-protected attribute 11d1 “workclass” (, which means these attributes are not directly connected). The absence of an edge means that the protected attribute “sex” may have correlation with the non-protected attribute “workclass”, but have no causal relation with “workclass”. In this case, “workclass” (the fourth attribute) is a non-protected attribute 11d1 that is not to be modified.

[0073] Alternatively, a non-protected attribute 11d1 having, among the non-protected attributes 11d1 each for which index 11d2 is set in the causal graph 11d, a causal relation with the protected attribute 11b may be limited to a non-protected attribute 11d1 having index 11d2 equal to or larger than a given threshold value. In other words, the data rewriting unit 14 may determine a non-protected attribute 11d1 having an index 11d2 less than the given threshold value to be a non-protected attribute 11d1 not to be modified among non-protected attributes 11d1 each for which the index 11d2 are set in the causal graph 11d.

[0074] FIG. 7 is a diagram illustrating an example of the reduction ratio 14a of correlation on the basis of the causal graph 11d. In FIG. 7, the parameter 11c is assumed to be “0.8”. As illustrated in FIG. 7, the data rewriting unit 14 calculates a reduction ratio 14a to be applied to the values of a non-protected attribute 11d1 on the basis of the parameter 11c and the index 11d2 set between the non-protected attribute 11d1 and the protected attribute 11b for each modification-target non-protected attribute 11d1.

[0075] A reduction ratio 14a may be a product of the parameter 11c and the index 11d2. Alternatively, the reduction ratio 14a may be a result of any calculation using the parameter 11c and the index 11d2.

[0076] In the example of FIG. 7, the data rewriting unit 14 calculates the reduction ratio 14a of the non-protected attribute 11d1 “edu_level” to be “0.8×0.1=0.08”, which is the multiplication result of the parameter 11c “0.8” and the index 11d2 “0.1” between the non-protected attribute 11d1 “edu_level” and the protected attribute 11b “sex”.

[0077] The data rewriting unit 14 modifies values of multiple non-protected attributes 11d1 included in the data 11a, using the reduction ratios 14a calculated for the respective non-protected attributes 11d1, and stores the data 11a after the modification as the modified data 11e into the memory unit 11. Each of the non-protected attributes 11d1 is an example of a third attribute.

[0078] The data rewriting unit 14 may modify the values of a non-protected attribute 11d1 in the data 11a, for example, according to a condition for reducing differences in the probability distributions of the values of the non-protected attribute 11d1, the probability distributions being one for each value of the protected attribute 11b. In other words, the data rewriting unit 14 may modify values of the non-protected attribute 11d1 in the training data in accordance with a condition for reducing a difference between distributions of the values of the non-protected attribute 11d1 corresponding to each value of the protected attribute 11b.

[0079] The condition is, for example, a condition that the values of a non-protected attribute 11d1 having a stronger causal relation with the protected attribute 11b are reduced at a higher degree, or a condition that the values of a non-protected attribute 11d1 having a weaker causal relation with the protected attribute 11b are reduced at a lower degree. In other words, for example, the condition includes a condition that more intensively reduces a difference between distributions of values of a non-protected attribute 11d1 (third attribute), the non-protected attribute 11d1 having a stronger causal relation with the protected attribute 11b than the causal relation between a non-protected attribute 11d1 (first attribute) and the protected attribute 11b (second attribute).

[0080] For example, in FIG. 7, the non-protected attribute 11d1 “Marital_status” having an index 11d2 of “0.8” has a stronger causal relation with the protected attribute 11b than the non-protected attribute 11d1 “edu_level” having an index 11d2 of “0.1”. In this case, the data rewriting unit 14 may modify (e.g., by reducing) the values of “Marital_status” at a larger degree than the values of “edu_level”.

[0081] For example, the data rewriting unit 14 may use a result of multiplying the value of a non-protected attribute 11d1 and a value of “1−(calculated reduction ratio)” as the value (the modified value) after the modification of the non-protected attribute 11d1. The manner of modifying the data 11a using reduction ratio 14a is not limited to the above-described example, and various manners may be adopted in accordance with a manner of calculating the reduction ratio 14a.

[0082] The outputting unit 15 outputs the output data. An example of the output data is the modified data 11e. In addition to the modified data 11e, the output data may include one or both of a machine learning model 11f and an inference result 11g that are to be described below.

[0083] In the “outputting” of the output data, the outputting unit 15 may transmit (provide) the output data to another non-illustrated computer, or may store the output data in memory unit 11 to manage the output data to be obtainable from the data modification apparatus 1 or another computer. Alternatively, in the “outputting” of the output data, the outputting unit 15 may output the information indicating the output data on the screen of an output device, for example, the data modification apparatus 1, or may alternatively output the output data in various other manners.

[0084] As described above, the data modification apparatus 1 may include a machine learning unit 16, and may further include an inference processing unit 17.

[0085] In a machine learning phase, the machine learning unit 16 executes a machine learning process that trains the machine learning model 11f on the basis of the modified data 11e including the values of the non-protected attribute 11d1 modified using the reduction ratio 14a. The machine learning model 11f may be a Neural Network (NN) model that includes parameters having been subjected to machine learning. The machine learning process may be implemented by various known techniques.

[0086] In the inferring phase, the inference processing unit 17 carries out an inference process using the machine learning model 11f trained on the basis of the modified data 11e. For example, the inference processing unit 17 inputs target data (not illustrated) of the inference process into the machine learning model 11f, and stores an inference result 11g outputted from the machine learning model 11f into the memory unit 11.

[0087] Next description will now be made in relation to an example of operation of the data modification apparatus 1 of the one embodiment. FIG. 8 is a flow diagram schematically illustrating an example of operation of the data modification apparatus 1 of the one embodiment.

[0088] As illustrated in FIG. 8, the obtaining unit 12 of the data modification apparatus 1 obtains the data 11a, the protected attribute 11b, and the parameter 11c (Step S1), and stores them into the memory unit 11.

[0089] The causal graph generating unit 13 generates a causal graph 11d based on the data 11a and the protected attribute 11b (Step S2), and stores the causal graph into the memory unit 11.

[0090] The data rewriting unit 14 selects an unselected non-protected attribute 11d1 among the non-protected attributes 11d1 in the data 11a (Step S3).

[0091] The data rewriting unit 14 determines whether or not the selected non-protected attribute 11d1 is a non-protected attribute 11d1 having a causal relation with the protected attribute 11b on the basis of the causal graph 11d (Step S4). For example, the data rewriting unit 14 may determine whether or not an index 11d2 exists between the selected non-protected attribute 11d1 and the protected attribute 11b (or whether or not the index 11d2 is equal to or larger than a given threshold) on the basis of the causal graph 11d.

[0092] If the selected non-protected attribute (third attribute) 11d1 is determined to have a causal relation with the protected attribute 11b (YES in Step S4), the process proceeds to Step S5. On the other hand, if the selected non-protected attribute 11d1 is determined not to have a causal relation with a protected attribute 11b (NO in Step S4), the process proceeds to Step S6.

[0093] In Step S5, the data rewriting unit 14 adjusts the parameter 11c on the basis of the causal relation between the selected non-protected attribute 11d1 and the protected attribute 11b, and then the process proceeds to Step S6. As an example, the data rewriting unit 14 may calculate the reduction ratio 14a based on the index 11d2, which indicates the strength of the causal relation between the selected non-protected attribute 11d1 and protected attribute 11b, and parameter 11c.

[0094] In Step S6, the data rewriting unit 14 determines whether or not an unselected non-protected attribute 11d1 is left among the non-protected attributes 11d1 in data 11a. If an unselected non-protected attribute 11d1 is determined to be left (YES in Step S6), the process proceeds to Step S3.

[0095] If an unselected non-protected attribute 11d1 is determined not to be left (NO in Step S6), the data rewriting unit 14 executes a DIR for modifying values of each non-protected attribute 11d1 included in the data 11a on the basis of the reduction ratio 14a calculated in Step S5 (Step S7).

[0096] The outputting unit 15 outputs the modified data 11e generated by the data rewriting unit 14 executing the DIR (Step S8), and the process ends.

[0097] In data modification apparatus 1 according to an embodiment, the controlling unit 18 specifies, from the multiple attributes included in, a non-protected attribute 11d1 that has a causal relation with the protected attribute 11b among the multiple attributes. In addition, the controlling unit 18 modifies the values of the non-protected attribute 11d1 of the data 11a, for example, according to a condition for reducing differences in the probability distribution of the values of the non-protected attribute 11d1 for each value of the protected attributes 11b.

[0098] As the above, according to the data modification apparatus 1, the values of a non-protected attribute 11d1 having a causal relation with the protected attribute 11b can be modified. This can suppress the modification of the value of a non-protected attribute 11d1 which (e.g., accidentally) has correlation with the protected attribute 11b but which has no causal relation with the protected attribute 11b.

[0099] Further, according to the data modification apparatus 1, the value of a non-protected attribute 11d1 can be modified to an appropriate value according to the condition. For example, in reducing the correlation between the protected attribute 11b and a non-protected attribute 11d1, the data modification apparatus 1 can adjust the amount of reduction in the correlation in accordance with the strength of the causal relation between the protected attribute 11b and the non-protected attribute 11d1 in question. Consequently, as compared with a case where multiple non-protected attributes 11d1 are uniformly modified on the basis of the parameter 11c, it is possible to suppress degradation of the accuracy of the inference result caused by machine learning model 11f trained with the modified data 11e.

[0100] As described above, according to the data modification apparatus 1 of the one embodiment, it is possible to appropriately adjust (e.g., set to a minimum) a range and a degree of modification of the data 11a, and to generate modified data 11e in which biases such as discrimination are mitigated.

[0101] FIG. 9 is a diagram illustrating an example of an inference result obtained with a machine learning model 11f trained by modified data 11e according to the one embodiment. The horizontal axis of FIG. 9 indicates the fairness, and the vertical axis indicates the accuracy. The shaded circles are plots of an example of an inference result obtained with the machine learning model 11f. The white circles are plots of an inference result obtained with a machine learning model trained with the data generated by a normal DIR (DIR using the parameter 11c illustrated in FIG. 5) serving as a comparative example.

[0102] According to the method of the one embodiment, as illustrated by the shaded circles, it is possible to suppress the degradation of the accuracy of an inference result (or to improve the accuracy) while ensuring the fairness of the inference result as compared with the white circles.

[0103] The technique according to the one embodiment described above can be implemented by changing or modifying as follows.

[0104] For example, the obtaining unit 12, the causal graph generating unit 13, the data rewriting unit 14 and the outputting unit 15 (and the machine learning unit 16 and the inference processing unit 17) included in the data modification apparatus 1 illustrated in FIG. 2 may be merged at any combination, or may each be divided.

[0105] In addition, the data modification apparatus 1 illustrated in FIG. 2 may have a configuration (system) that achieves each processing function by multiple apparatuses cooperating with each other via a network. As an example, the memory unit 11 may be a DB server; the obtaining unit 12 and the outputting unit 15 may be a Web server or an application server; the causal graph generating unit 13, the data rewriting unit 14, the machine learning unit 16, and the inference processing unit 17 may be an application server. In this case, the processing function as the data modification apparatus 1 may be achieved by the DB server, the application server, and the web server cooperating with one another via a network.

[0106] The one embodiment assumes that one (gender “sex”) among the multiple attributes included in the data 11a is the protected attribute 11b, but the number of protected attributes 11b is not limited to one. Alternatively, the data may include multiple protected attributes 11b.

[0107] Here, the data modification apparatus 1 may generate a causal graph 11d for each protected attribute 11b.

[0108] Furthermore, the data modification apparatus 1 may generate the modified data 11e for each protected attribute 11b. Alternatively, the data modification apparatus 1 may generate one set of the modified data 11e related to two or more protected attributes 11b by combining (e.g., multiplying) the respective reduction ratios 14a of the two or more protected attributes 11b for each non-protected attribute 11d1.

[0109] As one aspect, the one embodiment can suppress the degradation in accuracy of an inferring result made by a machine learning model.

[0110] Throughout the descriptions, the indefinite article “a” or “an” does not exclude a plurality.

[0111] All examples and conditional language recited herein are intended for the pedagogical purposes of aiding the reader in understanding the invention and the concepts contributed by the inventor to further the art, and are not to be construed limitations to such specifically recited examples and conditions, nor does the organization of such examples in the specification relate to a showing of the superiority and inferiority of the invention. Although one or more embodiments of the present inventions have been described in detail, it should be understood that the various changes, substitutions, and alterations could be made hereto without departing from the spirit and scope of the invention.