Method for Providing an Explanation Dataset for an AI Module, Computer-Readable Storage Medium, Device and System

Abstract

Explaining the decisions of AI modules to a user is difficult.

The invention relates to methods for providing an explanation dataset (2) for an AI module (31), the methods comprising: receiving a user dataset (20) which specifies at least one input dataset (21) of an AI module (31), wherein the AI module (31) is adapted to compute an output dataset (3) for the input dataset (21), wherein the user dataset (20) comprises at least one target specification (25) which specifies a value of a data item (26) in an output dataset (3) of the AI module (31); loading at least one optimization task (16) which specifies a specific metric (14) and/or a similarity metric (15); computing at least one solution of the at least one optimization task (16) as an explanation dataset (2) taking the user dataset (20) and the AI module (31) into consideration and applying at least one optimization method (17), wherein the AI module (31) is adapted to compute for the explanation dataset (2) an output dataset (3) which comprises the data item (26) specified by the target specification (25); providing the explanation dataset (2) for the AI module (31).

Claims

1. A method for providing an explanation dataset for an AI module, the method comprising: receiving a user dataset specifying at least one input dataset of an AI module, wherein the AI module is adapted to compute an output dataset for the input dataset using at least one of a regression and a classification, wherein the user dataset comprises at least one target specification specifying a value of a data item in an output dataset of the AI module; loading at least one optimization task specifying a specific metric and/or a similarity metric; computing at least one solution of the at least one optimization task as an explanation dataset, based on the user dataset and the AI module, by at least applying at least one optimization method, wherein the AI module is adapted to compute for the explanation dataset an output dataset comprising the data item specified by the target specification; and providing the explanation dataset for the AI module.

2. The method of claim 1, wherein the user dataset comprises at least one constraint for the at least one optimization task, wherein the optimization method computes the at least one optimization task based on the at least one constraint of the user dataset.

3. The method of claim 2, wherein the at least one constraint comprises an allowance specification, wherein the allowance specification specifies in which feature categories defined by the input dataset the explanation dataset differs from the input dataset.

4. The method of claim 2, wherein the at least one constraint comprises at least one weight, wherein a weight specifies a preference for a change of a feature category of the input dataset in the explanation dataset.

5. The method of claim 2, wherein the at least one constraint comprises at least one range specification, wherein the at least one range specification specifies a permitted value range of a feature category of the explanation dataset, wherein the permitted value range includes a maximum and/or minimum permitted deviation from a value in the input dataset.

6. The method of claim 2, wherein the explanation dataset comprises a plurality of variations of the input dataset, each satisfying the at least one constraint.

7. The method of claim 1, further comprising receiving at least one provider dataset, wherein the provider dataset comprises at least one constraint for the at least one optimization task, and wherein the optimization method computes the at least one optimization task based on the at least one constraint of the provider dataset.

8. The method of claim 7, wherein the at least one constraint of the provider dataset specifies an output count, and wherein the output count specifies how many variations of the input dataset are computed and comprised by the explanation dataset.

9. The method of claim 1, wherein the at least one optimization method comprises a gradient method and/or a Newton method.

10. The method of claim 1, wherein the specific metric is minimal if the target specification matches the data item of the output dataset of the AI module, and wherein the specific metric is formed as cross entropy and/or as mean square deviation.

11. The method of claim 1, wherein the similarity metric is adapted as an Lpnorm including an L0, L1 and/or L2 metric.

12. The method of claim 2, wherein the optimization task is given by the formula $\min_{δ} M_{sp} (δ) + λ M_{i m} (δ),$ wherein M.sub.sp specifies the specific metric and M.sub.im specifies the similarity metric, and δ is selected from a set of the allowable changes of the input dataset.

13. The method of claim 1, further comprising-computing an output dataset using the AI module, wherein the explanation dataset is used as an input dataset of the AI module.

14. A computer-readable storage medium containing instructions, which when executed by at least one processor, result in operations comprising: receiving a user dataset specifying at least one input dataset of an AI module, wherein the AI module is adapted to compute an output dataset for the input dataset using at least one of a regression and a classification, wherein the user dataset comprises at least one target specification specifying a value of a data item in an output dataset of the AI module; loading at least one optimization task specifying a specific metric and/or a similarity metric; computing at least one solution of the at least one optimization task as an explanation dataset, based on the user dataset and the AI module, by at least applying at least one optimization method, wherein the AI module is adapted to compute for the explanation dataset an output dataset comprising the data item specified by the target specification; and providing the explanation dataset for the AI module.

15. A device for providing an explanation dataset, comprising: a receiving unit adapted to receive a user dataset specifying at least one input dataset of an AI module, wherein the AI module is adapted to compute an output dataset for the input dataset using at least one of a regression and a classification, wherein the user dataset comprises at least one target specification specifying a value of a data item in an output dataset of the AI module; an optimization unit adapted to load an optimization task specifying a specific metric and/or a similarity metric and further adapted to compute at least one solution of the at least one optimization task as an explanation dataset, based on the user dataset and the AI module, using at least one optimization method, wherein the AI module is adapted to compute for the explanation dataset an output dataset comprising the data item specified by the target specification; and a providing unit adapted to provide the explanation dataset.

16. The device of claim 14, further comprising: an AI unit adapted to compute an output dataset, wherein the explanation dataset is used as an input dataset of the AI module.

17. A system, comprising: at least one server unit, comprising the device of claim 15 and a server communication interface; and at least one client unit having a client communication interface, adapted to send a request to the server communication interface, via a communication network; wherein the server communication interface is adapted to provide an application programmable interface adapted to receive a user dataset and transmit an explanation dataset.

Description

DESCRIPTION OF DRAWINGS

[0063] In the following, the invention is explained in more detail by means of exemplary embodiments, wherein:

[0064] FIG. 1: shows a schematic representation of a system;

[0065] FIG. 2: shows a schematic representation of the operation of an AI module with an explanation dataset;

[0066] FIG. 3: shows a schematic representation of a device for providing an explanation dataset;

[0067] FIG. 4: shows a schematic representation of an optimization unit;

[0068] FIG. 5: shows an example of an input dataset;

[0069] FIG. 6: shows an example of an explanation dataset;

[0070] FIG. 7: shows a schematic representation of a distributed system.

[0071] In the following, the same reference number is used for identical or similarly acting parts.

DETAILED DESCRIPTION

[0072] FIG. 1 shows a schematic representation of a system 1 which determines an explanation dataset 2 for a user dataset 20. The system 1 has a device 10 which is adapted to determine the explanation dataset 2 taking into account the user dataset 20 and/or a provider dataset 30. Such a device 10 may also be referred to as a counterfactory 10.

[0073] The user dataset 20 has an input dataset 21. The input dataset 21 includes a plurality of data items that form an input to an AI module 31.

[0074] In one exemplary embodiment, the input dataset 21 may comprise image data, wherein the data items represent brightness values for pixels. In another exemplary embodiment, the input dataset 21 may comprise characteristics of a bank customer, wherein the data items of the input dataset 21 may indicate, for example, the customer's income, occupation, and age.

[0075] The user dataset 20 further has an allowance specification 22 which specifies in which feature categories the explanation dataset 2 may differ from the input dataset 21. Thus, in the exemplary embodiment shown, the user has the ability to specify which features are allowed to change and which are not. The allowance specification 22 may therefore also be regarded as a blacklist or whitelist. For example, in the aforementioned example of a bank customer, the user can specify that the characteristic “age” in the explanation dataset 2 is not allowed to change, since the user has no control over it. The allowance specification 22 may be specified as a vector, wherein the number of dimensions of the vector corresponds to the number of feature categories of the input dataset 21. Each data item of the vector can indicate whether a feature category may be changed.

[0076] In addition, the user dataset 20 comprises at least one weight 23, which in the exemplary embodiment shown indicates a preference as to which feature categories in the explanation dataset 2 should preferably be changed or which should not, without blocking them completely. Thus, in the example of the bank customer, the bank customer could indicate that a job change should be more likely than an increase in income in the explanation dataset 2.

[0077] Further, in the exemplary embodiment shown, the user dataset 20 includes at least one range specification 24 indicating the ranges within which the variation of a data item of the input dataset 21 is allowed to move. This is useful when certain variations are not possible. For example, in the case of an input dataset 21 specifying image data, the range specification 24 can be used to ensure that a variation of brightness values is again a permissible brightness value, for example, in the range from 0 to 255.

[0078] Finally, the user dataset 20 comprises a target specification 25 indicating the desired result to be determined by the AI module 31. For example, in a classification example, the target specification 25 may specify a class. In the case of a regression example, the target specification 25 may specify a particular value. For example, in the example of the bank customer described above, the target specification 25 may indicate that a loan is to be granted.

[0079] The provider dataset 30 comprises the AI module 31, which is adapted to perform a classification and/or regression. The AI module 31 may be a software component provided to the device 10. For example, the AI module 31 may be provided as a library of an object-oriented programming language. However, it is also conceivable that the AI module 31 is provided via an application programmable interface (API). In this case, a description of the API is provided to the device 10 as the AI module 31.

[0080] The AI module 31 may be any AI module. For example, an AI module trained according to the principles of supervised learning and/or non-supervised learning. For example, the AI module 31 may be an artificial neural network. However, any other implementation of an AI module is conceivable as long as it performs regression and/or classification for an input dataset 21.

[0081] In the exemplary embodiment shown, the provider dataset 30 further comprises an allowance specification 32 that, like the allowance specification 22 of the user dataset 20, includes an indication of which feature categories may be modified. The provider of the AI module 31 can thus, for example, prevent changes to certain feature categories from being suggested as explanations, such as a skin color.

[0082] Furthermore, the provider dataset 30 has an output count 33 indicating how many different explanations or variations the explanation dataset 2 should comprise. This can ensure that the user only receives a manageable number of explanations.

[0083] Further, in the exemplary embodiment shown, the provider dataset 30 includes a deviation specification 34. The deviation specification 34 indicates the minimum degree to which the explanation dataset 2 must deviate from the input dataset 21. For example, a declaration that a loan would have been granted if the salary had been increased by a few cents could have a very negative effect on a customer. It is thus possible to specify that a certain minimum change should be included in the explanation dataset 2.

[0084] Overall, the allowance specification 22, the at least one weight 23, the range specification 24, the target specification 25, the allowance specification 32, the output count 33, and the deviation specification 34 define constraints that are considered by the device 10 when providing the explanation dataset 2.

[0085] FIG. 2 shows a schematic representation of the result of processing the explanation dataset 2 by an AI module 31. In FIG. 2, it is schematically shown that an explanation dataset 2 provided by a device/counterfactory 10 can be used as an input dataset for an AI module 31 so that it determines an output dataset 3. The output dataset 3 specifies a data item 26, which may specify a regression result or a classification result. In the exemplary embodiment of FIG. 2, the data item 26 corresponds to the target specification 25.

[0086] FIG. 3 shows a schematic detailed view of the device/counterfactory 10. The counterfactory 10 receives a user dataset 20 and/or a provider dataset 30 through a receiving unit 11. An optimization unit 12 is adapted to determine an explanation dataset 2 using the user dataset 20 and the provider dataset 30, which is provided by a providing unit 13.

[0087] The operation of the optimization unit 12 is shown in more detail in FIG. 4. The optimization unit 12 is adapted to define an optimization task 16 using a specific metric 14 and a similarity metric 15. This optimization task 16 is solved in the solver unit 18 using an optimization method 17 and the constraints specified by the user dataset 20 or provider dataset 30, wherein the solution is provided as an explanation dataset 2.

[0088] An optimization task 16 is given by the formula:

[00002] $\min_{δ} M_{s p} (δ) + λ M_{i m} (δ),$

[0089] wherein M.sub.sp indicates the specific metric 14 and M.sub.im the similarity metric 15, and δ is selected from the set of allowable changes in the input dataset 21.

[0090] A plurality of optimization tasks 16 can also be determined by choosing different specific metrics. For example, the specific metric 14 can be formed as a cross entropy or as a mean square deviation. The similarity metric 15 can be formed as L0, L1 or as L2 norm.

[0091] Thus, many optimization tasks 16 can be defined by combining the different metrics. For example, a first optimization task can use the cross entropy as specific metric 14 and the L0, L1 or as L2 norm as similarity metric 15. A second optimization task may use the mean square deviation as specific metric 14 and the L0, L1 or as L2 norm as similarity metric 15.

[0092] By combining different concrete metrics as specific metric 14 and similarity metric 15, respectively, different possible explanations encompassed by the explanation dataset 2 are computed by solving the formula shown above.

[0093] Furthermore, different optimization methods 17 can additionally or alternatively be used to solve the optimization tasks 16 so that an even larger number of explanations can be determined to be provided as an explanation dataset 2.

[0094] FIGS. 5 and 6 show an example of an input dataset 21 and an explanation dataset 2 for a customer of a bank who wishes to obtain a loan. With the input dataset 21 shown, an AI module 31 determines that the customer will not receive a loan. The AI module 31 thus performs a classification. The input dataset 21 includes the feature categories AGE 41, INCOME 41′, and OCCUPATION 41″. Each of the characteristic categories 41, 41′, 41″ is assigned a value 42, 42′, 42″.

[0095] FIG. 6 shows an explanation dataset 2 provided by the device/counterfactory 10. A target specification 25 is passed to the device 10 as part of the user dataset 20, such that the classification by the AI module 31 should result in a loan being granted by modifying the data item 42, 42′, 42″. As a constraint, it is passed to the device/counterfactory 10 as an allowance specification 22 that only the data items of the feature categories INCOME 41′ and OCCUPATION 41″ may be modified.

[0096] The explanation dataset 2 is essentially the same as the input dataset 21. Only in the characteristic category INCOME 41′ is the value changed. The user thus receives with the explanation dataset 2 a value for the income which is necessary to obtain a classification by the AI module 31 with otherwise unchanged characteristics, so that the loan is granted.

[0097] FIG. 7 shows a distributed system 4 that includes a server 50 and a client 60. The server 50 and the client 60 can communicate via a communication network 70, such as the Internet. For this purpose, the client 60 has a client communication interface 63 communicatively connected to a client computer unit 62. The client 60 further comprises a client storage unit 61 adapted to store an input dataset 21.

[0098] The server 50 includes a server communication interface 53 communicatively connected to a server computer unit 52. The server computer unit 52 is adapted to execute a program that implements the counterfactory 10.

[0099] In the exemplary embodiment shown, the functionality of the counterfactory 10 is provided by means of an API via the server communication interface 53. This means that the client 60 is adapted to transmit a user dataset 20 to the server 50 or the server communication interface 53 via an API call. The server 50 or server computer unit 52 loads a provider dataset 30 from a server storage unit 51. Additionally or alternatively, the server 50 may receive the provider dataset 30 from a second client via the server communication interface 53.

[0100] The server computer unit 52 is further adapted to determine an explanation dataset 2 in consideration of the user dataset 20 and the provider dataset 30, and to transmit the explanation dataset 2 to the client 60 via the server communication interface 53.

[0101] By using an API it is also possible to enable so-called continuous auditing. This means that the function of the AI module 31 can be checked at any time.

[0102] In one exemplary embodiment, it is also possible for the server 50 to execute the AI module 31 and store the results, i.e., the respective output datasets 3. A user or client 60 may then query an explanation dataset 2 at a later time. In this context, it is optionally possible that a state of the AI module 31 used is also stored in the server memory unit 51 for the respective output datasets 3, so that different versions of the AI module 31 can be traced over time. It may be advantageous to store a hash value for the state of the AI module 31.

LIST OF REFERENCE SIGNS

[0103] 1 System [0104] 2 Explanation dataset [0105] 3 Output dataset [0106] 4 Distributed system [0107] 10 Fixture/Counterfactory [0108] 11 Receiving unit [0109] 12 Optimization unit [0110] 13 Providing unit [0111] 14 Specific metric [0112] 15 Similarity metric [0113] 16 Optimization task [0114] 17 Optimization method [0115] 18 Solver [0116] 20 User dataset [0117] 21 Input dataset [0118] 22 Allowance specification [0119] 23 Weight [0120] 24 Range specification [0121] 25 Target specification [0122] 26 Data item [0123] 30 Provider dataset [0124] 31 AI module [0125] 32 Allowance specification/Whitelist [0126] 33 Output count [0127] 34 Deviation specification [0128] 41, 41′, 41″ Feature category [0129] 42, 42′, 42′,42′″ Data item [0130] 50 Server [0131] 51 Server storage unit [0132] 52 Server computer unit [0133] 53 Server communication interface/API [0134] 60 Client [0135] 61 Client storage unit [0136] 62 Client computer unit [0137] 63 Client communication interface [0138] 70 Communication network

Method for Providing an Explanation Dataset for an AI Module, Computer-Readable Storage Medium, Device and System

Inventors

Cpc classification

Classification Explorer

G06F18/214

PHYSICS

Classification Explorer

G06N20/00

PHYSICS

Classification Explorer

G06Q40/03

PHYSICS

Classification Explorer

G06F18/22

PHYSICS

Classification Explorer

G06N5/045

PHYSICS

International classification

Classification Explorer

G06N20/00

PHYSICS

Classification Explorer

G06N5/04

PHYSICS

Abstract

Claims

Description