Test apparatus for performing a test on a device under test and data set filter for filtering a data set to obtain a best setting of a device under test

Abstract

A test apparatus for performing a test on a device under test includes a data storage unit being configured to store sets of input data applied to the device under test during the test and to store the respective output data of the device under test, the output data being obtained from the device under test as a response to the input data including values of setting variables related to settings of the device under test and values of input variables including further information, each set of input data representing one test case; and a data processor configured to process the data stored in the data storage unit such that a best combination of setting variables of the device under test is determined for one or more combinations of the input variables to obtain an optimized setting of the device under test for the one or more combinations of the input variables.

Claims

1. A method of performing a test on a device under test, the method comprising: storing sets of input data applied to the device under test during the test in a memory; storing in the memory output data derived from the device under test as a response of the device under test to the input data, wherein the input data comprises values of setting variables related to settings of the device under test and values of input variables comprising known information, wherein each set of input data represents one test case; and processing the data stored in the memory using a hardware processor, wherein the hardware processor configured to: determine a select combination of the setting variables of a plurality of candidate combinations of the setting variables of the device under test for testing one or more combinations of the input variables; acquire a selected setting of the device under test of a plurality of candidate settings for testing the one or more combinations of the input variables; perform a quantization of at least one input variable or at least one setting variable of the input data to form a discrete representation of the input data; process the discrete representation of the input data to determine the selected setting of the device under test for testing the one or more combinations of the input variables, wherein the selected setting is based on a selection target, wherein the selection target is a function of output variables, and further wherein values of the output variables form the output data; and assign a unique value to each candidate combination of the input variables or to each candidate combination of the setting variables.

2. A test apparatus for performing a test on a device under test, the test apparatus comprising; a memory and a hardware processor coupled to the memory; the memory configured to store sets of input data applied to the device under test during the test and to store output data of the device under test, wherein the output data is acquired from the device under test as a response of the device under test to the input data, wherein the input data comprises values of setting variables related to settings of the device under test and values of input variables comprising further information, wherein each set of input data represents one test case; and the hardware processor coupled to the memory and configured to: process the data stored in the memory; determine a select combination of the setting variables of a plurality of candidate combinations of the setting variables of the device under test for testing one or more combinations of the input variables; acquire a selected setting of the device under test of a plurality of candidate settings for testing the one or more combinations of the input variables; perform a quantization of at least one input variable or at least one setting variable of the input data to form a discrete representation of the input data; and process the discrete representation of the input data to determine the selected setting of the device under test for testing the one or more combinations of the input variables, wherein the selected setting is based on a selection target, wherein the selection target is a function of output variables, and further wherein values of the output variables form the output data; and assign a unique value to each candidate combination of the input variables or to each candidate combination of the setting variables, wherein the memory comprises the data storage unit and wherein the hardware processor comprises the data processor.

3. The test apparatus according to claim 2, wherein the hardware processor further configured to generate a standardized selection target based on the selection target, wherein a small value of the standardized selection target is associated with setting of the device under test, wherein the hardware processor configured to determine the selected setting of the device under test based on the standardized selection target.

4. The test apparatus according to claim 2, wherein the data hardware processor further configured to assign the unique value to each candidate combination of the input variables using the formula: $n\left(k\right)=1+\underset{p=1}{\overset{P}{.Math.}}\left(n_p\left(k\right)-1\right)\underset{i=1}{\overset{p-1}{.Math.}}{N}_i$ where k is a current test case, n(k) is a unique value of each candidate combination of the input variables, np(k) is an index representing possible values of an input variable p, P is a total number of input variables p, and N/ is a total number of possible values of input variable i; and wherein the hardware processor further configured to assign the unique value to each candidate combination of the setting variables using the formula: $m\left(k\right)=1+\underset{p=1}{\overset{P}{.Math.}}\left(m_p\left(k\right)-1\right)\underset{j=1}{\overset{p-1}{.Math.}}{M}_j$ where k is a current test case, m(k) is a unique value of each candidate combination of the setting variables, mp(k) is an index representing possible values of a setting variable p, P is a total number of setting variables p, and Mj is a total number of possible values of setting variable j.

5. The test apparatus according to claim 2, wherein the data hardware processor further configured to assign indices representing test cases to a respective combination of input variables of each possible combination of input variables and to a respective combination of the setting variables of each possible combination of the setting variables.

6. The test apparatus according to claim 5, wherein the hardware processor further configured to form the assignment of indices in a table, wherein a position along a first dimension of the table indicates a unique value of a current input variable within all possible combinations of input variables or wherein the hardware processor further configured to form the assignment of indices in a table, wherein a position along a second dimension of the table indicates a unique value of a current setting variable within all possible combinations of the setting variables.

7. The test apparatus according to claim 2, wherein the hardware processor further configured to: assign a result of the selection target for each test case to a respective combination of input variables of each possible combination of input variables and to a respective combination of the setting variables of each possible combination of the setting variables; and responsive to a determination that multiple results of test cases are assigned to a same combination of input variables and a same combination of output variables, determine a single value representing the multiple results.

8. The test apparatus according to claim 7, wherein the hardware processor further configured to calculate the single value representing the multiple results of the selection target using one of: a minimum value of the multiple results of optimization target, a maximum value of the multiple results of the optimization target, and an average value of the multiple results of the selection target.

9. The test apparatus according to claim 2, wherein the hardware processor further configured to determine an index of the select combination of setting variables for the one or more candidate combinations of input variables, wherein the index indicates a unique value of the select combination of setting variables within all possible candidate combinations of values of the setting variables.

10. The test apparatus according to claim 2, wherein the further information comprises at least one of: a stimulus waveform, an input pattern, a stimulus signal, an environment condition, and an environment status.

11. The test apparatus according to claim 2, further comprising: a data set filter for filtering a data set to acquire the selected setting of the device under test for the input variables using a selection vector, wherein the selection vector represents a relation between an index of input variables and an index of a combination of the setting variables and indicates the selected setting of the device under test.

12. A non-transitory digital storage medium having a computer program stored thereon to perform the method of performing a test on a device under test, the method comprising: storing in a memory sets of input data applied to the device under test during the test; storing in the memory output data derived from the device under test as a response of the device under test to the input data, wherein the input data comprises values of setting variables related to settings of the device under test and values of input variables comprising known information, wherein each set of input data represents one test case; and processing the data stored in the memory using a hardware processor, wherein the hardware processor configured to: determine a select combination of the setting variables of a plurality of candidate combinations of the setting variables of the device under test for testing one or more combinations of the input variables; acquire a selected setting of the device under test of a plurality of candidate settings for testing the one or more combinations of the input variables; perform a quantization of at least one input variable or at least one setting variable of the input data to form a discrete representation of the input data; and process the discrete representation of the input data to determine the selected setting of the device under test for testing the one or more combinations of the input variables, wherein the selected setting is based on a selection target, wherein the selection target is a function of output variables, and further wherein values of the output variables form the output data; and assign a unique value to each candidate combination of the input variables or to each candidate combination of the setting variables.

13. The method of claim 12, wherein the hardware processor s further configured to generate a standardized selection target based on the selection target, wherein a small value of the standardized selection target is associated with setting of the device under test, wherein the hardware processor configured to determine the selected setting of the device under test based on the standardized selection target.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

(1) Embodiments of the present invention will be detailed subsequently referring to the appended drawings, in which:

(2) FIG. 1 shows a schematic block diagram of the test apparatus;

(3) FIG. 2 shows an exemplary flow diagram showing an exemplary data set and the respective tables of corresponding processing steps to derive the best settings of the device under test;

(4) FIG. 3 shows a schematic block diagram of a data set filter;

(5) FIG. 4 shows a schematic block diagram of a method for performing a test on a device under test; and

(6) FIG. 5 shows a schematic block diagram of a method for filtering a data set using a data set filter.

DETAILED DESCRIPTION OF THE INVENTION

(7) In the following, embodiments of the invention will be described in further detail. Elements shown in the respective figures having the same or a similar functionality will have associated therewith the same reference signs.

(8) FIG. 1 shows a schematic block diagram of the test apparatus 2 for performing a test on a device under test. The test apparatus 2 comprises a data storage unit 4 and a data processor 6. The data storage unit is configured to store sets of input data 8 applied to the device under test 3 (drawn with dashed lines since it is not part of the test apparatus) during the test and to store the respective output data 10 of the device under test. The output data 10 is obtained from the device under test as a response of the device under test to the input data 8. The input data 8 comprises values of setting variables related to settings of the device under test and values of input variables comprising further information. Each set of input data represents one test case. Moreover, the data processor 6 is configured to process the data 12 stored in the data storage unit 4 such that a best combination of setting variables of the device under test is determined for one or more combinations of the input variables to obtain an optimized setting of the device under test for the one or more combinations of the input variables. The further information may comprise at least one of a stimulus waveform, an input pattern, a stimulus signal, an environment condition, or an environment status.

(9) FIG. 2 shows a schematic flow diagram with a sequence of exemplary tables 202, 204, 206, 208 representing results of selected steps which may be performed by the data processor to derive the best settings 11 of the device under test, e.g. represented by the parameter vector 16″. The best setting of the device under test may refer to a (optimal or best) combination of values of the setting variables for a current set of values of input variables. The first table 202 shows a short extract or a short excerpt of the data storage unit 4 showing four different test cases 14. Moreover, the data storage unit comprises input variables x.sub.1 and x.sub.2 8a and setting variables y.sub.1 and y.sub.2 8b. Output data of the device under test has been omitted for simplification.

(10) In other words, given is a table (database, data storage unit) 4 with K test cases 14, k=1 . . . K with multiple variables.

(11) The goal is to find the (best) values of Q variables y(k)=y.sub.1(k), . . . , y.sub.Q(k) as a function of P variables x=x.sub.1(k), . . . , x.sub.P(k) that optimize variable values g(k). Optionally, test cases with the optimum combination of x variables and y variables are selected.

(12) Variables x.sub.p and y.sub.q can be real-valued, integer, Boolean, categorical, or of any other suitable data type.

(13) The user can specify whether a good value of variable g is a. small or b. large or c. deviates little from a desired value g.sub.0 or d. deviates a lot from an undesired value g.sub.0.

(14) The user can chose to find the A. best worst case or B. best average or C. best (best) case

(15) According to embodiments, the data processor 6 is configured to perform a quantization of the input data 8 of at least one input variable 8a and/or setting variable 8b to form a discrete representation 8′ of the input data. Therefore, the data processor may process the discrete representation 8′ of the input data 8 such that the best setting of the device under test 3 for the one or more combinations of the input variables 8a is determined. This is advantageous, since it forms a countable or discrete number or amount of possible value combinations of all input variables 8a and/or setting variables 8b.

(16) In other words, if variables x.sub.p(k) or y.sub.q(k) take on too many values in k=1 . . . K (e.g. when they are real valued or integer), their values are quantized to N.sub.p discrete values for variables x.sub.p and M.sub.q different discrete values for variables y.sub.q. For Boolean or categorical variables, quantization is not necessary.
x.sub.p,1,x.sub.p,2, . . . ,x.sub.p,N.sub.p for p=1 . . . P
y.sub.q,1,y.sub.q,2, . . . ,y.sub.q,M.sub.q for q=1 . . . Q  (1)

(17) Variables values x.sub.p(k) and y.sub.q(k) can now be represented as indices, 1≤n.sub.p(k)≤N.sub.p, and 1≤m.sub.q(k)≤M.sub.q.
n.sub.p(k):x.sub.p(k)=x.sub.p,n.sub.p.sub.(k)
m.sub.q(k):y.sub.q(k)=y.sub.q,m.sub.q.sub.(k)  (2)

(18) To simplify notation, from now on, x.sub.p(k) and y.sub.q(k), refer to the quantized variable values 8′.

(19) The second table 204 shows the storage unit with quantized values as a result of the quantization of the first table 202 representing the storage unit 4. The quantization to integer values is exemplary and may be further applied to any other quantization pattern or quantization steps.

(20) According to a further embodiment, all possible combinations of input variables 8a and setting variables 8b may be assigned or allocated to a (unique) placeholder or value or representative, such as for example an integer value. In other words, after quantization, there are N=N.sub.1.Math.N.sub.2.Math. . . . .Math.N.sub.P different combinations x.sub.n of variable values. For each test k, the combination x(k) is described by an index n(k)=1 . . . N, such that
x.sub.n(k)=x(k),  (3)
which can e.g. be computed as

(21) $\begin{array}{cc}\begin{array}{c}n\left(k\right)=1+\left(n_1\left(k\right)-1\right)+\left(n_2\left(k\right)-1\right)N_1+\left(n_3\left(k\right)-1\right)N_1{N}_2+\mathrm{.Math.}\\ =1+\underset{p=1}{\overset{P}{.Math.}}\left(n_p\left(k\right)-1\right)\underset{i=1}{\overset{p-1}{.Math.}}{N}_i\end{array}& \left(4\right)\end{array}$
For a given n, the reverse mapping is

(22) $\begin{array}{cc}\begin{array}{c}{n}_1=n\%N_1\\ n_2=\mathrm{round}\left(\frac{n}{N_1}\right)\%N_2\\ \mathrm{.Math.}\mathrm{ }\\ n_p=\mathrm{round}\left(\frac{n}{{\Pi }_i^{p-1}{N}_i}\right)\%\mathrm{Np}\end{array}& \left(5\right)\end{array}$
where ‘%’ denotes the modulo operation.

(23) Similar for variables y:

(24) $\begin{array}{cc}{y}_{m\left(k\right)}=y\left(k\right)& \left(6\right)\\ \begin{array}{c}m\left(k\right)=1+\left(m_1\left(k\right)-1\right)+\left(m_2\left(k\right)-1\right)M_1+\left(m_3\left(k\right)-1\right)M_1{M}_2+\mathrm{.Math.}\\ =1+\underset{p=1}{\overset{P}{.Math.}}\left(m_p\left(k\right)-1\right)\underset{j=1}{\overset{p-1}{.Math.}}{M}_j,\end{array}& \left(7\right)\\ m_q=\mathrm{round}\left(\frac{m}{{\Pi }_j^{q-1}{M}_j}\right)\%M_q& \left(8\right)\end{array}$

(25) In other words, the data processor 6 is configured to assign a unique value to each possible combination of input variables and/or to each possible combination of setting variables. More specifically, the data processor 6 is configured to assign the unique value to each possible combination of input variables 8a using formula (4), where k is the current test case, n(k) is the unique value of each possible combination of input variables, n.sub.p(k) is an index representing possible values of the input variable p, P is the total number of input variables p, and N.sub.i is the total number of possible values of input variable i. Additionally or alternatively, the data processor 6 is configured to assign the unique value to each possible combination of setting variables using formula (7) where k is the current test case, m(k) is the unique value of each possible combination of setting variables, m.sub.p(k) is an index representing possible values of the setting variable p, P is the total number of setting variables p, and M.sub.j is the total number of possible values of setting variable j.

(26) According to further embodiments, the data processor 6 may assign indices representing the test cases 14 to a respective combination of input variables 8a of each possible combination of input variables and to a respective combination of setting variables 8b of each possible combination of setting variables. The aforementioned assignment may be formed or applied in a table 206 such that a position along the first dimension of the table indicates a unique value 18a of a current input variable 8a within all possible combinations of input variables and/or a position along the second dimension of the table 206 indicates a unique value 18b of a current setting variable 8b within all possible combinations of setting variables. This is advantageous, since each test case representing the used non-quantized values, is directly related to the quantized representation thereof and therefore, an output or a result of the optimization value with non-quantized values may be calculated and assigned directly to the quantized representation and is easily accessible by means of computer programming languages.

(27) In other words, test cases for combinations of x and y shall be found. This may be applied by identifying the sets (n,m) of test cases for all combinations n=1 . . . N of variables x and all combinations m=1 . . . M of variables y.
(n,m)={k:(x(k)=x.sub.n){circumflex over ( )}(y(k)=y.sub.m)}  (9)

(28) Note, this set (or each field) can contain no or one or multiple test cases or indices thereof, which causes no problem in the following steps. No test cases may be applied or derived, if the test does not cover all possible variable combinations, e.g. if the test is aborted before the test is completely performed. Multiple test cases may be applied or derived due to the quantization of the variables of the test cases. This may be seen in the context of a possible application scenario.

(29) For example, a device under test may be tested using the randomly created test cases. For large data sets, it is most likely that the duration of the test, started at the end of a working day, exceeds one night or even a weekend and that therefore, the test is still running in the morning of the next working day and therefore, results are not present. The same counts if the test is interrupted during processing, e.g. due to an error in the test routine or a power outage. Nonetheless, in contrast to known methods which fill a parameter space covering the results of the test in an organized, sorted or regular manner, for example one dimension after the other, the proposed device already covers a huge variety of the parameter space. However, the coverage may be not as dense as in the known approaches. This counts for those dimensions, where the parameter space is already filled regularly by the known approaches, wherein the coverage of those areas of the variable space, which is not already regularly filled by the known approaches may be much denser. However, according to the invention, it is possible to determine dependencies or relations between all used variables of the test cases at an early stage of the test. Using one of the known approaches, dependencies on one or more of the used variables may not have been examined at the same test stage. Moreover, the proposed random test case generation provides the same results, only derived in a different order, than a deterministic approach, if the whole test is performed. Nonetheless, a huge amount of tests is aborted or interrupted during processing. In this case, the random test case generation outperforms the deterministic approach since variables of the random test case generation are nevertheless varied or have a high variation wherein in a classical nested loop for example, the variable of the outmost loop is varied comparably slowly.

(30) Moreover, the data processor 6 may determine a best setting of the device under test 3 based on an optimization target 16, which may be a function of output variables, wherein values of the output variables form the output data 10. In other words, e.g. to form a quality metric such as the (processed) optimization target or figure of merit, the optimization target may weight different output variables derived from the device under test 3 to receive a single function which can be optimized. Based on the optimization target, the data processor may generate a standardized or processed optimization target, such that a small value of the standardized optimization target refers to a good setting of the device under test. The data processor may therefore determine the best setting 11 of the device under test based on the standardized optimization target. This is advantageous, since by using the standardized optimization target, the optimization targets are easily exchangeable or replaceable by other possible standardized optimization targets, since a minimum of the chosen standardized optimization target is in all cases the best case. Otherwise, by replacing a (non-standardized) optimization target, the best achievable value or limit of the optimization target is obviously also to be replaced.

(31) In other words, to simplify optimization, it is convenient to transform variable g(k) into a figure of merit f(k) for each test that can be minimized, i.e. where small is good.

(32) $\begin{array}{cc}f\left(k\right)=\{\begin{array}{cc}+g\left(k\right)& \mathrm{Case}a.\\ -g\left(k\right)& \mathrm{Case}b.\\ -.Math.g\left(k\right)-g_0.Math.& \mathrm{Case}c.\\ .Math.g\left(k\right)-g_0.Math.& \mathrm{Case}d.\end{array}& \left(10\right)\end{array}$

(33) Letters a, b, c, d refer to the options for a good value of variable g described above.

(34) According to further embodiments, the data processor 6 may assign a result of an optimization target 16 for each test case to a respective combination of input variables of each possible combination of input variables 8a and to a respective combination of setting variables 8b of each possible combination of setting variables, wherein, if multiple results of test cases are assigned to the same combination of input variables and the same combination of output variables, a single value 20 representing the multiple results is determined. The single value 20 representing the multiple results of the optimization target may be calculated using the maximum value of the multiple results of the optimization target, the maximum value of the multiple results of the optimization target, or an average value of the multiple results of the optimization target. Embodiments show that the assignment of the result of the optimization target may be applied in a further table. This is advantageous, since each result of the (standardized) optimization target derived using e.g. the non-quantized values of a test case 14, is directly related to the respective combination of (quantized) values of input variables 8a and the respective combination of (quantized) variables of setting variables 8b. Therefore, an output or a result of the optimization value with non-quantized values may be calculated and assigned directly to the quantized representation and is easily accessible by means of computer programming languages.

(35) Therefore, the data processor may assign, for each non-empty field of a table, where indices representing the test cases are stored, a result of an optimization target based on the output values referring to the one or more indices stored in the field to a corresponding field of a further table, wherein, if multiple indices are related to one field, a single value representing the multiple results of the optimization target is assigned to the corresponding field of the further table. In other words, the searched optimization function y(x) 16, is essentially also a function of indices, m(n). Because there can be multiple test cases (or one or none) (n,m) (cf. table 206) for each pair (n,m), the figure of merit is now consolidated to F(n,m) (cf. table 208) across all test cases in set (n,m).

(36) $\begin{array}{cc}f\left(n,m\right)=\{\begin{array}{cc}& \mathrm{Case}A.\\ & \mathrm{Case}B.\\ & \mathrm{Case}C.\end{array}& \left(11\right)\end{array}$

(37) When (n,m) is empty, F(n,m) is set to infinity so that it does not affect minimization. Letters A, B, C refer to the options described above.

(38) According to further embodiments, the data processor may determine an index of the best combination of setting variables for the one or more combinations of input variables within at least one test case, wherein the index indicates a unique value of the best combination of setting variables within all possible combinations of values of setting variables. This is advantageous, since similar values of input variables 8a may be quantized to the same representation 8′ thereof and, in the context of probably not having a complete test set, not tested with the same combinations of values of input variables. However, it is most likely, that the combination of setting variables which performs best for one test case, performs best or at least near the optimum for all further test cases related to the same quantized values of input variables.

(39) In other words, to find the optimum function y(x), for each combination n=1 . . . N of variables x 8a, the best combination μ(n) 16″ across all combinations m=1 . . . M of variables y 8b that minimize the consolidated figure of merit F(n,m) shall be found. Note that a small value of F(n,m) is good.

(40) $\begin{array}{cc}\mu \left(n\right)=\underset{m=1\mathrm{.Math.}M}{\mathrm{argmin}}\left\{F\left(n,m\right)\right\}& \left(12\right)\end{array}$

(41) This defines the optimum combination μ(n) 16″ of indices for variable values y 8b as a function of combination n of variable values x. In terms of variables x and y, the optimum function is
y.sub.μ(n)(x.sub.n)  (13)

(42) To find the optimal test cases, the set .sub.opt of test cases that satisfy the optimality condition is given by
.sub.opt={k:m(k)=μ(n(k))}.  (14)

(43) One way to show the effectiveness of optimization is to show the distribution of g(k) for test cases in .sub.opt compared to all test cases.

(44) Performing optimization, or calibration, is thus equivalent to applying a data set filter based on an expression that is true, when
m(k)=μ(n(k))  (15)

(45) Therefore, a data set filter for filtering a data set may obtain a best setting 11 of a device under test for current input variables using an optimization vector 16″, the optimization vector representing a relation between an index 18a of input variables 8a and an index 18b of a combination of setting variables 8b indicating the best setting of the device under test.

(46) A schematic representation of the data set filter 100 is shown in FIG. 3. Therefore, input to the optimization vector may be values of input variables 8a, e.g. from the data storage unit 4. Therefore, a currently tested device under test, e.g. not the same but of the same type where the optimization vector was derived from, may be evaluated using the optimization vector 16″. This is advantageous, since the finding of the best variable combinations has been performed beforehand and the best value combination may be obtained by only pointing on the corresponding field in the data storage unit. Therefore, and according to further embodiments, the data set filter 100 may be implemented in the device (the earlier device under test now in use) such that the setting variables are set, in real-time, meaning that the output of the device is obtained using the best value combination of setting variables derived from the optimization vector, based on the currently applied values of input variables. Moreover, the data set filter may be part of the test apparatus 2, for example, implemented in the data processor 6 or derived from the data processor 6. However, further embodiments show that the data processor 6 calculates the optimization vector μ(n) which is used by the data set filter 100 to determine the best setting of the device under test. In other words, the test apparatus comprises the data set filter, wherein the data processor is configured to form the optimization vector.

(47) According to embodiments applicable test cases for all combinations of x and y are extracted, where both have been quantized to obtain a tractable number of combinations. Then, for each combination of x, the best combination of y is determined that minimizes a consolidated (e.g. worst case) optimization metric across the applicable test cases. Overall this defines the optimal function ŷ(x). Performing optimization or calibration translates to applying a filter that returns test cases where y(k)=ŷ(x(k)). To verify the effectiveness of optimization or calibration, the behavior of those test cases that satisfy the optimality criteria can be compared against all test cases, e.g. by comparing their distributions of optimization target variable g(k).

(48) FIG. 4 shows a flowchart of a method 300 for performing a test on a device under test 3. The method 300 comprises a step 302 of storing sets of input data applied to the device under test during test, a step 304 of storing the respective output data of the device under test derived from the device under test as a response of the device under test to the input data, wherein the input data comprises values of variables related to settings of the device under test and values of input variables comprising known information, wherein each set of input data represents one test case, and a step 306 of processing the data stored in the data storage unit such that the best setting of the device under test for one or more combinations of the further variables is determined.

(49) FIG. 5 shows a schematic flowchart of a method 400 for filtering a data set to obtain a best setting of a device under test for current input variables using an optimization vector. The method 400 comprises a step 402 of forming the optimization vector representing a relation between an index of input variables and an index of a combination of setting variables indicating the best setting of the device under test.

(50) Although the present invention has been described in the context of block diagrams where the blocks represent actual or logical hardware components, the present invention can also be implemented by a computer-implemented method. In the latter case, the blocks represent corresponding method steps where these steps stand for the functionalities performed by corresponding logical or physical hardware blocks.

(51) Further embodiments of the invention relate to the following examples:

(52) 1. Optimization algorithm operating on a fixed subset of (x,y) pairs. Optionally, x and y are multiple variables, e.g. vectors. Further optionally, x and y variables can be a mix of numeric and categorical data. Therefore, numeric variables may be quantized.
2. Moreover, the calibration (of a device under test) may be implemented as a dataset filter.

(53) Although some aspects have been described in the context of an apparatus, it is clear that these aspects also represent a description of the corresponding method, where a block or device corresponds to a method step or a feature of a method step. Analogously, aspects described in the context of a method step also represent a description of a corresponding block or item or feature of a corresponding apparatus. Some or all of the method steps may be executed by (or using) a hardware apparatus, like for example, a microprocessor, a programmable computer or an electronic circuit. In some embodiments, some one or more of the most important method steps may be executed by such an apparatus.

(54) Depending on certain implementation requirements, embodiments of the invention can be implemented in hardware or in software. The implementation can be performed using a digital storage medium, for example a floppy disc, a DVD, a Blu-Ray, a CD, a ROM, a PROM, and EPROM, an EEPROM or a FLASH memory, having electronically readable control signals stored thereon, which cooperate (or are capable of cooperating) with a programmable computer system such that the respective method is performed. Therefore, the digital storage medium may be computer readable.

(55) Some embodiments according to the invention comprise a data carrier having electronically readable control signals, which are capable of cooperating with a programmable computer system, such that one of the methods described herein is performed.

(56) Generally, embodiments of the present invention can be implemented as a computer program product with a program code, the program code being operative for performing one of the methods when the computer program product runs on a computer. The program code may, for example, be stored on a machine readable carrier.

(57) Other embodiments comprise the computer program for performing one of the methods described herein, stored on a machine readable carrier.

(58) In other words, an embodiment of the inventive method is, therefore, a computer program having a program code for performing one of the methods described herein, when the computer program runs on a computer.

(59) A further embodiment of the inventive method is, therefore, a data carrier (or a non-transitory storage medium such as a digital storage medium, or a computer-readable medium) comprising, recorded thereon, the computer program for performing one of the methods described herein. The data carrier, the digital storage medium or the recorded medium are typically tangible and/or non-transitory.

(60) A further embodiment of the invention method is, therefore, a data stream or a sequence of signals representing the computer program for performing one of the methods described herein. The data stream or the sequence of signals may, for example, be configured to be transferred via a data communication connection, for example, via the internet.

(61) A further embodiment comprises a processing means, for example, a computer or a programmable logic device, configured to, or adapted to, perform one of the methods described herein.

(62) A further embodiment comprises a computer having installed thereon the computer program for performing one of the methods described herein.

(63) A further embodiment according to the invention comprises an apparatus or a system configured to transfer (for example, electronically or optically) a computer program for performing one of the methods described herein to a receiver. The receiver may, for example, be a computer, a mobile device, a memory device or the like. The apparatus or system may, for example, comprise a file server for transferring the computer program to the receiver.

(64) In some embodiments, a programmable logic device (for example, a field programmable gate array) may be used to perform some or all of the functionalities of the methods described herein. In some embodiments, a field programmable gate array may cooperate with a microprocessor in order to perform one of the methods described herein. Generally, the methods are performed by any hardware apparatus.

(65) While this invention has been described in terms of several embodiments, there are alterations, permutations, and equivalents which fall within the scope of this invention. It should also be noted that there are many alternative ways of implementing the methods and compositions of the present invention. It is therefore intended that the following appended claims be interpreted as including all such alterations, permutations and equivalents as fall within the true spirit and scope of the present invention.