The disclosed embodiments provide a system for evaluating a performance of a mobile application. During operation, the system obtains, for a statistical model used in a quasi-experimental design, a first predicted outcome produced from a first set of data that is collected from two substantially identical versions of a mobile application. Next, the system uses the first predicted outcome to assess a bias of the statistical model. The system then improves an accuracy of the statistical model by using the assessed bias to normalize a second predicted outcome of the statistical model.

The disclosed embodiments provide a system for evaluating a performance of a mobile application. During operation, the system obtains a first set of data associated with adopters of a new version of a mobile application in a partial rollout of the new version and a second set of data associated with non-adopters of the new version in the partial rollout. Next, the system applies a statistical model to the first and second sets of data to select a subset of the non-adopters as potential adopters of the new version. The system then reduces a bias in a quasi-experimental design associated with the mobile application by using the first set of data and a third set of data associated with the potential adopters to estimate an average treatment effect (ATE) between the new version and an older version of the mobile application.

A code base is parsed to identify methods having changes in a code base since a last code commit. Thereafter, a call graph is traversed to identify test cases implicated by the identified methods having changes in the code base. The call graph can be a directed call graph comprising a plurality of connected nodes in which a first subset of the connected nodes are method nodes representing each method in the code base in which unidirectional edges connecting method nodes correspond to invocations by a calling method to a callee method, and in which a second subset of the connected nodes are test case nodes representing each of a plurality of available test cases to test the code base. The test case nodes are each coupled to one or more method nodes by unidirectional edges that correspond to the test case coverage of the method.

A method includes receiving an execute packet that includes a first instruction and a second instruction and executing the first instruction and the second instruction using a pipeline. Executing the first and second instructions includes storing a result of the first instruction in a holding register; determining whether an event that interrupts execution of the execute packet occurs prior to completion of the executing of the second instruction; and based on the event not occurring, committing the result of the first instruction after completion of the executing of the second instruction.

A test support device which can test all configurations of a system including a specific system component, comprehensively and quickly, is provided. A test support device (100) includes a storage unit (102) and an extraction processing unit (101). A configuration of a system is defined with a plurality of functionalities to be maintained by the system and a set of programs implementing the functionalities respectively. The storage unit (102) stores component-relation information (201) which indicates an identifier of each of one or more systems, an identifier of each of a plurality of functionalities to be maintained by each of the systems, and an identifier of each of one or more programs being capable of implementing each of the functionalities. The extraction processing unit (101) extracts, when a functionality or a program is specified, a configuration of a system including the specified functionality or the specified program on the basis of the component-relation information (201), and outputs the extracted configuration of a system.

Implementations for generating test plans for testing computer products based on product usage data are described. An example method may include obtaining, by a processing device, data associated with usage of a computer product, identifying, from the obtained data, a first set of parameters relevant to testing the computer product and a first set of values corresponding to the first set of parameters, comparing, by the processing device, the first set of parameters and the first set of values to a second set of parameters and a second set of values associated with a test plan to test the computer product, and generating, by the processing device, a modified version of the test plan in view of the comparison.

In various implementations, a supplemental system may be provided. The supplemental system may be coupled to business intelligence environment(s) to facilitate deployment, version control, testing, and/or other processes.

A system, method, and computer program product are provided. At least one testing project to be performed is identified, and a diagram is generated from testing activities including parameters with multiple values which includes one or more test flows including the testing activities. The one or more test flows include a plurality of possible testing scenarios. Further, scenarios are extracted from the generated diagram, and a test list to be executed is generated utilizing the extracted scenarios where each test case in the test list retains a link to a corresponding testing activity in the generated diagram. Still yet, the test list is, and at least one report is generated based on the execution of the test list which shows the generated diagram and a result of the execution corresponding to a testing activity based on an associated retained link.

Methods and systems for implementing proxy layers an a proxy layer management framework for performing integration testing of test components in a target test environment. Proxy layers are established between components that interact in a system and provide an interface based on a communication protocol, such as HTTP or RPC. Proxy layers axe configured, via a proxy layer manager to reroute communication, based on the communication protocol, to and from test components to simulate integration testing of the components in the target environment. Proxy layers may also apply rules that reroute traffic based on certain conditions for more effective, safer, and specific testing. Also, the routed communications may be traced and logged for testing and debugging.

Technical solutions are described for identifying false positive tests. An example method includes receiving an input identifying a defect in a portion of a product code. The method also includes identifying a first automated test that is associated with the portion of the product code. The method also includes lowering a first test confidence value that is associated with the first automated test by a predetermined amount in response to the first automated test being a passing test. The method also includes marking the first automated test as a false positive test.