A method for testing a system under test (SUT) in an active environment includes executing, by the testing system, on the SUT, a test from a set of tests. The method further includes, monitoring a first execution time to complete the test on the SUT in the active environment. Based on the first execution time being different than a second execution time of the test, marking, by the testing system, a code path associated with the test. The second execution time is a duration to complete execution of the test on the SUT in a clean execution environment. The method further includes communicating, by the testing system, the code path for analyzing a soft failure of the SUT in the active environment, wherein the soft failure occurs in the active environment during execution of the SUT based at least on a parameter of the active environment.

According to certain aspects of the disclosure, non-transitory computer readable media, systems and methods are disclosed for creating and executing test scenarios for automated testing of web pages. A computer system, for example, may perform operations including: recording one or more actions performed by a user on a web page, the one or more actions resulting in a web page element being displayed in the web browser application, recording an identifier of the web page element, in response to detecting a user input indicating a selection of the web page element, and recording a characteristic of the web page element; and generating, by the testing extension, a test scenario specifying a testable outcome resulting from the one or more actions, the testable outcome being an occurrence of an element having the identifier and the characteristic.

System and method for testing changes to binary code of a computer program include: collecting test coverage data from an executed set of tests of an original computer program; calculating a baseline report containing correlations between the executed set of tests and blocks of binary code of the original computer program; determining binary code changes between the original computer program and a modified version of the computer program; identifying one or more tests to be executed for verifying the binary code changes.

A method for generating test cases. The test cases are used to test a plurality of functional units of a logic system design during a simulation process of the logic system design. The method includes acquiring a constraint for generating test cases, and generating a first set of test cases according to the constraint. The first set of test cases is used to cover a part of the plurality of functional units. The method further includes performing a test on the logic system design based on the first set of test cases, determining uncovered functional units that are not covered by the test and a current coverage of the test, and, in response to the current coverage failing to satisfy a preset test coverage condition, generating a second set of test cases according to the uncovered functional units.

Embodiments of the present disclosure provide a method, an apparatus, and a storage medium for generating a plurality of test cases by a Portable Stimulus Standard (PSS) tool in a PSS environment. The test cases are used to test a logic system design. The method comprises: acquiring a configuration file and a coverage target of the logic system design; generating a scenario model according to the configuration file; generating the plurality of test cases according to the scenario model; determining whether the plurality of test cases satisfy the coverage target; in response to the plurality of test cases failing to satisfy the coverage target, determining a difference between the plurality of test cases and the coverage target; and updating the scenario model according to the difference.

Dynamically generating data tests includes: receiving test templates and metadata for a plurality of data objects and, on a trigger condition: based on at least the metadata and the test templates, determining a current set of templated data tests, wherein determining the current set of templated data tests comprises: determining at least one templated data test, from a prior set of templated data tests, to cease using; determining at least one templated data test to add to the current set of templated data tests; and determining, within the current set of templated data tests, at least one templated data tests to regenerate; generating the at least one templated data test to add to the current set of templated data tests; regenerating the least one templated data test to regenerate; and executing templated data tests, within the set of current templated data tests, on the plurality of data objects.

Generating test input values for functional components based on test coverage analysis is disclosed herein. In one example, a processor device of a computing device performs a test coverage analysis of a plurality of programming instructions of a functional component that takes a plurality of input parameters. Based on the test coverage analysis, the processor device identifies a subset of the programming instructions for which testing has not been performed. The processor device then identifies a data type for each of the functional component's input parameters, and generates test input values based on the data types and the test coverage analysis, including a test input value that causes executable instructions corresponding to the subset of programming instructions to be executed during subsequent testing. The test input values may be used during subsequent unit testing to ensure full test coverage of the functional component.

Methods, systems, and computer-readable media for analysis of code coverage differences across environments are disclosed. A code coverage profiling system determines a first code coverage profile associated with execution of program code in a first environment. The first code coverage profile indicates one or more portions of the program code that were executed in the first environment. The code coverage profiling system determines a second code coverage profile associated with execution of the program code in a second environment. The second code coverage profile indicates one or more portions of the program code that were executed in the second environment. The code coverage profiling system performs a comparison of the first code coverage profile and the second code coverage profile. The comparison determines a difference between the portions of the program code that were executed in the first and second environments.

A code development engine can be programmed to evaluate build code that can be representative of program code at an instance of time during or after a software development of the program code to identify and correct coding errors in the build code. A code run-time simulation engine can be programmed to simulate the build code in a modeled program code environment for the program code to identify and correct coding failures in the build code. A build code output module can be programmed to evaluate the build code to determine whether the build code is acceptable for use in a program code environment based on a level of acceptable risk for the build code in response to the coding error and/or coding failure being corrected in the build code.

A method for analyzing a computer program ecosystem includes performing a static analysis, including identifying static dependencies among elements of the ecosystem based on values of parameters in one or more parameter sets associated with the ecosystem, the elements of the ecosystem including the computer programs of the ecosystem and data resources associated with the computer programs. The method includes performing a runtime analysis, including identifying elements of the ecosystem that were utilized during execution of the ecosystem to process data records. The method includes performing a schedule analysis, including identifying a computer program of the ecosystem that has a schedule dependency from another computer program of the ecosystem. The method includes identifying a subset of the elements of the ecosystem as an ecosystem unit based on the results of the static, runtime, and schedule analyses. The method includes migrating the ecosystem unit, testing the ecosystem unit, or both.