A test manager is connected to a hosted version control system containing text files stored in a repository. The test manager receives notification by the version control hosted service of one or more files containing formatted plain text. The formatted plain text includes template language constructs that are pre-processed by the test manager, along with optional defined data, to render manual tests instructions for guiding a human tester to perform operations and observe behavior for a system under test. The user interface is also configured to receive status information from the human tester to be associated with the rendered manual test instructions.

Provided is a system and method for dynamically adjusting a descriptive document with instructions for a user interface based on configuration settings and software version and/or release for a particular user and document data stored in a technical data object. The process ensures that the content of the descriptive document matches what a user will see on a screen when viewing the user interface. In one example, the method may include retrieving user interface configuration settings generating a simulated user interface that includes a plurality of user interface elements and formatting the user interface elements based on the user interface configuration settings, composing a window with text from a document and screenshot images of one or more formatted user interface elements from the simulated user interface corresponding to the text, and rendering the window via a display of a computing device.

A vehicle control device includes a verification management unit that executes old control software unit 112 representing an old version of control software and new control software unit 113 representing a new version of control software in sequence or in parallel, and an output verification unit that when an output value from old control software unit 112 and an output value from new control software unit 113 do not match, outputs information indicating the output values' not matching.

A method and system for predicting post-upgrade outcomes in information technology environments through proactive upgrade issue testing. At least with respect to computing, an upgrade may entail the replacement of a product—in the form of hardware, software, and/or firmware—with a newer or better version, which may serve to bring the product up to date, improve the characteristic(s) of the product, and/or resolve any issue(s) inflicting the product. Further, the process of upgrading a product in view of an issue therein is often reactive—that is, it is often required that a customer experiences and reports an issue for a product to a vendor prior to the issuance of an upgrade to the customer that addresses the issue for the product by the vendor. Presently, what is lacking is a mechanism through which product upgrades may be presented to customers proactively in order to minimize product failure and/or downtime.

Methods and systems for an automated micro-scheduler testing framework that allows tests to be automatically scheduled or rescheduled based on information such as results of previously-executed tests or other external information are provided. In large-scale development environments, where individual changes to a code repository cannot be specifically fully tested due to scalability and resource issues, micro-scheduler servers may be configured and designed to automatically identify target tests and request that the target tests be executed by a continuous integration system to automatically identify and resolve breakages introduced into a codebase managed by the continuous build system in a large-scale environment.

The system can identify data stored in repositories that indicate changes in the version of the application relative to a prior version of the application tested or deployed before receipt of the request to test the performance of the version of the application. The system can determine, based on the data and using machine learning with historical data associated with applications tested or deployed to test performance of the version, and without execution of the tests, a score for each of a plurality of tests configured to test performance of the version of the application. The system can select, based on the scores, a subset of the tests to execute, and provide an indication of the selected subset of the tests to cause execution of the subset of the tests to evaluate performance of the version of the application prior to deployment of the version of the application.

Technology is disclosed herein for semi receiver side write training in a non-volatile memory system. The transmitting device has delay taps that control the delay between a data strobe signal and data signals sent on the communication bus. The delay taps on the transmitting device are more precise that can typically be fabricated on the receiving device (e.g., NAND memory die). However, the receiving device performs the comparisons between test data and expected data, which alleviates the need to read back the test data. After the different delays have been tested, the receiving device informs the transmitting device of the shortest and longest delays for which data was validly received. The transmitting device then sets the delay taps based on this information. Moreover, the write training can be performed in parallel on many receiving devices, which is very efficient.

A dynamic experimentation evaluation system provides a framework in which a continuous stream of metric data is monitored to establish a causal relationship between changes in a software program and the effect of user-observable behavior. In one aspect, an A/B test is performed continuously on a stream of metric data representing the usage of a control version of software product and the usage of a treatment version of the software product. A sequential probability ratio test (SPRT) is used as the test statistic to determine when to terminate the test and produce results within a specific confidence interval and controlled error rate.

Systems and methods for automated test data microservices are provided. Test versions of software (such as an Application Programming Interface (API)) may be configured to automatically generate test data and to call a microservice to manage the test data. The microservice may automatically add and remove the test data from an operational data store to facilitate the testing process and to automatically perform setup and teardown stages of the testing process.

A system can determine a group of changed lines in a file comprising computer-executable instructions. The system can, for the respective changed lines, produce a mapping that identifies respective tests that test the respective changed lines. The system can, from the mapping and for the respective tests, determine a merge that identifies respective ones of the respective changed lines that are tested by the respective tests. The system can, from the merge and for the respective tests of the group of tests, determine respective coverage ratios that represent respective ratios between the respective ones of the respective changed lines of the group of changed lines that are tested by the respective tests and respective total changed lines in the file. The system can select an updated group of tests from the group of tests based on the respective ratios.