A code development engine can be programmed to evaluate build code that can be representative of program code at an instance of time during or after a software development of the program code to identify and correct coding errors in the build code. A code run-time simulation engine can be programmed to simulate the build code in a modeled program code environment for the program code to identify and correct coding failures in the build code. A build code output module can be programmed to evaluate the build code to determine whether the build code is acceptable for use in a program code environment based on a level of acceptable risk for the build code in response to the coding error and/or coding failure being corrected in the build code.

A hardware monitor arranged to assess performance of a hardware design for an integrated circuit to complete a task. The hardware monitor includes monitoring and counting logic configured to count a number of cycles between start and completion of the symbolic task in the hardware design; and property evaluation logic configured to evaluate one or more formal properties related to the counted number of cycles to assess the performance of the hardware design in completing the symbolic task. The hardware monitor may be used by a formal verification tool to exhaustively verify that the hardware design meets a desired performance goal and/or to exhaustively identify a performance metric (e.g. best case and/or worst case performance) with respect to completion of the task.

A system and a computer-implemented method for analyzing workflow of test automation associated with a robotic process automation (RPA) application are disclosed herein. The computer-implemented method includes receiving the workflow of the test automation associated with the RPA application and analyzing, via an Artificial Intelligence (AI) model associated with a workflow analyzer module, the workflow of the test automation based on a set of pre-defined test automation rules. The computer-implemented method further includes determining one or more metrics associated with the analyzed workflow of the test automation and generating, via the AI model, corrective activity data based on the determined one or more metrics.

Techniques for static code analysis tool and configuration recommendation via codebase analysis are described. Multiple codebases are tested using multiple static analysis tools and corresponding configurations, and a machine learning model is trained based on the results and characteristics of the codebases. Users may provide a codebase to be analyzed and job preferences indicating what characteristics of static analysis they desire, the codebase may be analyzed to generate input data for the model, and the model may identify one or more similar testing runs. These candidate runs may be filtered and/or ordered based on the user's stated job preferences, and the resulting tools and configurations associated with these runs may be returned to the user or used to perform static analysis of the user's codebase.

The present invention achieves automation of validation of a catalog created by a cooperative service catalog creator through a GUI. A catalog verification device 60 is a catalog verification device 60 that verifies the catalog created by the catalog creation assistance system that assists creation of the catalog used in the orchestrator 50. The catalog verification device 60 includes a BG cooperation function unit 61 that is a function unit for communicating with a catalog creation device 40 included in the catalog creation assistance system, and a catalog verification function unit 62 that verifies the catalog created by the catalog creation assistance system based on information acquired by the BG cooperation function unit 61, in which validation of the catalog is performed from perspectives of a mandatory check, a syntax check, and sequence of rules for cooperation among services by performing syntactic analysis of a file when the cooperative service catalog is created by and stored in the catalog creation assistance system.

Systems and methods are disclosed for automating and monitoring software development operations. The systems may facilitate a user to submit a request to receive information related to a software application development across a development operations (DevOps) pipeline, and to efficiently receive an accurate response to the request. A natural language processing application may use query parameters from the request to form a query. The query may be sent to an artificial intelligence markup language (AIML) interpreter to retrieve the requested information from a database. Alternatively or additionally, the query may be sent to an application associated with an integration of a plurality of DevOps tools in the DevOps pipeline. The application may develop a dynamic response to the request.

Apparatus and methods for refactoring infrastructure. The methods may include (a) defining parameters of an application landscape. The methods may include (b) stress-testing an application in a simulated environment based on: the parameters; and a simulated input to the application. The methods may include (c) identifying a state of stress of the application based on output of the stress-test. The methods may include (d) repeating (b)-(c) with a different simulated input until the state of stress satisfies a predetermined stochastic threshold. The methods may include (e) providing the state of stress to an upside down reinforcement learning (“UDRL”) engine. The methods may include (f) comparing a throughput corresponding to the state of stress to a benchmark throughput. The methods may include (g) redefining the parameters. The methods may include (h) repeating (a)-(f) until a threshold proximity to the benchmark throughput is reached.

Systems and/or techniques for facilitating online-monitoring of machine learning models are provided. In various embodiments, a system can receive monitoring settings associated with a machine learning model to be monitored. In various cases, the monitoring settings can identify a first set of data features that are generated as output by the machine learning model. In various cases, the monitoring settings can identify a second set of data features that are received as input by the machine learning model. In various aspects, the system can compute a first set of statistical metrics based on the first set of data features. In various cases, the first set of statistical metrics can characterize a performance quality of the machine learning model. In various instances, the system can compute a second set of statistical metrics based on the second set of data features. In various cases, the second set of statistical metrics can characterize trends or distributions of input data associated with the machine learning model. In various aspects, the system can store the first set of statistical metrics and the second set of statistical metrics in a data warehouse that is accessible to an operator. In various embodiments, the system can render the first set of statistical metrics and the second set of statistical metrics on an electronic interface, such that the first set of statistical metrics and the second set of statistical metrics are viewable to the operator.

An assessment system can generate a software quality value based on testing results and analysis of a multitude of factors that impact a readiness evaluation. For example, the system generates a software quality score (e.g., an Applause Quality Score “AQS”) that enables development teams to understand the level of quality they are achieving for a given release and build-over-build. In various examples, the system generates a data-driven score to enable development teams or quality assurance teams to make decisions for when a build is ready for release. In further embodiments, the system can integrate user interfaces that present a software quality score in a user dashboard that is linked to version control systems. On review and acceptance of the score, a user can trigger the release of their new code or product.

A debug support program causes a computer to execute a step of extracting a set of variables having a dependency relation from a plurality of variables written in a plurality of process blocks that are included in a sequence program and each describe a process per device, that is to say, a process per slave device; a step of collecting log information recording information regarding input of a value to each of the plurality of variables written in the process blocks; a step of correcting an extraction result regarding the dependency relation on the basis of contents of the collected log information; and a step of presenting a corrected extraction result.