System and techniques for test scenario verification, for a simulation of an autonomous vehicle safety action, are described. In an example, measuring performance of a test scenario used in testing an autonomous driving safety requirement includes: defining a test environment for a test scenario that tests compliance with a safety requirement including a minimum safe distance requirement; identifying test procedures to use in the test scenario that define actions for testing the minimum safe distance requirement; identifying test parameters to use with the identified test procedures, such as velocity, amount of braking, timing of braking, and rate of acceleration or deceleration; and creating the test scenario for use in an autonomous driving test simulator. Use of the test scenario includes applying the identified test procedures and the identified test parameters to identify a response of a test vehicle to the minimum safe distance requirement.

A CI/CD assist device accepts configuration data and test data collectively from a terminal (a vendor terminal) of a provider that provides a network service to a customer. The configuration data specifies a functional unit required to provide the network service, and the test data specifies test content for the network service or the functional unit. Each of an NOS, an NOS, and an NOS automatically builds the functional unit specified by the configuration data that is accepted by the CI/CD assist device. Each of a test device, a test device, and a test device automatically conducts a test on the network service or the functional unit based on the test data. The network service and the functional unit are built in each environment.

A debugging management platform and an operating method for the same are provided. In the operating method, the debugging management platform operates a debugging agent service for establishing a debugging channel between a software development platform and a test platform. When receiving debugging packets are issued by the software development platform or the test platform, the debugging agent service analyzes the debugging packets and checks if the debugging packets meet an information security standard. The debugging packets are forwarded to the test platform or the software development platform if the debugging packets meet the information security standard. If the debugging packets do not meet the information security standard, the debugging packets are not forwarded, so as to ensure information security of the debugging packets that are forwarded between different environments.

Systems and methods for automated test data microservices are provided. Test versions of software (such as an Application Programming Interface (API)) may be configured to automatically generate test data and to call a microservice to manage the test data. The microservice may automatically add and remove the test data from an operational data store to facilitate the testing process and to automatically perform setup and teardown stages of the testing process.

Systems and methods for testing components or scenarios with execution history are disclosed. A method may include: receiving, at a testing interface and from an application or program executed by a user electronic device, an identification of a test and one or more data layers of a plurality of data layers in pod to test, the plurality of data layers including a data collection layer, a data ingestion layer, a data messaging layer, a data enrichment layer, and a data connect layer; receiving, by the testing interface, a selection of testing parameters or values for the identified test; retrieving, by the testing interface, the identified test; executing, by the testing interface, the identified test on the identified one or more data layers using the selected testing parameters or values; retrieving, by testing interface, results of the execution of the test; and outputting, by the testing interface, the results.

A computer-implemented method, according to one embodiment, includes: receiving, at a computer, a request to facilitate a testing environment, where the request specifies a number and type of resources to be included in the testing environment. A database which lists available resources in systems and/or devices that are in communication with the computer is inspected and the available resources are compared to the number and type of resources specified in the request to be included in the testing environment. In response to determining that a valid combination of the available resources meets the number and type of resources specified in the request to be included in the testing environment, the database is updated to indicate that each of the resources in the valid combination are in use. Moreover, the request is satisfied by returning information about the resources in the valid combination.

Immersive web-based simulator for digital assistant-based applications is provided. A system can provide, for display in a web browser, an inner iframe configured to load, in a secure, access restricted computing environment, an application configured to integrate with a digital assistant. The application can be provided by a third-party developer device. The system can provide, for display in a web browser, an outer iframe configured with a two-way communication protocol to communicate with the inner iframe. The system can provide a state machine to identify a current state of the application loaded in the inner frame, and load a next state of the application responsive to a control input.

A mixed mode programming method permitting users to program with graphical coding blocks and textual code within the same programming tool. The mixed mode preserves the advantages of graphical block programming while introducing textual coding as needed for instructional reasons and/or for functional reasons. Converting a graphical code block or group of blocks to a textual block lets the user see a portion of the textual code in the context of a larger program. Within one programming tool the mixed mode method allows users to learn programming and build purely graphical blocks; then transition into mixed graphical and textual code and ultimately lead to their ability to program in purely textual code. The mixed mode further allows users to program using any combination of drag-and-drop graphical blocks and typed textual code in various forms.

A software robot is designed to carry out an activity (e.g., a mouse click, a text input, etc.) on a target element (e.g., a button, an input field, etc.) of a user interface. The robot's code specification is configured to include an on-screen image of the target element and a text displayed by the target element. The robot is configured to automatically identify the target element at runtime according to an element ID specified in the source-code of the user interface, and when such identification fails, to identify the target element according to the text and image stored in the robot's code.

A software update system according to one embodiment of the present disclosure is configured to update software used in a vehicle based on update data of the software, the update data being transmitted to the vehicle from an external device that is communicably connected to the vehicle. The software update system includes: a software update unit configured to update the software based on the update data; a vehicle data acquisition unit configured to acquire respective pieces of second vehicle data about states of the vehicle before and after the software update by the update unit; and an effect evaluation unit configured to evaluate an effect of the software update based on the respective pieces of second vehicle data before and after the software update.