The present invention generally relates to systems and methods for detecting and/or isolating any causes of defective and/or partially defective IoT device or individual sensor device(s). In embodiments the present invention generally relates to fixing, replacing, and/or troubleshooting IoT devices and/or individual sensor device(s) that are defective and/or partially defective.

A system and method for detecting non-compliances using machine learning uses anomaly detection on an input dataset of unlabeled observations to produce output observations with corresponding probability scores of the output observations being anomalous. A portion of the output observations are labeled as being compliant observations based on the corresponding probability scores, which are added to a training dataset of compliant and non-compliant observations to derive an augmented dataset of compliant and non-compliant observations. The augmented dataset of compliant and non-compliant observations is then used to train a machine learning model for non-compliance detection.

Disclosed herein are methods and systems that apply a multi-layer Hidden Markov Model (HMM) for intrusion detection. The methods and systems employ a dimension reduction technique to extract only important features from network packet data and apply a decomposition algorithm to lower levels of data to construct lower level HMMs (representing partial solutions), which lower level HMMs are then combined to form a final, global solution. The multi-layer approach can be expanded beyond the exemplary case of 2 layers in order to capture multi-phase attacks over longer spans of time. A pyramid of HMMs can resolve disparate digital events and signatures across protocols and platforms to actionable information where lower layers identify discrete events (such as network scan) and higher layers identify new states which are the result of multi-phase events of the lower layers.

A reinforcement learning method and system that selects actions to be performed by a reinforcement learning agent interacting with an environment. A causal model is implemented by a hindsight model neural network and trained using hindsight i.e. using future environment state trajectories. As the method and system does not have access to this future information when selecting an action, the hindsight model neural network is used to train a model neural network which is conditioned on data from current observations, which learns to predict an output of the hindsight model neural network.

A reinforcement learning system and method that selects actions to be performed by an agent interacting with an environment. The system uses a combination of reinforcement learning and a look ahead search: Reinforcement learning Q-values are used to guide the look ahead search and the search is used in turn to improve the Q-values. The system learns from a combination of real experience and simulated, model-based experience.

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for using an environment model to simulate state transitions of an environment being interacted with by an agent that is controlled using a policy neural network. One of the methods includes initializing an internal representation of a state of the environment at a current time point; repeatedly performing the following operations: receiving an action to be performed by the agent; generating, based on the internal representation, a predicted latent representation that is a prediction of a latent representation that would have been generated by the policy neural network by processing an observation characterizing the state of the environment corresponding to the internal representation; and updating the internal representation to simulate a state transition caused by the agent performing the received action by processing the predicted latent representation and the received action using the environment model.

An approach is provided for generating an electric vehicle score (EVScore), a rating scale representing, for a user, a future estimated ease of ownership and operation of an EV within a defined geographic region. The method includes receiving a plurality of regional engine inputs pertaining to a defined geographic region, and one or more user inputs pertaining to the user. The method also includes processing, via a predictive model, the plurality of regional engine inputs and the one or more user inputs to generate one or more intermediate scores for the defined geographic region. The method also includes receiving a plurality of projected inputs pertaining to projected ownership and operation costs and benefits of electric vehicles (EVs) for the defined geographic region. The method also includes processing, via a trained machine learning model, the plurality of projected inputs and the one or more intermediate scores to generate the EVS core.

Methods and systems are described herein for generating dynamic conversational responses. Conversational responses include communications between a user and a system that may maintain a conversational tone, cadence, or speech pattern similar to a human during an interactive exchange between the user and the system. The interactive exchange may include the system responding to one or more user actions (which may include user inactions), and/or predicting responses prior to receiving a user action.

Provided are processes of balancing between exploration and optimization with knowledge discovery processes applied to unstructured data with tight interrogation budgets. Natural language texts may be processed, such as into respective vectors, by a natural language processing model. An output vector of (or intermediate vector within) an example NLP model may include over 500 dimensions, and in many cases 700-800 dimensions. A process may manage and measure semantic coverage by defining geometric characteristics, such as size or a relative distance matrix, of a sematic space corresponding to an evaluation during which the natural language texts are obtained based on the vectors of the natural language texts. A system executing the process may generate a visualization of the semantic space, which may be reduced to or is a latent embedding space, by reducing the dimensionality of vectors while preserving their relative distances between the high and reduced dimensionality forms.

There is provided systems and methods for training a classifier. The method comprises: obtaining a classifier for classifying data into one of a plurality of classes; retrieving training data comprising a set of observations and a set of corresponding labels, each label representing an assigned class for a corresponding observation; and applying an agent trained by a reinforcement learning system to generate labeled data from unlabeled observations and train the classifier using the training data and the labeled data according to a policy determined by the reinforcement learning system.