Techniques for improved neural network modeling are provided. In one embodiment, a system comprises a memory that stores computer-executable components and a processor that executes the components. The computer-executable components can comprise a loss function logic component that determines a penalty based on a training term, the training term being a function of a relationship between an output scalar value of a first neuron of a plurality of neurons of a neural network model, a plurality of input values from the first neuron, and one or more tunable weights of connections between the plurality of neurons; an optimizer component that receives the penalty from the loss function component, and changes one or more of the tunable weights based on the penalty; and an output component that generates one or more output values indicating whether a defined pattern is detected in unprocessed input values received at the neural network evaluation component.

The present disclosure relates to systems, methods, and non-transitory computer readable media for selecting hyper-parameter sets by utilizing a modified Bayesian optimization approach based on a combination of accuracy and training efficiency metrics of a machine learning model. For example, the disclosed systems can fit accuracy regression and efficiency regression models to observed metrics associated with hyper-parameter sets of a machine learning model. The disclosed systems can also implement a trade-off acquisition function that implements an accuracy-training efficiency balance metric to explore the hyper-parameter feature space and select hyper-parameters for training the machine learning model considering a balance between accuracy and training efficiency.

This disclosure describes techniques for improving allocation of computing resources to computation of machine learning tasks, including on massive computing systems hosting machine learning models. A method includes a computing system, based on a computational metric trend and/or a predicted computational metric of a past task model, allocating a computing resource for computing of a machine learning task by a current task model prior to runtime of the current task model; computing the machine learning task by executing a copy of the current task model; quantifying a computational metric of the copy of the current task model; determining a computational metric trend based on the computational metric; deriving a predicted computational metric of the copy of the current task model based on the computational metric; and, based on the computational metric trend, changing allocation of a computing resource for computing of the machine learning task by the current task model.

A machine learning model including an autoencoder may be trained based on training data that includes sequences of non-anomalous performance metrics from an information technology system but excludes sequences of anomalous performance metrics. The trained machine learning model may process a sequence of performance metrics from the information technology system by generating an encoded representation of the sequence of performance metrics and generating, based on the encoded representation, a reconstruction of the sequence of performance metrics. An occurrence of the anomaly at the information technology system may be detected based on a reconstruction error present in reconstruction of the sequence of performance metrics. Related systems, methods, and articles of manufacture are provided.

A method, apparatus, and computer program product are provided for training a neural network or providing a pre-trained neural network with the weight-updates being compressible using at least a weight-update compression loss function and/or task loss function. The weight-update compression loss function can comprise a weight-update vector defined as a latest weight vector minus an initial weight vector before training. A pre-trained neural network can be compressed by pruning one or more small-valued weights. The training of the neural network can consider the compressibility of the neural network, for instance, using a compression loss function, such as a task loss and/or a weight-update compression loss. The compressed neural network can be applied within a decoding loop of an encoder side or in a post-processing stage, as well as at a decoder side.

A method (800) of training a text-to-speech (TTS) model (108) includes obtaining training data (150) including reference input text (104) that includes a sequence of characters, a sequence of reference audio features (402) representative of the sequence of characters, and a sequence of reference phone labels (502) representative of distinct speech sounds of the reference audio features. For each of a plurality of time steps, the method includes generating a corresponding predicted audio feature (120) based on a respective portion of the reference input text for the time step and generating, using a phone label mapping network (510), a corresponding predicted phone label (520) associated with the predicted audio feature. The method also includes aligning the predicted phone label with the reference phone label to determine a corresponding predicted phone label loss (622) and updating the TTS model based on the corresponding predicted phone label loss.

Provided is a method for classifying information technology (IT) service request messages. The method may include receiving data associated with an IT service request message, determining a plurality of number values associated with a plurality of characters included in the IT service request message, generating a vector that includes index values, generating a first bitmap based on generating the vector, generating a second bitmap based on the first bitmap, where the second bitmap has a first dimension and a second dimension, and where the first dimension and the second dimension are equal, and determining a classification of the IT service request message using a neural network algorithm. A system and computer program product are also disclosed.

Example methods, apparatuses, and/or articles of manufacture are disclosed that may be implemented, in whole or in part, using one or more computing devices to select options for decisions in connection with design features of a computing device. In a particular implementation, design options for two or more design decisions of neural network processing device may be selected based, at least in part, on combination of function values that are computed based, at least in part, on a tensor expressing sample neural network weights.

Aspects of data modification for neural networks are described herein. The aspects may include a connection value generator configured to receive one or more groups of input data and one or more weight values and generate one or more connection values based on the one or more weight values. The aspects may further include a pruning module configured to modify the one or more groups of input data and the one or more weight values based on the connection values. Further still, the aspects may include a computing unit configured to update the one or more weight values and/or calculate one or more input gradients.

Disclosed are a method of encoding and decoding an audio signal and an encoder and a decoder performing the method. The method of encoding an audio signal includes identifying an input signal, and generating a bitstring of each encoding layer by applying, to the input signal, an encoding model including a plurality of successive encoding layers that encodes the input signal, in which a current encoding layer among the encoding layers is trained to generate a bitstring of the current encoding layer by encoding an encoded signal which is a signal encoded in a previous encoding layer and quantizing an encoded signal which is a signal encoded in the current encoding layer.