Techniques for shrinking a storage space involve determining a used storage space in a storage pool allocated to a plurality of file systems, and determining a usage level of a storage space in the storage pool based on the used storage space in and a storage capacity of the storage pool. The techniques further involve shrinking a storage space from one or more of the plurality of file systems based on the usage level of the storage pool. Such techniques may automatically shrink storage space in one or more file systems from the global level of the storage pool, which determines an auto shrink strategy according to overall performance of the storage pool, thereby improving efficiency of auto shrink and balancing system performance and saving space.

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for precisely tracking memory usage in a multi-process computing environment. One of the methods includes implementing an instance of a memory usage tracker (MUT) in each process running in a node of a computer system. A MUT can maintain an account of memory usage for each of multiple logical owners running on a process on which the MUT is running. The MUT can determine an actual memory quota for each owner, and enforce the actual memory quota of the owner. Enforcing the actual memory quota of the owner can include receiving each memory allocation request, checking each allocation request and a current state of the account against the actual quota, approving or rejecting each allocation request, communicating the approval or rejection to an underlying memory manager, and updating the owner account for each approved allocation request.

Techniques for operating a computing system to perform neural network operations are disclosed. In one example, a method comprises receiving a neural network model, determining a sequence of neural network operations based on data dependency in the neural network model, and determining a set of instructions to map the sequence of neural network operations to the processing resources of the neural network processor. The method further comprises determining, based on a set of memory access operations included in the set of instructions, a first set of memory references associated with a first location of an external memory to store the input data and a second set of memory references associated with a second location of the external memory to store the output data, and generating an instruction file including the set of instructions, the first set of memory references and the second set of memory references.

Methods, systems, and computer storage media storing instructions for managing processing system efficiency. One of the methods includes obtaining data splitting a plurality of general-purpose processing units in a processing system into a high-priority domain and a low-priority domain, wherein the general-purpose processing units in the high-priority domain are assigned to perform one or more tasks comprising one or more high-priority tasks, and the general-purpose processing units in the low-priority domain are assigned to perform one or more low-priority tasks; and during runtime of the processing system, obtaining memory usage measurements that characterize usage of system memory by the high-priority domain and the low-priority domain; and adjusting, based on the memory usage measurements, a configuration of (i) the high-priority domain, (ii) the low-priority domain, or (iii) both to adjust utilization of the system memory by the general-purpose processing units.

A real-time computational device includes a programmable real-time processor, a communications input port which is connected to the programmable real-time processor through a first broadside interface, and a communications output port which is connected to the programmable real-time processor through a second broadside interface. Both broadside interfaces enable 1024 bits of data to be transferred across each of the broadside interfaces in a single clock cycle of the programmable real-time processor.

The present invention is generally directed to systems and methods of determining and provisioning peak memory requirements in Structured Query Language Processing engines. More specifically, methods may include determining or obtaining a query execution plan; gathering statistics associated with each database table; breaking the query execution plan into one or more subtasks: calculating an estimated memory usage for each subtask using the statistics; determining or obtaining a dependency graph of the one or more subtasks; based at least in part on the dependency graph, determining which subtasks can execute concurrently on a single worker node; and totaling the amount of estimated memory for each subtask that can execute concurrently on a single worker node and setting this amount of estimated memory as the estimated peak memory requirement for the specefic database query.

In an embodiment a method for managing access rights of software tasks executed by a processing unit (CPU) using a cache memory containing execution data of the tasks in memory locations, each execution data having an attribute representative of a level of access right of the respective task, includes changing the attributes of the locations of the cache memory when the access rights of at least one task changes and retaining the execution data contained in the locations of the cache memory.

A virtual compute instance is migrated between hosts using remote direct memory access (RDMA). The hosts are equipped with RDMA-enabled network interface controllers for carrying out RDMA operations between them. Upon failure of a first host and copying of page tables of the virtual compute instance to the first host's memory, a first RDMA operation is performed to transfer the page tables from the first host's memory to the second host's memory. Then, second RDMA operations are performed to transfer data pages of the virtual compute instance from the first host's memory to the second host's memory, with references to memory locations of the data pages specified in the page tables. The page tables of the virtual compute instance are reconstructed to reference memory locations of the data pages in the second host's memory and stored therein.

A computer-implemented method, a computer program product, and a computer system for determining optimal data access for deep learning applications on a cluster. A server determines candidate cache locations for one or more compute nodes in the cluster. The server fetches a mini-batch of a dataset located at a remote storage service into the candidate cache locations. The server collects information about time periods of completing a job on the one or more nodes, where the job is executed against fetched mini-batch at the candidate cache locations and the mini-batch at the remote storage location. The server selects, from the candidate cache locations and the remote storage location, a cache location. The server fetches the data of the dataset from the remote storage service to the cache location, and the one or more nodes execute the job against fetched data of the dataset at the cache location.

Configuration and dynamic profiling of storage class memory (SCM) devices is provided. Information is retrieved that includes historical SCM device configurations, historical SCM device utilization, functional and non-functional properties of a plurality of SCM devices on a host node, current real time utilization of the plurality of SCM devices by an application workload of a customer running on the host node, and relationships between the plurality of SCM devices, needs of the customer, and resource capabilities and real time resource utilization on the host node. A configuration of each respective SCM device is determined based on retrieved information and an artificial intelligence-predicted SCM device future utilization trajectory of the customer. Each respective SCM device is dynamically configured with a set of SCM device partitions according to a corresponding SCM device profile based on the determined configuration of each respective SCM device of the plurality of SCM devices.