An allocation request for requesting allocation of a target virtual area with respect to target data issued to a system program includes a target ID corresponding to the target data. In response to the allocation request, whether or not the target ID is included in data map information is determined. When it is included in the data map information, the system program determines whether or not a target physical area is included in a storage apparatus. When the target physical area is included in the storage apparatus, the system program reserves a free area in a non-volatile memory as a target memory area, copies target data stored in the storage apparatus to the target memory area, changes the target physical area in the data map information to the target memory area, and writes an association between the target virtual area and the target memory area into the volatile memory.

The present invention is generally directed to a caching framework that provides a common abstraction across one or more big data engines, comprising a cache filesystem including a cache filesystem interface used by applications to access cloud storage through a cache subsystem, the cache filesystem interface in communication with a big data engine extension and a cache manager; the big data engine extension, providing cluster information to the cache filesystem and working with the cache filesystem interface to determine which nodes cache which part of a file; and a cache manager for maintaining metadata about the cache, the metadata comprising the status of blocks for each file. The invention may provide common abstraction across big data engines that does not require changes to the setup of infrastructure or user workloads, allows sharing of cached data and caching only the parts of files that are required, can process columnar format.

A data-relationship-based FAST cache system includes a storage controller that is coupled to first storage device(s) and second storage device(s). The storage controller identifies a relationship between first data stored in the first storage device(s) and second data stored in the first storage device (s), with the relationship based on a difference between a first number of accesses of the first data associated with a first time period and a second number of accesses of the second data associated with the first time period being within an access difference threshold range. Subsequent to identifying the relationship, the storage controller determines that the first data has been accessed in the first storage device(s) a number of times within a second time period that exceeds a FAST cache threshold and, in response, moves both the first data and the second data to the second storage device(s) based on the relationship.

Utilizing a storage replica data structure includes receiving, at a hyper-kernel running on a computing node in a plurality of interconnected computing nodes, an indication of an operation pertaining to at least one of a guest physical memory address or a stable storage address. A guest operating system is run on a virtual environment that is defined by a set of hyper-kernels running on the plurality of interconnected computing nodes. It further includes updating a storage replica data structure. The storage replica data structure comprises a set of entries. The set of entries in the storage replica data structure comprises associations among guest physical memory addresses, physical memory addresses, and stable storage addresses

A data storage device is disclosed comprising a volatile memory, a primary and a first secondary non-volatile memory (NVM), and control circuitry coupled to the volatile memory and the primary and first secondary NVM and configured to (a) write metadata and user data associated with a host write command to the volatile memory; (b) write the user data to the primary NVM; (c) continue to write metadata associated with each of one or more host write commands to the volatile memory, and when a first condition is met, write metadata that has accumulated in the volatile memory to the first secondary NVM; and (d) repeat (c), and when a second condition is met, then write at least a portion of the metadata that has accumulated in the first secondary NVM or the volatile memory to the primary NVM.

A method of controlling a storage device and a random access memory includes, when a size of write-requested data is greater than a threshold, writing the write-requested data in the storage device and writing an address of the storage device in which the write-requested data is written in the random access memory. When the size of the write-requested data is smaller than or equal to the threshold, the write-requested data is written in the random access memory. The threshold is correlated to a size greater than a size of an area allocated to store the address in the random access memory.

A method and apparatus for cache management and eviction polices using unsupervised reinforcement learning schemes is disclosed.

Method and systems for a storage system are provided. Simulated cache blocks of a cache system are tracked using cache metadata while performing a workload having a plurality of storage operations. The cache metadata is segmented, each segment corresponding to a cache size. Predictive statistics are determined for each cache size using a corresponding segment of the cache metadata. The predictive statistics are used to determine an amount of data that is written for each cache size within certain duration. The process then determines if each cache size provides an endurance level after executing a certain number of write operations, where the endurance level indicates a desired life-cycle for each cache size.

Embodiments of systems and methods that ensure integrity of data during unexpected power interruption of loss are disclosed. In some embodiments, critical data is saved quickly and efficiently using backup power. Data integrity is ensured even when the reliability of backup power sources is an issue. In some embodiments, by skipping the updating and saving of system data while operating on backup power, significant reduction of time for saving critical data can be achieved. System data can be restored next time the data storage system is restarted. Improvements of data storage system reliability are thereby attained.

A method may include storing at least a portion of a metadata buffer of a persistent data structure in volatile memory, and storing at least a portion of a data buffer of the persistent data structure in persistent memory. A system may include a processor, a volatile memory coupled to the processor, and a persistent memory coupled to the processor. The processor may be configured to execute procedures including storing at least a portion of a metadata buffer of a persistent data structure in volatile memory, and storing at least a portion of a data buffer of the persistent data structure in persistent memory. A method may include storing at least a portion of a transient part of a persistent data structure in volatile memory, and storing at least a portion of a persistent part of the persistent data structure in persistent memory.