One example method includes receiving a group of files, two or more of the files being of different respective file types, creating a backup saveset that includes the group of files, classifying each of the files in the backup saveset based in part on the respective file types of the files, assigning a respective storage media type to each of the classified files in the backup saveset, and transmitting the backup saveset to a storage site.

A system comprises a plurality of computing devices that are communicatively coupled via a network and have a file system distributed among them, and comprises one or more file system request buffers residing on one or more of the plurality of computing devices. File system choking management circuitry that resides on one or more of the plurality of computing devices is operable to separately control: a first rate at which a first type of file system requests (e.g., one of data requests, data read requests, data write requests, metadata requests, metadata read requests, and metadata write requests) are fetched from the one or more buffers , and a second rate at which a second type of file system requests (e.g., another of data requests, data read requests, data write requests, metadata requests, metadata read requests, and metadata write requests) are fetched from the one or more buffers.

A file system manager implemented at a provider network identifies a storage device of a first group of storage devices of a provider network as an initial location of a file system object. Based on an access metric associated with the object, the file system manager initiates a transfer of contents of the object to a second storage device of a different storage device group, without receiving a client request specifying the transfer. In response to an access request received via a file system programmatic interface, contents of the object are provided from the second storage device. Based on a second access metric, the object is transferred back to the first group of storage devices.

There is provided a storage system that can avoid an increase in inter-node communication in the cooperation between file service and block service. The storage system includes a management section that manages a file processing section of a node in a currently active system and a file processing section of a node in a standby system as a pair, sets the file processing section of the node in the currently active system to be operational, manages a block processing section of the node in the currently active system and a block processing section of the node in the standby system as a pair, and sets the block processing section of the node in the currently active system to be operational.

Devices and techniques for managing flash memory are disclosed herein. A memory controller may receive a first program request comprising first host data to be written to the flash memory. The flash memory may comprise a number of storage units with each storage unit comprising a number of storage sub-units. If the first host data is less than a remainder threshold, the memory controller may generate a first program data unit comprising the first host data and first log data describing the flash memory. The memory controller may program the program data unit to the first storage unit of the flash memory, where the first log data is written to a first storage sub-unit of the number of storage sub-unit. The memory controller may also store an indication that the first storage sub-unit is invalid.

An embodiment may involve a network interface module; volatile memory configured to temporarily store data packets received from the network interface module; high-speed non-volatile memory; an interface connecting to low-speed non-volatile memory; a first set of processors configured to perform a first set of operations that involve: (i) reading the data packets from the volatile memory, (ii) arranging the data packets into chunks, each chunk containing a respective plurality of the data packets, and (iii) writing the chunks to the high-speed non-volatile memory; and a second set of processors configured to perform a second set of operations in parallel to the first set of operations, where the second set of operations involve: (i) reading the chunks from the high-speed non-volatile memory, (ii) compressing the chunks, (iii) arranging the chunks into blocks, each block containing a respective plurality of the chunks, and (iv) writing the blocks to the low-speed non-volatile memory.

A plurality of different views of data associated with a storage domain stored on a deduplicated storage are traversed to determine data chunks belonging to each view of the plurality of different views of data associated with the storage domain. A request for a metric associated with disk space utilization of a group of one or more selected views of data included in the plurality of different views of data associated with the storage domain that are stored on the deduplicated storage is received. Data chunks belonging to the one or more selected views of data associated with the storage domain of the group but not other views of the plurality of different views of data associated with the storage domain that are stored on the deduplicated storage are identified. An incremental disk space utilization of the group is determined, including by determining a total size of the identified data chunks. The metric associated with disk space utilization is provided based on the determined incremental disk space utilization of the group.

A method for dynamically storing keys and values includes receiving a request for storing one or more keys in a key value Solid State drive (KV-SSD). The method further includes performing a storage operation for storing each key of the one or more keys in a node of a data structure of the KV-SSD. The storage operation includes allocating a first region in the node for storing the key, such that a size of the first region is equal to a size of the key. The storage operation further includes allocating a second region in the node for storing key metadata associated with the key, such that the second region is of a predetermined size. The storage operation further includes storing the key in the first region and the key metadata in the second region of the node.

A plurality of computing devices are communicatively coupled to each other via a network, and each of the plurality of computing devices is operably coupled to one or more of a plurality of storage devices. The computing devices may use local caches and storing snapshots in a coherent manner when accessing the plurality of storage devices spread across multiple data centers.

Provided are systems and methods for paging data into main memory from checkpoint data stored on disk. In one example, the method may include one or more of receiving a request for a database record in main memory, determining whether the database record has been previously stored in the main memory, in response to determining that the database record has been previously stored in the main memory, identifying a slice where the database record was stored from among a plurality of slices included in the main memory, and paging content of the identified slice including a copy of the requested database record into the main memory from a snapshot captured of content included in the identified slice and previously stored on disk. Accordingly, documents can be paged into main memory on-demand from snapshots of slice content rather than paging an entire partition of content.