A server in a cloud-based environment interfaces with storage devices that store shared content accessible by two or more users. Individual items within the shared content are associated with respective object metadata that is also stored in the cloud-based environment. Download requests initiate downloads of instances of a virtual file system module to two or more user devices associated with two or more users. The downloaded virtual file system modules capture local metadata that pertains to local object operations directed by the users over the shared content. Changed object metadata attributes are delivered to the server and to other user devices that are accessing the shared content. Peer-to-peer connections can be established between the two or more user devices. Object can be divided into smaller portions such that processing the individual smaller portions of a larger object reduces the likelihood of a conflict between user operations over the shared content.

The disclosure herein describes writing data to a log-structured merge (LSM) tree file system on an object storage platform. Write data instructions indicating data for writing to the LSM tree file system are received. Based on the received instructions, the data is written to the first data cache. Based on an instruction to transfer data in the live data cache to the LSM tree file system, the first data cache is converted to a stable cache. A second data cache configured as a live data cache is then generated based on cloning the first data cache. The data in the first data cache is then written to the LSM tree file system. Use of a stable cache and a cloned live data cache enables parallel writing data to the file system by the stable cache and handling write data instructions by the live data cache.

The described technology is generally directed towards fault tolerant cluster data handling techniques, as well as devices and computer readable media configured to perform the disclosed fault tolerant cluster data handling techniques. Nodes in a computing cluster can be configured to generate wire format resources corresponding to operating system resources. A wire format resource can comprise a cache key and a hint information to locate data, such as a file, corresponding to the operating system resource. The wire format resource can be stored in a resource cache along with a pointer that points to the operating system resource. The wire format resource can also be provided to client devices. Nodes in the computing cluster can be configured to receive and process client instructions that include wire format resources, as well as to use hint information to re-allocate data associated with a wire format resource.

Embodiments are described for prioritizing input/output (I/O) operations dispatched to a solid-state device (SSD) cache in a network, by defining a maximum write I/O operation size for writing data to the SSD cache, splitting large write I/O operations into smaller write I/O operations, each with a size less than the maximum write I/O operation size, interleaving cache read I/O operations in between the smaller write I/O operations, and performing the cache read I/O operations and the smaller write I/O operations in an order created by the interleaving. The network may comprise a deduplication backup system storing data to storage media including the SSD cache.

A system and method for providing dynamic I/O virtualization is herein disclosed. According to one embodiment, a device capable of performing hypervisor-agnostic and device-agnostic I/O virtualization includes a host computer interface, memory, I/O devices (GPU, disk, NIC), and efficient communication mechanisms for virtual machines to communicate their intention to perform I/O operations on the device. According to one embodiment, the communication mechanism may use shared memory. According to some embodiments, the device may be implemented purely in hardware, in software, or using a combination of hardware and software. According to some embodiments, the device may share its memory with guest processes to perform optimizations including but not limited to a shared page cache and a shared heap.

A cloud-native global file system in which a local filer creates objects and forward them to a cloud-based object store is augmented to include a reshapable caching scheme for the local filer. Like striped caches, the approach uses a stripe, but the striping is implemented via a true RAID 0 (disk striping) rather than as a striped LV (logical volume) device. This approach allows for a “reshape” operation to convert from a n-way stripe set to a n+-way stripe set. Preferably, a reshape involves redistributing each block on disk to its new calculated home. For example, going from a single disk to a two disk set would move every other block from disk 1 to disk 2, and rearrange the blocks on disk 1 to fill in the “holes”. Performance after the reshape matches that of a striped cache. In one embodiment, the cache is structured as a “degraded” RAID 4.

A method and an apparatus for implementing a buffer cache for a persistent file system in a non-volatile memory is provided. A set of data is maintained in one or more extents in a non-volatile random-access memory (NVRAM) of a computing device. At least one buffer header is allocated in a dynamic random-access memory (DRAM) of the computing device. In response to a read request by a first process executing on the computing device to access one or more first data blocks in a first extent of the one or more extents, the first process is granted direct read access of the first extent in the NVRAM. A reference to the first extent in the NVRAM is stored in a first buffer header. The first buffer header is associated with the first process. The first process uses the first buffer header to directly access the one or more first data blocks in the NVRAM.

A storage apparatus managing method applied to a first storage apparatus and a second storage apparatus coupled to the electronic apparatus is disclosed. The first storage apparatus includes a local registering region and a global registering region. The storage apparatus managing method includes: when the global registering region does not have a target data unit, reading the target data unit from the local registering region or from the second storage apparatus; and copying the target data unit to the global registering region. When the target data unit is copied to the global registering region, the target data unit is copied to a global registering buffer region, or otherwise in response to the global registering buffer region not having enough space, the target data unit is copied to a global registering file region.

A new serial bus interface module that enables constrained sensor systems to better match flash-based storage devices' (SD card) read and write performance. The serial bus interface module augments existing flash-based storage with non-volatile random-access memory to form a hybrid storage system using the most popularly used master-slave bus architecture. Together with PSC-like features, the serial bus interface module not only enables slave-to-slave transfer (therefore eliminating the double-transaction problem) but also reads caching (one source to multi-sink) and buffering while flushing. These transaction types enable multi-sector write for significantly faster speed and lower energy overhead, while the use of non-volatile memory for metadata caching means low risk of file-system corruption in the event of power failure. The serial bus interface also enables the direct data transfer from sensors to storage or communication modules without requiring the microprocessor's intervention.

A host runs an operating system kernel. A plurality of virtual private servers (containers) is supported within the kernel. The containers use the same files. A template cache has pre-calculated file checksums. The checksum is calculated each time the file is created and/or modified. Each file has an inode containing an attribute indicating whether the checksum is calculated and stored along with the file. If the checksum is present, the process is redirected to access the file with the same inode in the shared disk cache.