Improving performance of garbage collection (GC) processes in a deduplicated file system having a layered processing architecture that maintains a log structured file system storing data and metadata in an append-only log arranged as a monotonically increasing log data structure of a plurality of data blocks wherein a head of the log increases in chronological order and no allocated data block is overwritten. The storage layer reserves a set of data block IDs within the log specifically for the garbage collection process, and assigns data blocks from the reserved set to GC I/O processes requiring acknowledgment in a possible out-of-order manner relative to an order of data blocks in the log. It strictly imposes using in-order I/O acknowledgement for other non-GC processes using the storage layer, where these processes may be deduplication backup processes using a segment store layer at the same protocol level as the GC layer.

A server in a cloud-based environment interfaces with storage devices that store shared content accessible by two or more users. Individual items within the shared content are associated with respective object metadata that is also stored in the cloud-based environment. Download requests initiate downloads of instances of a virtual file system module to two or more user devices associated with two or more users. The downloaded virtual file system modules capture local metadata that pertains to local object operations directed by the users over the shared content. Changed object metadata attributes are delivered to the server and to other user devices that are accessing the shared content. Peer-to-peer connections can be established between the two or more user devices. Object can be divided into smaller portions such that processing the individual smaller portions of a larger object reduces the likelihood of a conflict between user operations over the shared content.

A method of performing a file transaction, the method comprising: providing a transaction instruction to perform a set of one or more transaction operations on a device; responsive to determining the transaction instruction comprises one or more write transaction operations, wherein the one or more write transaction operations collectively relate to a first file group comprising at least one file object, the first file group has a first size, and each of the at least one file objects comprises identification information, if the first size does not exceed available device storage: writing each of the at least one file objects with an uncommitted file type; after the first file group is written, storing transaction information on the device storage indicating that the transaction is committed; responsive to determining that one or more pre-existing file objects share identification information with any of the first file group, erasing the one or more pre-existing file objects; updating the type of each of the at least one file objects in the first file group to a finalised type.

In embodiments, secure compression algorithms are provided that may be employed as a single operation on raw data to produce compressed and encrypted data. In embodiments, the algorithms described herein may be performed using any type of dictionary based encryption. In one embodiment, upon adding a new prefix to a dictionary table, the dictionary table may be permuted to randomize the entries into the table. The randomization may be based upon a permutation value generated by a deterministic pseudo-random generator and/or pseudo-random function. Other embodiments of randomization may be employed to provide secure compression. For example, instead of permuting the entire table upon adding a prefix, the prefix may be randomly added to the table.

Techniques for management of data storage in distributed storage systems are provided. A method may include receiving, by a computer system, a request to write data to a volume. The method may include identifying, by the computer system, a zone segment mapped to the volume. The zone segment may include a plurality of zones. The method may include identifying, by the computer system, a segment pointer indicating a write location in a zone of the zone segment. The method may include writing, by the computer system, the data to one or more zones of the plurality of zones of the zone segment, starting at the write location. The method may also include updating, by the computer system, the segment pointer according to a data endpoint of the data in the zone segment.

A method for managing log files for recording operations on data stored in a database is provided, wherein a set of log files having an allocated first portion of storage is updated, the allocated first portion of storage is monitored, and a second portion of storage is allocated in dependence on a determination that an available portion of storage is below a predetermined size. A method for generating a snapshot is provided, wherein data entries are included in the snapshot in dependence on a determined relative order of log records in a set of log files. A method of replicating a binary large object is provided, wherein the binary large object is sent to a second database in response to identifying a log record comprising data indicating the binary large object. A method for deleting one or more binary large objects is provided, wherein the data indicating one or more binary large objects that have been logically deleted are stored in a snapshot and the one or more binary large objects are deleted using the data indicating the one or more binary large objects after a previous snapshot is deleted.

Data writing methods and computing devices are provided. An example data writing method is applied to a computer system, and the computer system includes a file system and a flash memory-based storage system. The example data writing method includes obtaining a target logical address, where the target logical address is an address allocated from a first logical block to target data to be written into the flash memory-based storage system, the first logical block is one of multiple logical blocks in the file system, and the flash memory-based storage system includes multiple physical blocks. It is determined that the target logical address belongs to the first logical block. The target data is written into a first physical block based on a correspondence between the first logical block and the first physical block, where the first physical block is one of the multiple physical blocks.

In one embodiment, a system for managing a virtualization environment includes a plurality of host machines, wherein each of the host machines comprises a hypervisor and one or more user virtual machines (user VMs), and a virtual machine controller, one or more virtual disks comprising a plurality of storage devices, a virtualized file server (VFS) comprising a plurality of file server virtual machines (FSVMs), wherein each of the FSVMs is running on one of the host machines. The VFS may be configured to receive a request for storage system information from a user and generate and send a response to the request, wherein the response is customized according to configuration information of the VFS that is specific to the user. The storage system information requested may include a total size of storage available to the user, and the user may have an associated storage quota limit.

A method of synchronously executing input/output operations (IOs) for a plurality of applications using a storage device with a file system includes the steps of: receiving a first write IO including an instruction to write first data at a first address of the file system; determining that, within a first range of the file system comprising the first address, there are no pending unmap IOs for deallocating storage space of the storage device from files of the plurality of applications; after determining that there are no pending unmap IOs within the first range, locking the first range to prevent incoming unmap IOs from deallocating storage space within the first range from the files of the plurality of applications; after locking the first range, writing the first data to the storage device at the first address; and after writing the first data, unlocking the first range.

A facility for managing storage of program customization resources on a computing system is described. The computing system has an amount of storage space, some of which is free. Some of the program customization resources are selected for use. The facility determines that the amount of free storage space is inadequate. In response, the facility identifies at least a portion of the plurality of program customization resources stored on the computing system that are not selected for use, and causes the identified program customization resources to be deleted from the computing system.