A method for processing data exactly once using transactional stream writes includes receiving, from a client, a batch of data blocks for storage on memory hardware in communication with the data processing hardware. The batch of data blocks is associated with a corresponding sequence number and represents a number of rows of a table stored on the memory hardware. The method also includes partitioning the batch of data blocks into a plurality of sub-batches of data blocks. For each sub-batch of data blocks, the method further includes assigning the sub-batch of data blocks to a buffered stream; writing, using the assigned buffered stream, the sub-batch of data blocks to the memory hardware; updating a storage log with an intent to commit the sub-batch of data blocks using the assigned buffered stream; and committing the sub-batch of data blocks to the memory hardware.

In one example, a method for identifying and allocating resources in a computing system, including checking, while one or more backup processes are running, database connections in an auto scaling group to determine if a number of database connections in use in connection with the backup processes has decreased since a prior check was performed. When the number of database connections in use has decreased, an identification is made as to which of a plurality of queues each respectively associated with one of the backup processes has the greatest need for additional database connections. Next, various metrics are evaluated and, based on the evaluation of the metrics, one or more available database connections are assigned to the queue with the greatest need for additional database connections.

Systems and methods for file sharing over secure connections. An example method comprises: receiving a client request identifying a file sharing host and a file residing on the file sharing host; establishing a secure client connection; responsive to identifying a management connection with the file sharing host, transmitting an identifier and a parameter of the secure client connection via the management connection; receiving a host request to establish a secure host connection, the host request comprising the identifier of the secure client connection; establishing the secure host connection using the parameter of the secure client connection identified by the received identifier; forwarding, over the secure host connection, a first data packet received over the secure client connection, the first data packet comprising at least part of the client request; and forwarding, over the secure client connection, a second data packet received over the secure host connection, the second data packet comprising at least part of the file identified by the client request.

A system and method for data storage by shredding and deshredding of the data allows for various combinations of processing of the data to provide various resultant storage of the data. Data storage and retrieval functions include various combinations of data redundancy generation, data compression and decompression, data encryption and decryption, and data integrity by signature generation and verification. Data shredding is performed by shredders and data deshredding is performed by deshredders that have some implementations that allocate processing internally in the shredder and deshredder either in parallel to multiple processors or sequentially to a single processor. Other implementations use multiple processing through multi-level shredders and deshredders. Redundancy generation includes implementations using non-systematic encoding, systematic encoding, or a hybrid combination. Shredder based tag generators and deshredder based tag readers are used in some implementations to allow the deshredders to adapt to various versions of the shredders.

A memory system is disclosed, including a memory component and a processing device configured to decode one or more codewords saved to a memory region of the memory component, detect that a number of bit errors corresponding to the decoding of the codeword exceeds a correction capability of the processing device, and execute an error recovery routine to reduce the number of detected bit errors to within the correction capability. The error recovery routine can include error recovery operations that are sequentially executed either until the number of bit errors is successfully reduced to within the correction capability or until a set of the error recovery operations has been executed. The error recovery operations can be ordered according to one or more factors, including energy used to execute a respective error recovery operation, a duration of the respective operation, and/or a likelihood of success of the respective operation.

A recording/reproducing device and a recording/reproducing method, which are capable of improving reliability of data while securing compatibility with a file format employed in a recording medium are provided. Provided is a reconstruction method of reproduction data acquired from a reproducing device that reproduces data from a recording medium in which the data is recorded, performs error correction using a second error correction code specified in a file format employed in the recording medium, and outputs reproduction data, and the reconstruction method of the reproduction data includes obtaining reproduction data from the reproducing device, reading a first error correction code different from the second error correction code from the reproduction data, and performing error correction on the reproduction data using the first error correction code and reconstructing user data.

A method is used in managing multi-step storage management operations. A policy is defined for a task of a multi-step storage management operation. The multi-step storage management operation includes multiple tasks. The policy for the task indicates directions for reacting to results of the task of the multi-step storage management operation. The task is invoked. The policy for the task is invoked based on results of the task.

A computing device includes interfaces and communicates with a dispersed storage network (DSN) and issues write slice requests to a set of storage units (SUs) to store distributedly a set of encoded data slices (EDSs) associated with a data object. When at least a write threshold number of favorable write slice responses are received, the computing device identifies an EDS without write confirmation and an SU associated therewith. The computing device stores the EDS without write confirmation in a slice cache memory and, based on a determination to reissue a write slice request associated with the EDS without write confirmation, the computing device reissues the write slice request. When a favorable write slice response is received from the SU associated with the EDS without write confirmation, the computing device deletes the EDS of the at least the write threshold number of EDSs without write confirmation from the slice cache memory.

A method for accessing a flash memory module is provided. The flash memory module is a 3D flash memory module including a plurality of flash memory chips, each flash memory chip includes a plurality of blocks, each block includes a plurality of pages, and the method includes: configuring the flash memory chips to set at least a first super block and at least a second super block of the flash memory chips; and allocating the second super block to store a plurality of temporary parities generated when data is written into the first super block.

Described embodiments provide for recovering an error, which has occurred in video contents, by using hidden data. A video content providing server generates video contents, into which information for video error recovery is inserted. The server includes: a block segmentation unit that segments an original image into at least one block having a pre-set size; a block description information (BDI) generation unit that generates block description information (BDI) for each of the at least one block, each BDI including information necessary to recover each of the segmented blocks; a BDI insertion unit that inserts each of the generated BDI into a block that is different from a block to be recovered by using the generated BDI; and a provisional video generation unit that generates provision video contents by combining at least one block, into which the BDI has been inserted.