A method, and the associated design, schema and techniques for processing digital data, whether random or not, through encoding and decoding losslessly and correctly for purposes of encryption/decryption or compression/decompression or both, including the use of Digital Lensing, Unlimited Code System, and other associated techniques. There is no assumption of or requirement for the digital information to be processed before processing.

An encoding device 100 encodes a plurality of input text files to a plurality of encoded files by using a static dictionary unit 121 and a dynamic dictionary unit 122. The dynamic dictionary unit 122 is generated in accordance with word appearance frequencies in the plurality of text files. The encoding device 100 generates a coupled encoded file that includes the plurality of encoded files, information on the dynamic dictionary unit 122, and position information that indicates positions of the respective plurality of encoded files.

A method for determining an encoding used for a sequence of bytes may be provided. The method comprises providing a set of candidate code pages and transforming them into different groups of sequences of bytes, wherein each group of sequences of bytes corresponds to one of the candidate code pages. Thereby each code point is transformed by applying a transformation from one of the candidate code pages to a reference code point value relating to a reference encoding for each code point. The method comprises further separating each of the transformed sequences of bytes into groups of tokens, wherein each group of tokens relates to one candidate code page, and providing an index relating to a text corpus. Furthermore, the method comprises selecting a code page from the set of candidate code pages at least partially based on how many tokens are found in the index.

The present disclosure involves systems, software, and computer implemented methods for improved design and implementation of data access metrics for automated physical database design. An example method includes identifying a database workload for which index advisor access counters are to be tracked. Each SQL statement in the database workload is executed. For each SQL statement, attribute sets are determined for which a selection predicate filters a result for an SQL statement. An output cardinality of each selection predicate is determined. A logarithmic counter for an attribute set corresponding to the selection predicate is determined based on the output cardinality of the selection predicate. The determined logarithmic counter is incremented. Respective values for logarithmic counters of the determined attributes are provided to an index advisor. The index advisor determines attribute sets for which to propose an index based on the logarithmic counters of the respective attribute sets.

Aspects of the present disclosure provide techniques for compressing data packets for cellular internet of things (CIoT) communications. An example method generally includes establishing at least one prefill buffer common to one or more UEs, wherein the prefill buffer includes a plurality of common strings, generating a compressed packet by finding matches to the common strings in at least one of a header portion or payload portion of the packet and associating identifiers with the common strings, and transmitting the packet.

A system configured to generate a set of compression dictionary snapshots. The system can determine a subset of a set of compression dictionary definitions, the subset having a first subset comprising one or more definitions that have changed since a time of a previous snapshot and a second subset having one or more definitions associated with a predetermined portion of the dictionary. The system can further generate and store snapshots based at least in part on the determined subset of one or more definitions and determine a plurality of active snapshots from the set of snapshots such that the set of one or more definitions is included in the plurality of active snapshots.

This disclosure relates to data storage in a blockchain-type ledger storing data based on a blockchain data structure. In one aspect, a method includes obtaining service data. A compression threshold on which selection of a target object for performing data compression is based is determined. The target object includes a client device or a database server. At least one of a hardware performance parameter or a network performance parameter is obtained. A compression weight is determined based on at least one of the hardware performance parameter or the network performance parameter. When the compression weight is greater than a predetermined value, data compression is performed on the service data at the client device. When the compression weight is less than or equal to the predetermined value, the client device sends the service data to the database server so that the database server performs data compression on the service data.

An information processing device includes: a memory; and a processor coupled to the memory and the processor configured to: generate compressed data, in sets of a prescribed size, in respect of one set of object data, in accordance with each of a plurality of compression methods; and select compressed data of the compression method which has completed compression of the object data first, among the plurality of compression methods.

A data model is trained to predict compressibility of binary data structures based on component entropy and predict relative compression efficiency for various compression algorithms based on component size. A recommendation engine in a storage system uses the data model to predict compressibility of binary data and determines whether to compress the binary data based on predicted compressibility. If the recommendation engine determines that compression of the binary data is justified, then a compression algorithm is recommended based on predicted relative compression efficiency. For example, the compression algorithm predicted to yield the greatest compression ratio or shortest compression/decompression time may be recommended.

A method of sending blocks of data from a client to be stored at a storage server, wherein for each block compression and encryption is performed at the client, and deduplication is performed at the server. Security is thus enhanced as the block is compressed and encrypted when it is sent over an unsecured network and when it is stored in potentially a third-party backup system. Provisions are made to enable addition of new compression algorithms and for retirement of old compression algorithms, while ensuring that a client would not receive a block which was compressed using an unsupported, e.g., retired, compression algorithm. In some examples a compression algorithm ID is tied to an encryption key version to enable refresh of blocks compressed with old algorithm