G06F16/24561

Near-memory acceleration for database operations

Despite the increase of memory capacity and CPU computing power, memory performance remains the bottleneck of in-memory database management systems due to ever-increasing data volumes and application demands. Because the scale of data workloads has out-paced traditional CPU caches and memory bandwidth, one can improve data movement from memory to computing units to improve performance in in-memory database scenarios. A near-memory database accelerator framework offloads data-intensive database operations via or to a near-memory computation engine. The database accelerator's system architecture can include a database accelerator software module/driver and a memory module with a database accelerator engine. An application programming interface (API) can be provided to support database accelerator functionality. Memory of the database accelerator can be directly accessible by the CPU.

Database optimization using record correlation and intermediate storage media

An embodiment includes deriving usage data associated with records of a database by monitoring requests to perform read operations on the records of the database. The embodiment generates record correlation data representative of correlations between respective groups of records of the database by parsing the usage data associated with the records of the database. The embodiment stores a plurality of records received as respective write requests during a first time interval in an intermediate storage medium. The embodiment identifies a correlation in the record correlation data between a first record of the plurality of records and a second record of the plurality of records. The embodiment selects, responsive to identifying the correlation, a first location in the database for writing the first record and a second location in the database for writing the second record based on a proximity of the first location to the second location.

Optimization of Database Write Operations By Combining and Parallelizing Operations Based on a Hash Value of Primary Keys
20230049797 · 2023-02-16 ·

Methods and systems disclosed herein may optimize write operations in a transaction to reduce the number of operations to the point where each of the write operations may be performed in parallel. The writer optimizer may review a first write operation and a second write operation to determine whether the first write operation and the second write operation write to the same row in the same table. When the first write operation and the second write operation write to the same row in the same table, the first write operation and the second write operation may be combined to generate a functionally-equivalent third write operation. The third write operation may reduce the number of operations in the transaction by replacing both the first write operation and the second write operation.

Methods and systems for searching directory access groups

Methods and systems for searching directory access groups are disclosed. A set of groups associated with a logon user is determined. The set of groups is partitioned into one or more disjoint subsets, wherein each of the disjoint subsets is represented by a data representation including a root node and one or more intermediate nodes. For each of the disjoint subsets, the disjoint subset is path compressed to flatten a structure of the data representation representing the disjoint subset. The data representation is cached to a database cache.

Evicting data associated with a data intake and query system from a local storage

Systems and methods are disclosed for making space available in a local storage of a data intake and query system. A cache manager of the data intake and query system may determine an amount of storage space of a local data store that is available for use to perform a query. The cache manager may then use one or more eviction policies associated with content stored at the local data store to purge content items to evict from the local storage. The system may then retrieve content for performing the query from a remote storage and store the retrieved content at the local storage.

Visualization Data Reuse In A Data Analysis System
20220358129 · 2022-11-10 ·

A method includes obtaining first data expressing a first usage intent; obtaining an answer object; obtaining, in accordance with first access control data, a first visualization data for the answer object; associating, with the first visualization data, a first generation-identifier that corresponds to at least a subset of the first access control data; generating first instructions for rendering the first visualization data on a user device of the first user; receiving a first snapshot of the first visualization data; storing a first association between the first generation-identifier and the first snapshot; obtaining second data expressing a second usage intent from a second user; identifying the answer object responsive to the second data expressing the second usage intent from the second user; and, responsive to second access control data corresponding to the second user matching the first generation-identifier, outputting second instructions for rendering the first snapshot to the second user.

METHOD AND APPARATUS FOR STORING AND QUERYING TIME SERIES DATA, AND SERVER AND STORAGE MEDIUM THEREOF

Disclosed are a method and apparatus for storing and querying time series data. The method includes: determining a data type of data to be stored; compressing the data to be stored by a data compression method corresponding to the data type; storing compressed data to a data storage table corresponding to the data type; receiving a query request including a query data type and a query time condition; querying target data that meets the query time conditions from a data storage table corresponding to the query data type. In the embodiments of the present disclosure, different compression methods are adopted for different types of data, which improves the compression efficiency of time series data and save storage resources. Moreover, when performing data query, time series data that meets a query time condition is searched in a data storage table corresponding to a query data type, which improves the query efficiency of different types of time series data.

Data Transfer Resiliency During Bulk To Streaming Transition

An indication to migrate requested data objects stored in a source database environment to a destination database environment is received. Some of data objects have many-to-one relationships with other data objects in the source database environment. At least one snapshot file generated by the source database environment is transferred to a destination database environment in bulk transfer mode. Subsequent incoming data received during bulk transfer mode, after the indication is stored in a temporary table. Upon completion of migration of the requested data objects the system transitions from bulk transfer mode to streaming mode. The subsequent incoming data from the temporary table is transferred to the destination database environment in response to the transition to streaming mode. Additional data received after the temporary table is empty is transferred from the source database environment to the destination database environment without use of the temporary table.

AGGREGATE QUERY OPTIMIZATION

A method, system and computer program product for aggregate query optimization. A dataset with plurality of values divided into dimensions and measurements is received. Pre-calculated values obtained by aggregates of measurements' values respective of plurality of combinations of dimensions' values are calculated and stored for one or more aggregate measurements of interest. Responsive to an input received of a first set of one or more dimensions and a second set of one or more aggregate measurements of interest, an aggregate result value of a respective member of the second set is calculated and outputted by retrieval of the pre-calculated value therefor respective of members of the first set.

DISTRIBUTED CARDINALITY OPTIMIZATION
20230084658 · 2023-03-16 ·

A cardinality query associated with a specific attribute is received. One or more in-scope attribute cardinality partitions is enforced on session record analytics. Vertical data compression is performed to eliminate a time dimension. Horizontal data compression is performed to eliminate one or more out-of-scope attributes. One or more like in-scope session records is aggregated. Magnitudes of each in-scope attribute of each enforced cardinality partition are summed. The sum is returned as a response to the cardinality query.