A computationally-efficient system for encoding a message object implements instructions including determining a token of the message object. The token identifies a structure of the message object. The instructions include obtaining a dictionary definition based on the token. The dictionary definition describes the structure of the message. The message includes multiple entries. Each of the entries is characterized by a data type and a field identifier. The dictionary definition defines an order of the entries. The instructions include, according to an order specified by the dictionary definition, selecting each entry of a set of entries in the message in sequence and writing the entry to a byte queue exclusive of the data type and the field identifier. The instructions include initiating transmission of the byte queue over a communications network.

The present teaching relates to a method, system and programming for searching a data storage. A key is extracted from a request and a metadata object associated with the key is identified. Further, a determination is made as to whether the metadata object is associated with a data structure stored in a first portion of the data storage. In response to a successful determination, the data structure is searched to retrieve a value associated with the key from the first portion. In response to an unsuccessful determination, a cache is searched to retrieve the value associated with the key, and in response to the key being absent in the cache, a file associated with the metadata object is searched to retrieve the value associated with the key, wherein the file is stored in a second portion of the data storage.

A system includes reception of a request to execute queries associated with an interactive data visualization, building, based on the request, of all queries required by the interactive data visualization on a background thread, reception of a query response associated with each built query on the background thread, building of a final response on the background thread based on the received query responses, and rendering, on a main thread, of the interactive data visualization based on the final response.

A unified access layer (UAL) and scalable query engine receive queries from various interfaces and executes the queries with respect to non-heterogeneous data management and analytic computing platforms that are sources of record for data they store. Query performance is monitored and used to generate a query performance model. The query performance model may be used to generate alternatives for queries of users or groups of users or to generate policies for achieving a target performance. Performance may be improved by monitoring queries and retrieving catalog data for databases referenced and generating a recommendation model according to them. Duplicative or overlapping sources may be identified based on the monitoring and transformations to improve accuracy and security may be suggested. A recommendation model may be generated based on analysis of queries received through the UAL. Transformations may be performed according to the recommendation model in order to improve performance.

The present teaching relates to a method, system and programming for recovering a data storage. In response to a failure of the data storage, a last checkpointed version is retrieved from a checkpoint file created previously for the failure of the data storage. A recovery table is updated to indicate the last checkpointed version and an incarnation number indicating a number of previous recovery of the data storage. The data storage is recovered based on the recovery table. A version number of the last checkpointed version is reset to zero upon a successful recovery of the data storage. The incarnation number is incremented.

The technology describes scanning tree data structures (trees) for multiple processes, at least partly in parallel. A service scans a tree from a beginning tree element to an ending tree element on behalf of a process; while scanning, another process can join in the scan at an intermediate tree element location (e.g., a key). For the subsequent process, the service scans the tree based on the intermediate location to the tree end, thereby visiting tree elements in parallel until the tree end, then continuing from the tree beginning element to the intermediate location for the subsequent process. The service basically completes a full carousel-type revolution for each process. One or more other processes can join an ongoing scan at any time, facilitating further parallel tree element visits, while still obtaining a full scan of the entire set of tree elements. The service handles changing tree versions during the scanning.

This disclosure provides various techniques that may allow for accessing values stored in a data structure that stores multiple values corresponding to database transactions using a skip list. A key may be used to traverse the skip list to access data associated with the key. The skip list maintains on ordering of multiple keys, each associated with a particular record in the data structure, using indirect links between data records in the data structure that reference buckets included in hash table. Each bucket includes pointers to one or more records in the skip list.

In accordance with an embodiment, described herein is a system and method for use of a dynamic flow process (referred to herein in some examples as Query Processing Dynamic Flow (QPDF)) in a multidimensional database computing environment. The dynamic flow process enables hybrid use of, for example, aggregate storage option, block storage option, or other types of storage containers, and provides a common flow to process a received input query in bottom-up mode. The approach can be used to reduce the size of the cube, which provides for efficient calculation of dynamic members.

A computer-implemented method, system and computer program product for improving performance of a distributed database. A query is received to store version data in the distributed database. Upon receiving the query to store the version data, the version data is stored in a row of a data page of a main table of a heap organized table/index organized table of the distributed database, where the row of the data page of the main table of the heap organized table/index organized table of the distributed database contains a pointer pointing to a later/previous version of the version data if the later/previous version of the version data is stored in the data page thereby chaining version data bi-directionally.

In accordance with an embodiment, described herein is a system and method for use of a dynamic flow process (referred to herein in some examples as Query Processing Dynamic Flow (QPDF)) in a multidimensional database computing environment. The dynamic flow process enables hybrid use of, for example, aggregate storage option, block storage option, or other types of storage containers, and provides a common flow to process a received input query in bottom-up mode. The approach can be used to reduce the size of the cube, which provides for efficient calculation of dynamic members.