A database engine includes one or more computing devices, each having one or more processors and memory. The memory stores programs configured for execution by the processors. The database engine receives a database query from a client, and parses the database query to build a query operator tree. The query operator tree includes a plurality of query operators. The database engine performs one or more optimization passes on the query operator tree, including a deduplication optimization pass, to form an optimized execution plan. The deduplication optimization pass includes determining that a first query operator is equivalent to a second query operator during a traversal of the query operator tree, and replacing the second query operator with a link to reuse results from the first query operator. The database engine executes the optimized execution plan to retrieve a result set from the database and returns the result set to the client.

To analyze open-source code at a large scale, a security domain graph language (SGL) has been created that functions as a vulnerability description language and facilitates program analysis queries. The SGL facilitates building and maintaining a graph database to catalogue vulnerabilities found in open-source components. This graphical database can be accessed via a database interface directly or accessed by an agent that interacts with the database interface. To build the graph database, a database interface processes an open-source component and creates graph structures which represent relationships present in the open-source component. The database interface transforms a vulnerability description into a canonical form based on a schema for the graph database and updates the database based on a determination of whether the vulnerability is a duplicate. This ensures quality and consistency of the vulnerability dataset maintained in the graph database.

A database engine receives a database query from a client. The database engine parses the database query to build a query operator tree that includes a plurality of query operators. The database engine performs one or more optimization passes on the query operator tree, including a deduplication optimization pass, to form an optimized execution plan. The deduplication optimization pass includes: creating a list of query operators via a first traversal of the query operator tree, determining a first query operator that is equivalent to a second query operator, based on a hash map, via a second traversal of the query operator tree, and substituting, via a third traversal of the query operator tree, the second query operator with a tree node that links to the first query operator. The database engine executes the optimized execution plan to retrieve a result set from the database, and returns the result set.

Disclosed herein is a computer-implemented method of evaluating a set of filter parameters being represented by a filter tree comprising a plurality of nodes, the method comprising: identifying whether a node is a root of a sub-tree comprising other nodes of the filter tree; generating a cost for said node by processing a sample input comprising a plurality of data items of a data source using the filter parameter and measuring the time taken for the plurality of data items to be processed; and determining a selectivity of said node based on an output of its filter parameter as a result of processing the sample input using the filter parameter; then ordering at least some of the plurality of nodes of the filter tree having the same parent node based on their relative costs and selectivities, for use in generating an ordered filter tree.

A batch data query method and apparatus are provided. According to the method, multiple logical query plan trees corresponding to multiple query statements are obtained. The multiple logical query plan trees are divided into multiple query groups according to operators of each logical query plan tree. Logical query plan trees in each query group have the following relationship: an inter-query write-read dependency relationship, an operator overlap relationship, or an operand overlap relationship. Then, inter-query optimization is performed on the logical query plan trees in each query group, and multiple corresponding physical query plan trees are generated according to the multiple optimized logical query plan trees. Finally, physical queries are executed according to the multiple physical query plan trees, to obtain query results of the multiple query statements.

The present teaching relates to method, system, and programming for evaluating expressions. An expression indicative of conditions and metadata associated therewith is obtained. A determination is made as to whether the expression corresponds to a modified version of an earlier expression based on the metadata. In response to a determination that the expression is the modified version of the earlier expression, a query associated with the modified expression is transmitted to a forecasting cluster so that the modified expression is to be evaluated by the forecasting cluster. In response to a determination that the expression does not have a corresponding earlier expression, the expression is evaluated.

A database engine receives a database query from a client. The database engine parses the database query to build a query operator tree that includes a plurality of query operators. The database engine performs one or more optimization passes on the query operator tree, including a deduplication optimization pass, to form an optimized execution plan. The deduplication optimization pass includes: creating a list of query operators via a first traversal of the query operator tree, determining a first query operator that is equivalent to a second query operator, based on a hash map, via a second traversal of the query operator tree, and substituting, via a third traversal of the query operator tree, the second query operator with a tree node that links to the first query operator. The database engine executes the optimized execution plan to retrieve a result set from the database, and returns the result set.

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for performing local distribution heuristics. One of the methods includes receiving an original expression having a conjunction comprising a context term and a disjunction of a plurality of other terms. The context term and the disjunction are evaluated according to one or more local distribution heuristics to determine that the context term is a candidate for distribution across the disjunction of the plurality of other terms. In response, a transformed expression is generated in which the candidate context term is distributed across the disjunction of the plurality of other terms.

Techniques are described herein for executing queries with a recursive clause using a plurality of processes that execute database operations in parallel. Each process of the plurality of processes either generate or are assigned a segment that is part of a temporary table. For each iteration of the recursive query, work granules are divided up and assigned to each of the plurality of processes. As each respective process produces a portion of a result set for a given iteration, that process appends said portion of the result set to the respective segment that the respective process manages. Each slave process then publishes, to one or more sources, a reference to the newly generated results. During the next iteration, any slave process may access any of the data from the previous iteration.

A database system includes a query optimizer which applies transformations to a logical representation of an input query in a declarative query language to generate expressions for a query execution plan. The query optimizer stores information, for each rule application, indicating a transformation applied and bindings for the transformation, the bindings indicating expressions input to the transformation. When a new query execution plan is to be generated for the input query, the query optimizer uses this stored information, for expressions in an original query execution plan, to guide the query optimizer to produce a current query execution plan while avoiding transformations not used to generate expressions in the original query execution plan.