A system, method, and computer-readable medium are disclosed for cognitive information processing. The cognitive information processing includes processing data from a plurality of data sources to provide cognitively processed insights via a augmented intelligence platform, the augmented intelligence platform comprising a cognitive process foundation platform, the cognitive process foundation platform comprising a cognitive composition platform, the cognitive composition platform being implemented to create custom extensions to the augmented intelligence platform.

During a query compilation process, a query is received that is directed to a set of source tables, each source table from the set of source tables being organized into at least one micro-partition and the query including at least one pruning operation. During the query compilation process, a modification of the query is performed for adjusting the at least one pruning operation, the modification being based on a set of statistics collected for previous pruning operations on at least a portion of the set of source tables and a set of heuristics, the set of statistics indicating at least an amount of execution time for each previous query associated with each of the previous pruning operations. The query is compiled including the modification of the query. The compiled query is provided to an execution node of a database system for execution.

A high-dimensional data nearest-neighbor query method based on variable-length hash codes is disclosed. Specifically, in this method, hash codes with the same code frequency are taken as a sub-data set, all the sub-data sets are ranked, a compression ratio is set for each sub-data set, the sub-data sets are compressed and trained according to the compression ratios, and hash codes and original codes corresponding to the trained sub-data sets are obtained; the hash code of each trained sub-data sets is copied to obtain multiple replicas, and the original codes and the corresponding replicas are strung to obtain strung hash codes which are integrated to form a final nearest-neighbor query table; and, a query code is obtained, and the nearest-neighbor query table is searched for a nearest-neighbor data set to complete query. The query efficiency and accuracy are greatly improved according to the invention.

Systems and method for improving query performance by querying an appropriate database engine based on the operation of the query request is provided. In one aspect, this approach involves querying a row-oriented database, querying a column-oriented database, or blacklisting the query request. In particular, updating the column-oriented database involves delete and insert operations. By maintaining updated databases and querying appropriate database engines, the response time of a query request may be improved.

A method includes determining a query expression indicating a query for execution against a plurality of rows. A superset-guaranteeing expression is generated in conjunctive normal form (CNF) based on the query expression. A query operator execution flow is generated to include a plurality of index-based IO operators based on the superset-guaranteeing expression and to further include at least one additional operator. Execution of the query is facilitated by applying the plurality of index-based IO operators to identify a first subset of rows as a proper subset of the plurality of rows based on index data stored of the plurality of rows, and by applying the at least one additional operator to the first subset of rows to identify a second subset of rows as a subset of the first subset of rows.

Methods and systems are provided for enhanced data loss prevention. The enhanced data loss prevention may be applied to data that is structured and/or semi-structured data, with the data comprising a plurality of records and a plurality of categories, with each record comprising a plurality of fields, each of which being associated with a different one of the plurality of categories. The enhanced data loss prevention may comprise selecting a subset of records of the data, with the selected subset comprising at least two records having between then one or more records not included in the subset. The fields of the selected subset may be scanned for sensitive information, and a likelihood to contain the sensitive information may be computed for each category based on the scanning. A subset of categories may be selected based on the computed likelihoods, and the sensitive information in the selected subset may be searched.

A method includes generating a plurality of parity blocks from a plurality of lines of data blocks. The plurality of lines of data blocks are stored in data sections of memory of a cluster of computing devices of the computing system by distributing storage of individual data blocks of the plurality of lines of data blocks among unique data sections of the cluster of computing devices. The plurality of parity blocks are stored in parity sections of memory of the cluster of computing devices by distributing storage of parity blocks of the plurality of parity blocks among unique parity sections of the cluster of computing devices.

Techniques are provided for processing a database command in a sharded database. The processing of the database command may include generating or otherwise accessing a shard key expression, and evaluating the shard key expression to identify one or more target shards that contain data used to execute the database command.

In accordance with some aspects of the present disclosure, a non-transitory computer readable medium is disclosed. The non-transitory computer readable medium includes instructions when executed by a processor cause the processor to receive, from a client, a tag-based object query including one or more parameters, map, using an index, the one or more parameters to a list of object names of corresponding objects stored in an object store, and provide, to the client, the list of object names. In some embodiments, the one or more parameters includes a tag. In some embodiments, the index and the object store are maintained natively. In some embodiments, the index and the object store are part of a flat namespace.

Data storage system conflict management techniques are described. In one example, data is sent by client devices to both a repository system and a search index system of a data storage system. Each of the client devices maintains, locally, records of changes to data of the repository system, e.g., through use of an invertible Bloom lookup table. The records are used as a basis by the client devices to send reports to an arbiter system and used to detect conflicts When detected, a conflict request is communicated to client devices involved in the conflict as well as to the repository system and responses used to generate a correction that is communicated to update the repository system and search index system.