In various example embodiments, systems and methods for building data cubes to be stored in a cube store are presented. In some embodiments, a metadata engine generates the cube metadata. In further embodiments, cube data is generated by a cube build engine based on the cube metadata and source data. The cube build engine performs a multi-stage MapReduce job on the source data to produce a multi-dimensional cube lattice having multiple cuboids. In further embodiments, the cube data is provided to the cube store.

In various example embodiments, systems and methods for building data cubes to be stored in a cube store are presented. In some embodiments, a metadata engine generates the cube metadata. In further embodiments, cube data is generated by a cube build engine based on the cube metadata and source data. The cube build engine performs a multi-stage MapReduce job on the source data to produce a multi-dimensional cube lattice having multiple cuboids. In further embodiments, the cube data is provided to the cube store.

In an approach to improve file recall between storage devices by optimizing file recall for multiple users. Embodiments of the present invention obtain one or more similarities among one or more users based on one or more usage points and divide the one or more users into a plurality of clusters based on a content of the one or more usage points. Further, embodiments select a cluster from the plurality of clusters based on cluster importance, and select the content used by more than a predetermined percentage of the weighted users pertaining to the selected cluster as an automated recall subject. Additionally, embodiments recal one or more selected content from a secondary storage device to a primary storage device.

In an approach to improve file recall between storage devices by optimizing file recall for multiple users. Embodiments of the present invention obtain one or more similarities among one or more users based on one or more usage points and divide the one or more users into a plurality of clusters based on a content of the one or more usage points. Further, embodiments select a cluster from the plurality of clusters based on cluster importance, and select the content used by more than a predetermined percentage of the weighted users pertaining to the selected cluster as an automated recall subject. Additionally, embodiments recal one or more selected content from a secondary storage device to a primary storage device.

A method for traversal of a filesystem tree, the method may include traversing the filesystem tree by multiple processing entities of a set of processing entities that belong to a storage system; wherein the traversing comprises multiple iterations of on-the fly allocation of workload, associated with parallel traversing of the filesystem tree, among the multiple processing entities; wherein a current iteration of the on the fly allocation is (a) executed by a current group of processing entities that are currently assigned to traverse current nodes of the filesystem tree, and (b) comprises re-allocating by the current group, a traversal task for traversing one or more child nodes of each of the current nodes of the filesystem tree, to a next group of processing entities; wherein the current group and the next group belong to the set.

A method for traversal of a filesystem tree, the method may include traversing the filesystem tree by multiple processing entities of a set of processing entities that belong to a storage system; wherein the traversing comprises multiple iterations of on-the fly allocation of workload, associated with parallel traversing of the filesystem tree, among the multiple processing entities; wherein a current iteration of the on the fly allocation is (a) executed by a current group of processing entities that are currently assigned to traverse current nodes of the filesystem tree, and (b) comprises re-allocating by the current group, a traversal task for traversing one or more child nodes of each of the current nodes of the filesystem tree, to a next group of processing entities; wherein the current group and the next group belong to the set.

File access statistics associated with a content file are maintained by a file tier agent of a secondary storage system. The content file is accessible via a primary storage system. A target storage tier corresponding to the file access statistics associated with the content file is determined according to one or more policies. At least a portion of data of the content file is migrated to the determined storage tier while accessibility of the content file via the primary storage system is maintained.

File access statistics associated with a content file are maintained by a file tier agent of a secondary storage system. The content file is accessible via a primary storage system. A target storage tier corresponding to the file access statistics associated with the content file is determined according to one or more policies. At least a portion of data of the content file is migrated to the determined storage tier while accessibility of the content file via the primary storage system is maintained.

A device, system and process for controlling document access using hierarchical paths is provided. A query, received from a requesting device, comprises: a search string for searching a document database; and an identifier associated with a user. A security permissions database is accessed using the identifier to receive permissible hierarchical document access paths indicating document access permissions associated with the identifier. A modified query includes the permissible hierarchical document access paths. The modified query used to access an index of the document database, the index comprising: a searchable content portion, and corresponding hierarchical document access paths, of a document. Document identifiers are received that identify only the documents having: respective searchable content portions that include the search string of the modified query; and at least one respective corresponding hierarchical document access path encompassed by the permissible hierarchical document access paths. The document identifiers are provided to the requesting device.

A device, system and process for controlling document access using hierarchical paths is provided. A query, received from a requesting device, comprises: a search string for searching a document database; and an identifier associated with a user. A security permissions database is accessed using the identifier to receive permissible hierarchical document access paths indicating document access permissions associated with the identifier. A modified query includes the permissible hierarchical document access paths. The modified query used to access an index of the document database, the index comprising: a searchable content portion, and corresponding hierarchical document access paths, of a document. Document identifiers are received that identify only the documents having: respective searchable content portions that include the search string of the modified query; and at least one respective corresponding hierarchical document access path encompassed by the permissible hierarchical document access paths. The document identifiers are provided to the requesting device.