A database dual-core storage system based on optical disk comprises a server, a magnetic disk storage device and an optical disk storage device connecting to the server via data connection, a database management system, a data processor and a data connector installed on the server, wherein the database management system is arranged for completing database management and data management of the magnetic disk storage device and the optical disk storage device in response to data requests; the data processor is arranged for configuring fields of a database base core and fields of a database extension core, writing data of corresponding fields into the database base core and the database extension core respectively in response to data requests; the data connector is arranged for creating data connection between the database base core and the database extension core in response to data requests. The integrity and safety of data are guaranteed.

Techniques are shown for enabling a network system to index data stored on a third-party server to allow the data stored on the third-party server to be searchable by searching functionality of the network system.

A method and/or system of managing metadata are disclosed that include connecting a source data storage system (DSS) that stores both data and metadata to a metadata management platform (MMP); scanning metadata records onto the MMP from the DSS; storing metadata attributes for at least one of the group consisting of directories and buckets on the DSS in a look-up table on the MMP; and adding updated metadata attributes to the look-up table on the MMP for each subsequent scan of the DSS.

Technology described herein can perform deletion of a snapshot or portion thereof. In an embodiment, a system can comprise a processor and a memory that stores executable instructions that, when executed by the processor, facilitate performance of operations. The operations can comprise, to delete a snapshot, or a portion of a snapshot, of a real filesystem, reading an inode mapping file (IMF) of the snapshot that indexes a virtual inode number (VIN) corresponding to a real inode. The operations further can comprise identifying the real inode of the snapshot referenced by the VIN, identifying a file object corresponding to the real inode, and deleting the file object from the snapshot.

A method, performed by one or more processors, including: receiving a user query; deriving, based on the user query, a search engine query for use with search engine software and a data access query for use with data access software; sending, to the search engine software, the search engine query to cause the search engine software to locate first data stored in a search engine index; sending, to the data access software, the data access query to cause the data access software to locate second data stored in a distributed data store; receiving, from the search engine software, the first data; and receiving, from the data access software, the second data.

Systems and methods are described for distilling data. First data associated with a user may be received. The first data associated with the user may comprise an anonymized hash of an identifier associated with the user. A database may be determined to comprise a first record indicating the anonymized hash. The first record may comprise second data associated with the user. Based on the determining that the database comprises the first record, a second record may be generated. The second record may comprise the first data associated with the user, the second data associated with the user, and the anonymized hash. Based on the determining that the database comprises the first record, the example method may be stored to the database. These and other user and/or data distillation methods and systems are described herein.

One aspect facilitates a global map in a distributed system. The system generates a first data structure which comprises key-value pairs, wherein, in a respective key-value pair, the respective key is an integer and the respective value comprises a pointer to a sparse array which includes a bitmap (such as an ordered two-bit bitmap). The system stores the first data structure as a first partition of the global map. The system searches, based on a pattern, the first data structure to obtain a first value. If the first value comprises a two-bit bitmap, the system converts, based on the pattern, the first value to a two-dimensional bitmap, and performs a function on the first value to obtain a first result. The system uses the first value or the first result as metadata to execute a read or write operation in a filesystem associated with the distributed system.

Network appliances can record log entries in log objects. An object store can receive the log objects and can use the log objects to create index objects and flow log objects. Each flow log object and index object can be associated with a time period wherein the flow log object includes flow log entries received during that time period. The index object includes shard tables that can be stored in different nonvolatile memories and can thereby be concurrently searched. Shard entries in the shard tables indicate flow entry indicators. The flow entry indicators indicate log entries in the flow log object. An internally indexed searchable object can include the flow log object and the index object. Numerous indexed fields in the flow log entries and can be indexed with each indexed field searchable via the shard entries.

The present disclosure provides a high-performance data lake system and a data storage method. The data storage method includes the following steps: S1: converting a file into a file stream; S2: converting the file stream into an array in which multiple subarrays are nested; and S3: converting the array into a resilient distributed dataset (RDD), and storing the RDD to a storage layer of a data lake. The present disclosure provides a nested field structure, which lays the foundation for parallel processing in reading, and effectively improves read performance. Furthermore, the present disclosure flexibly generates a number of nested subarrays according to hardware cores, such that the data lake achieves better extension performance, and can keep optimal writing efficiency for different users.

Methods, non-transitory machine readable media, and computing devices that ensure correctness of file system analytics are disclosed. With this technology, a first generation number for a volume is incremented in response to a modification of a rule set that defines properties of objects of a file system associated with the volume. A determination is made when a second generation number in a first inode for a first one of the objects matches the first generation number. The first inode is identified based on a traversal of a directory tree associated with the file system. The modified rule set is applied to the properties for the first one of the objects to obtain values, when the second generation number fails to match the first generation number. Analytics data is output after the traversal has completed. The analytics data is generated in response to a query and is based on the values.