A packet processing technique can include receiving a packet, and parsing the packet based on a protocol field to generate a parse result vector. The parse result vector is used to select between forwarding the packet to a virtual machine executing on a host processing integrated circuit, forwarding the packet to a physical media access controller, multicasting the packet to multiple virtual machines executing on the host processing integrated circuit, and sending the packet to a hypervisor.

A method, performed by one or more processors, including: receiving a user query; deriving, based on the user query, a search engine query for use with search engine software and a data access query for use with data access software; sending, to the search engine software, the search engine query to cause the search engine software to locate first data stored in a search engine index; sending, to the data access software, the data access query to cause the data access software to locate second data stored in a distributed data store; receiving, from the search engine software, the first data; and receiving, from the data access software, the second data.

This disclosure relates to managing data by an agent located within a mining operation. The data is stored as voxel data on a voxel net server. The server processes user input from a user controlling the agent within the mining operation and receives from the agent a request for voxel data associated with one or more voxels. The one or more voxels are a subset of voxels stored on the voxel net server and each of the one or more voxels is identified based on connections with voxels of previous requests. The server then queries a database representing the voxel net for the one or more voxels to retrieve associated voxel data based on the connections and returns the voxel data to the agent. Finally, the voxel data is displayed on a user device to the user.

Techniques of data flow control are disclosed herein. One example technique includes upon receiving a notification indicating a change to a content item in a source shard, parsing the content item to extract data representing attributes of the content item and identifying a partition of the system-wide index based on the extracted data representing the attributes of the content item. The example technique can also include transmitting, to a token issuer, a request for a token that represents a permission to write to the identified partition of the system-wide index and upon receiving a token issued by the token issuer in response to the transmitted request, transmitting the extracted data representing the attributes of the content item along with the received token to write the extracted data in the partition of the system-wide index.

Methods, systems, apparatuses, and computer program products are described herein for the generation and traversal of a hierarchical index structure. The structure indexes search keys from data ingested from different data sources and enables efficient retrieval of the keys. As data is ingested, index nodes are generated at the lowest level of the structure. The nodes are analyzed to determine whether such nodes comprise duplicate keys. Responsive to doing so, a new index node is generated located at a higher level of the structure. This process results in a DAG comprising orphan nodes including different search keys. When processing a query for search keys, the orphan index nodes are initially analyzed for the keys. Upon finding a search key, its child nodes are recursively searched until location information specifying the location of ingested data in which the search key is located is found.

A method, apparatus and device for deleting a distributed system file, and a storage medium, comprising: querying whether an incomplete file deletion operation exists under a sub-tree root corresponding to a certain MDS (S102); if there is an incomplete file deletion operation, continuing to delete data under the sub-tree root corresponding to the MDS, and determining whether the sub-tree root is a copy (S103); and if the sub-tree root is a copy, deleting the sub-tree root copy in a memory of the MDS (S104). By means of the steps, the number of copies of the sub-tree root in the distributed file system can be reduced, thereby reducing the number of interaction times of master copy locking between different MDSs, improving the file deletion efficiency, and improving user friendliness and differentiated competitiveness of a product.

An embodiment may involve a network interface module; volatile memory configured to temporarily store data packets received from the network interface module; high-speed non-volatile memory; an interface connecting to low-speed non-volatile memory; a first set of processors configured to perform a first set of operations that involve: (i) reading the data packets from the volatile memory, (ii) arranging the data packets into chunks, each chunk containing a respective plurality of the data packets, and (iii) writing the chunks to the high-speed non-volatile memory; and a second set of processors configured to perform a second set of operations in parallel to the first set of operations, where the second set of operations involve: (i) reading the chunks from the high-speed non-volatile memory, (ii) compressing the chunks, (iii) arranging the chunks into blocks, each block containing a respective plurality of the chunks, and (iv) writing the blocks to the low-speed non-volatile memory.

A method and/or system of managing metadata are disclosed that include connecting a source data storage system (DSS) that stores both data and metadata to a metadata management platform (MMP); scanning metadata records onto the MMP from the DSS; storing metadata attributes for at least one of the group consisting of directories and buckets on the DSS in a look-up table on the MMP; and adding updated metadata attributes to the look-up table on the MMP for each subsequent scan of the DSS.

Provided are a method and device for local random readahead of a file in a distributed file system. The method includes: creating a readahead structure pointer in an operation handle of a file; in response to reading of the file, accessing and updating information in the readahead structure pointer, and determining whether the reading is local random reading; and performing readahead in a case where the reading is local random reading, the number of instances of continuous reading in the updated information is greater than 1, and a current reading position is greater than or equal to a readahead trigger position in the updated information.

A mass file retrieval method and apparatus based on an AI training platform, and a device. The method includes: an AI training platform acquiring a retrieval task issued by a user, the AI training platform generating a retrieval thread flow according to the retrieval task, and controlling a service logic of a retrieval process according to the retrieval thread flow; and the AI training platform, performing sequential encoding on files in a database in units of folders, generating ordered queue folders, extracting retrieval key words from the retrieval task, and performing keyword retrieval on each ordered queue folder by combining a binary search and depth-first traversal.