Systems and methods for efficiently storing a distributed ledger of records. In an exemplary aspect, a method may include generating a record comprising a payload and a header, wherein the payload stores a state of a data object associated with a distributed ledger and the header stores a reference to state information in the payload. The method may further comprise including the record in a trunk filament comprising a first plurality of records indicative of historic states of the data object, wherein the trunk filament is part of a first lifeline. The method may include identifying a jet of the distributed ledger, wherein the jet is a logical structure storing a second lifeline with a second plurality of records. In response to determining that the first plurality of records is related to the second plurality of records, the method may include storing the first lifeline in the jet.

A high performance packet stream storage method. Original packet data from data traffic transmitted over a network is collected. Collected original packet data is written in a memory. Metadata from the collected original package data is extracted and metadata is written in the memory. The original packet data and the metadata is stored in a storage unit.

An apparatus, a method and a computer program for generating data packets according to a transport protocol from an application buffer comprising a plurality of data streams is provided. The apparatus comprises an input circuit configured to receive metadata comprising at least one of information about data packet types supported by the transport protocol, information about an offset and a length of the supported data packet types, and information about possible stream header start positions, possible payload start positions and possible offsets in the data streams. Further, the apparatus comprises a parsing circuit configured to identify offsets in an application buffer as possible segmentation points based on the metadata, to segment the application buffer at the possible segmentation points into segments for data packets, and to generate data packets according to the transport protocol based on the segments. Furthermore, an apparatus, a method and a computer program for processing the application buffer is provided.

A data link layer device and a packet encapsulation method are provided. The data link layer device includes a first and a second first-in-first-out (FIFO) module. The first FIFO module receives and stores multiple first data from an upper-layer module, and removes data gaps from the first data to store the first data in a continuous form. When the first FIFO module is not empty, the first FIFO module generates data of different lengths based on the current amount of data stored temporarily in the first FIFO module and a preset data length. When the data queue of the second FIFO module has enough space to receive the first data, the first FIFO module transfers the first data to the second FIFO module, and the first FIFO module transfers a header including the data length to a header queue of the second FIFO module.

A computer-implemented method of grouping network traffic metadata includes, based on a selected dimension of the network traffic metadata received from a network router, obtaining a statistic about a flow of network traffic metadata received over an interval for each instance of multiple instances of the dimension. The method further includes distributing the network traffic metadata into a plurality of groups for network traffic metadata from the smallest possible number of instances of the selected dimension to be distributed to each group, with the flow of network traffic metadata distributed optimally for a criteria regarding the statistic amongst the plurality of groups for minimizing cardinality of each group of the plurality of groups with respect to unselected dimensions of the network traffic metadata and providing each group to a columnar database for storage of the network traffic metadata distributed into each group in a different partition of the columnar database.

A method for data transmission may be implemented on an electronic device having one or more processors. The one or more processors may include a master queue including a master queue head and a plurality of primary ports that are connected to each other using a serial link. The method may include operating the master queue head to obtain a message. The method may also include operating the master queue head to segment the message into a plurality of segments. The method may also include operating the master queue head to transmit the plurality of segments to a first primary port of the plurality of primary ports in the master queue. The method may also include operating the first primary port to transmit the plurality of segments to a second primary port of the plurality of primary ports in the master queue.

A communication apparatus capable of efficiently creating transmission packets even when a free space of a storage unit storing transmission packet headers is insufficient includes a first storage unit to store a header of a transmission packet when the transmission packet is created in a first processing procedure, a first creation unit to create the transmission packet in the first processing procedure using the first storage unit, a second storage unit to store the transmission packet header when the transmission packet is created in a second processing procedure, a second creation unit to create the transmission packet in the second processing procedure using the second storage unit, and a control unit to control which one of the first and second creation units is used based on a data size necessary for the first creation unit to create the header and a free space of the first storage unit.

Some embodiments provide a network forwarding element with a data-plane forwarding circuit that has a parameter collecting circuit to store and distribute parameter values computed by several machines in a network. In some embodiments, the machines perform distributed computing operations, and the parameter values that compute are parameter values associated with the distributed computing operations. The parameter collecting circuit of the data-plane forwarding circuit (data plane) in some embodiments (1) stores a set of parameter values computed and sent by a first set of machines, and (2) distributes the collected parameter values to a second set of machines once it has collected the set of parameter values from all the machines in the first set. The first and second sets of machines are the same set of machines in some embodiments, while they are different sets of machines (e.g., one set has at least one machine that is not in the other set) in other embodiments. In some embodiments, the parameter collecting circuit performs computations on the parameter values that it collects and distributes the result of the computations once it has processed all the parameter values distributed by the first set of machines. The computations are aggregating operations (e.g., adding, averaging, etc.) that combine corresponding subset of parameter values distributed by the first set of machines.

In some examples, a system on chip (SOC) comprises a network switch configured to receive a packet and to identify a flow identifier (ID) corresponding to a header of the packet. The SOC comprises a direct memory access (DMA) controller coupled to the network switch, where the DMA controller is configured to divide the packet into first and second fragments based on the flow ID and to assign a first hardware queue to the first fragment and a second hardware queue to the second fragment, and wherein the DMA controller is further configured to assign memory regions to the first and second fragments based on the first and second hardware queues. The SOC comprises a snoopy cache configured to store the first fragment to the snoopy cache or to memory based on a first cache allocation command, where the first cache allocation command is based on the memory region assigned to the first fragment, where the snoopy cache is further configured to store the second fragment to the snoopy cache or to memory based on a second cache allocation command, and where the second cache allocation command is based on the memory region assigned to the second fragment.

A method for data transmission may be implemented on an electronic device having one or more processors. The one or more processors may include a master queue including a master queue head and a plurality of primary ports that are connected to each other using a serial link. The method may include operating the master queue head to obtain a message. The method may also include operating the master queue head to segment the message into a plurality of segments. The method may also include operating the master queue head to transmit the plurality of segments to a first primary port of the plurality of primary ports in the master queue. The method may also include operating the first primary port to transmit the plurality of segments to a second primary port of the plurality of primary ports in the master queue.