Some embodiments of the invention provide a WAN optimization method for optimizing traffic flows through a WAN that connects multiple sites each having at least one router. The method is performed at a first router at a first site. From a second router at a second site, the method receives a file, that includes a set of segment identifiers corresponding to a set of segments stored by the first router, in an optimized first data stream originating from a source device at the second site and destined to a destination device at the first site. For each segment identifier in the set, the method attempts to retrieve a corresponding segment from a kernel memory of the first router. When the corresponding segment is not stored in the kernel memory, the method performs an operation to DMA the segment into the kernel memory from a disk storage of the first router.

A packet received via a network is stored in a packet memory. Information in a header of the packet is analyzed to determine one or more ports to which the packet is to be forwarded. The packet is read from the packet memory. It is ascertained whether the packet is of a multicast-type. In response to determining that the packet is the multicast-type, packet data that is i) read from the packet memory and ii) corresponds to the packet, is stored in a packet cache; the packet data is read from the packet cache, and multiple instances of the packet are forwarded to appropriate ports without further accessing the packet memory.

An improved buffer for networking and other computing devices comprises multiple memory instances, each having a distinct set of entries. Transport data units (TDUs) are divided into storage data units (SDUs), and each SDU is stored within a separate entry of a separate memory instance in a logical bank. One or more grids of the memory instances are organized into overlapping logical banks. The logical banks are arranged into views. Different destinations or other entities are assigned different views of the buffer. A memory instance may be shared between logical banks in different views. When overlapping logical banks are accessed concurrently, data in a memory instance that they share may be recovered using a parity SDU in another memory instance. The shared buffering enables more efficient buffer usage in a network device with a traffic manager shared amongst egress bocks. Example read and write algorithms for such buffers are disclosed.

A method of streaming data elements from an industrial site to a remote server, including: receiving data elements from industrial devices on the industrial site, storing the data elements in a queue, periodically extracting a number N of data elements from the queue and aggregating the same into one data frame whose size does not exceed a maximum size, storing each data frame in a buffer, and periodically extracting a data frame from the buffer and sending the same to the remote server. A data element is removed from the queue and stored in a database if the data element's time in the queue exceeds a maximum queuing time. A data frame is stored in the database instead of being stored in the buffer if the buffer is full. The maximum queuing time and the number N are then optimised.