Queueing Systems

Abstract

A network element including buffer address control circuitry for reading a given entry from a queue in a memory of a device external to the network element, the queue having at least a first entry and a last entry, the given entry including a destination address in the memory, output circuitry for writing data included in a packet received from external to the network element to the destination address in the memory in accordance with the given entry, and next entry assignment circuitry for assigning a next entry by: when the given entry is other than the last entry in the first queue, assigning the next entry to be an entry in the first queue after the given entry, and when the given entry is the last entry in the first queue, assigning the next entry to be the first entry in the first queue. Related apparatus and methods are also described.

Claims

1. A method comprising: providing a network element comprising buffer address control circuitry and output circuitry; receiving, from external to the network element, a packet comprising data; reading, by the buffer address control circuitry, a given entry from a first queue maintained in a memory of a device external to the network element, the first queue having at least a first entry and a last entry, the given entry comprising a destination address in the memory; writing, by the output circuitry, the data to the destination address in the memory in accordance with the given entry; assigning, by the buffer address control circuitry, a next entry by: when the given entry is other than the last entry in the first queue, assigning the next entry to be an entry in the first queue after the given entry; and when the given entry is the last entry in the first queue, assigning the next entry to be the first entry in the first queue; and performing again the writing and assigning, using the next entry as the given entry and using another packet received from external to the network element and comprising data.

2. The method according to claim 1 and wherein the first queue comprises a reliable delivery queue (RDQ) and each entry in the RDQ in the first queue comprises a work queue entry (WQE).

3. The method according to claim 1 and also comprising: performing the following before reading the given entry from the first queue: reading, by the buffer address control circuitry, a second queue given entry from a second queue maintained in the memory of the device external to the network element, the second queue having at least a first second queue entry and a last second queue entry, the second queue given entry comprising a destination address in the memory; writing in accordance with the second queue given entry, by the output circuitry, data to the destination address in the memory; assigning, by the buffer address control circuitry, a next second queue entry by: when the second queue given entry is other than the last entry in the second queue, assigning the next second queue entry to be an entry in the second queue after the given entry, and performing again, using the next entry as the given entry and using another packet received from external to the network element and comprising data: the writing in accordance with the second queue given entry, and the assigning a next second queue entry; and when the second queue given entry is the last entry in the second queue, proceeding with the reading, by the buffer address control circuitry, a given entry from the first queue, using another packet received from external to the network element and comprising data.

4. The method according to claim 3 and wherein the second queue comprises a reliable delivery queue (RDQ) and each entry in the RDQ in the second queue comprises a work queue entry (WQE).

5. The method according to claim 1 and also comprising: providing a plurality of queues; choosing one queue from the plurality of queues and performing the following, for the chosen queue of the plurality of queues, before reading the given entry from the first queue: reading, by the buffer address control circuitry, a chosen queue given entry from the chosen queue maintained in the memory of the device external to the network element, the chosen queue having at least a first chosen queue entry and a last chosen queue entry, the chosen queue given entry comprising a destination address in the memory; writing in accordance with the chosen queue given entry, by the output circuitry, data to the destination address in the memory; and assigning, by the buffer address control circuitry, a next chosen queue entry by: when the chosen queue given entry is other than the last entry in the chosen queue, assigning the next chosen queue entry to be an entry in the chosen queue after the given entry, and performing again, using the next entry as the given entry and using another packet received from external to the network element and comprising data: the writing in accordance with the chosen queue given entry, and the assigning a next chosen queue entry; and when the chosen queue given entry is the last entry in the chosen queue, performing the following: when any of the plurality of queues has not yet been chosen, choosing a different queue from the plurality of queues, and performing again, using another packet received from external to the network element and comprising data, the reading a chosen queue given entry, the writing in accordance with the chosen queue given entry, and the assigning a next chosen queue entry; and when all of the plurality of queues have been chosen, using another packet received from external to the network element and comprising data and proceeding with the reading, by the buffer address control circuitry, a given entry from the first queue.

6. The method according to claim 5 and wherein each of the plurality of queues comprises a reliable delivery queue (RDQ) and each entry in each RDQ in the plurality of queues comprises a work queue entry (WQE).

7. The method according to claim 1 and wherein the packet comprises a plurality of packets each comprising data, and the method also comprises: before the proceeding with the reading a first given entry from the first queue: the network element discarding at least one of the plurality of packets.

8. The method according to claim 1 and wherein the packet comprises a plurality of packets each comprising data, and the method also comprises: before the proceeding with the reading a first given entry from the first queue: the network element storing at least one of the plurality of packets.

9. The method according to claim 1 and wherein the network element comprises a network interface controller (NIC).

10. The method according to claim 1 and wherein the network element comprises a switch.

11. A method comprising: providing a network element comprising buffer address control circuitry and output circuitry; receiving, from external to the network element, a packet comprising data; providing a plurality of queues; and choosing one queue from the plurality of queues and performing the following for the chosen queue of the plurality of queues: reading, by the buffer address control circuitry, a chosen queue given entry from the chosen queue maintained in a memory of the device external to the network element, the chosen queue having at least a first chosen queue entry and a last chosen queue entry, the chosen queue given entry comprising a destination address in the memory; writing in accordance with the chosen queue given entry, by the output circuitry, data to the destination address in the memory; and assigning, by the buffer address control circuitry, a next chosen queue entry by: when the chosen queue given entry is other than the last entry in the chosen queue, assigning the next chosen queue entry to be an entry in the chosen queue after the given entry, and performing again, using the next entry as the given entry and using another packet received from external to the network element and comprising data: the writing in accordance with the chosen queue given entry, and the assigning a next chosen queue entry; and when the chosen queue given entry is the last entry in the chosen queue, choosing a different queue from the plurality of queues, and performing again the reading a chosen queue given entry, the writing in accordance with the chosen queue given entry, and the assigning a next chosen queue entry.

12. The method according to claim 11 and wherein the network element comprises a network interface controller (NIC).

13. The method according to claim 11 and wherein the network element comprises a switch.

14. A network element comprising: buffer address control circuitry configured to read a given entry from a first queue maintained in a memory of a device external to the network element, the first queue having at least a first entry and a last entry, the given entry comprising a destination address in the memory; output circuitry configured to write data, the data being comprised in a packet received from external to the network element, to the destination address in the memory in accordance with the given entry; and next entry assignment circuitry configured to assign a next entry by: when the given entry is other than the last entry in the first queue, assigning the next entry to be an entry in the first queue after the given entry; and when the given entry is the last entry in the first queue, assigning the next entry to be the first entry in the first queue.

15. The network element according to claim 14 and wherein the first queue comprises a reliable delivery queue (RDQ) and each entry in the RDQ in the first queue comprises a work queue entry (WQE).

16. The network element according to claim 14 and wherein the buffer address control circuitry is also configured, before reading the given entry from the first queue, to read a second queue given entry from a second queue maintained in the memory of the device external to the network element, the second queue having at least a first second queue entry and a last second queue entry, the second queue given entry comprising a destination address in the memory, and the output circuitry is also configured to write data to the destination address in the second queue given entry, and the buffer address control circuitry is also configured to assign an next second queue entry by: when the second queue given entry is other than the last entry in the second queue, assigning the next second queue entry to be an entry in the second queue after the given entry, and when the second queue given entry is the last entry in the second queue, reading a given entry from the first queue.

17. The network element according to claim 16 and wherein the second queue comprises a reliable delivery queue (RDQ) and each entry in the RDQ in the second queue comprises a work queue entry (WQE).

18. The network element according to claim 14 and wherein the buffer address control circuitry is also configured, before reading the given entry from the first queue, to read, for each chosen queue from a plurality of queues, a chosen queue given entry from the chosen queue maintained in the memory of the device external to the network element, the chosen queue having at least a first chosen queue entry and a last chosen queue entry, the chosen queue given entry comprising a destination address in the memory, and the output circuitry is also configured to write data to the destination address in the chosen queue given entry, and the buffer address control circuitry is also configured to assign an next chosen queue entry by: when the chosen queue given entry is other than the last entry in the chosen queue, assigning the next chosen queue entry to be an entry in the chosen queue after the given entry; and when the chosen queue given entry is the last entry in the chosen queue, and each of the plurality of queues has already been processed as a chosen queue, reading a given entry from the first queue.

19. The method according to claim 14 and wherein the network element comprises a network interface controller (NIC).

20. The method according to claim 14 and wherein the network element comprises a switch.

21. A network element comprising: buffer address control circuitry configured to configured to read, for each chosen queue from a plurality of queues, a chosen queue given entry from the chosen queue maintained in a memory of a device external to the network element, the chosen queue having at least a first chosen queue entry and a last chosen queue entry, the chosen queue given entry comprising a destination address in the memory; and output circuitry configured to write data, the data being comprised in a packet received from external to the network element, to the destination address in the memory in accordance with the given entry, wherein the buffer address control circuitry is also configured to assign an next chosen queue entry by: when the chosen queue given entry is other than the last entry in the chosen queue, assigning the next chosen queue entry to be an entry in the chosen queue after the given entry; and when the chosen queue given entry is the last entry in the chosen queue, choosing a different queue from the plurality of queues, and using the different queue as the chosen queue.

22. The method according to claim 21 and wherein the network element comprises a network interface controller (NIC).

23. The method according to claim 21 and wherein the network element comprises a switch.

24. The network element according to claim 18 and wherein each of the plurality of queues comprises a reliable delivery queue (RDQ) and each entry in each RDQ in the plurality of queues comprises a work queue entry (WQE).

25. The network element according to claim 14, and wherein the packet comprises a plurality of packets, each packet comprising data, and the network element is also configured, before the next entry assignment circuitry assigns the next entry to be the first entry in the first queue, to discard at least one of the plurality of packets.

26. The network element according to claim 21, and wherein the packet comprises a plurality of packets, each packet comprising data, and the network element is also configured, before the next entry assignment circuitry assigns the next entry to be the first entry in the first queue, to discard at least one of the plurality of packets.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The present invention will be understood and appreciated more fully from the following detailed description, taken in conjunction with the drawings in which:

[0043] FIG. 1 is a simplified block-diagram illustration of an input-output queueing system, constructed and operative in accordance with an exemplary embodiment of the present invention;

[0044] FIG. 2 is a simplified block-diagram illustration of an input-output queueing system, constructed and operative in accordance with another exemplary embodiment of the present invention;

[0045] FIG. 3 is a simplified block-diagram illustration of an exemplary implementation of the system of FIG. 2;

[0046] FIG. 4 is a simplified flowchart illustration of an exemplary method of operation of the system of FIG. 2; and

[0047] FIG. 5 is a simplified flowchart illustration of another exemplary method of operation of the system of FIG. 2.

DETAILED DESCRIPTION OF EMBODIMENTS

[0048] Reference is now made to FIG. 1, which is a simplified block-diagram illustration of an input-output queueing system, constructed and operative in accordance with an exemplary embodiment of the present invention. The system of FIG. 1, generally designated 101, comprises the following:

[0049] a host memory 103, comprised in a host device (not shown); the host device may be, for example, an appropriate processor packaged with the network element, or may be an appropriate processor external to the network element and in communication therewith by an appropriate communication mechanism such as, by way of non-limiting example, PCI-e; and

[0050] a network element 105, which may, for example, comprise a switch (which may be any appropriate switch such as, by way of non-limiting example, a suitable switch based on a Spectrum-2 ASIC, such switches (one particular example of such a switch being a SN2700 switch) being commercially available from Mellanox Technologies Ltd.) or a network interface controller (NIC) (which may be any appropriate NIC such as, by way of one particular non-limiting example, a ConnectX-5 NIC, commercially available from Mellanox Technologies Ltd.).

[0051] The host memory 103 stores a plurality of work queue elements (WQE), shown in FIG. 1 as WQE0 107, WQE1 109, WQE2 111, WQE3 113, and (further WQEs not shown through) WQEn 115, it being appreciated that the particular number of WQEs shown in FIG. 1 is not meant to be limiting, and that in some cases there may be, by way of non-limiting example, a few hundred or a few thousand WQEs.

[0052] The plurality of WQEs are maintained in a received data queue (RDQ) 120. It is appreciated that, for the sake of simplicity of depiction, the plurality of WQEs are depicted as being in a single RDQ 120; in certain exemplary embodiments there may be a plurality of RDQs instead of a single RDQ.

[0053] Each of the plurality of WQEs comprises a host memory address; in the simplified depiction of FIG. 1:

[0054] the WQE0 107 stores a WQE0 host memory address 122;

[0055] the WQE1 109 stores a WQE1 host memory address 124;

[0056] the WQE2 111 stores a WQE2 host memory address 126;

[0057] the WQE3 113 stores a WQE3 host memory address 128: and

[0058] the WQEn 115 stores a WQEn host memory address 130.

[0059] Each of the host memory addresses 122, 124, 126, 128, and 130 can be viewed as a pointer into a location in the host memory 103.

[0060] An exemplary mode of operation of the exemplary embodiment of FIG. 1 is now briefly described. A plurality of incoming packets is received at the network element 105. For simplicity of depiction and description, in FIG. 1 the plurality of incoming packets are shown as: [0061] packet0 132; [0062] packet1 134; [0063] packet2 136; [0064] packet3 138; and

[0065] (other packets not shown, through) packetn 140.

It is appreciated that, in practice, a much larger number of packets may be received.

[0066] When a given packet, such as packet0 132, is received at the network element 105, the network element 105 reads a next WQE in the RDQ 120; in the particular example of packet0 132, the next WQE is the first WQE, WQE0 107. The network element 105 then determines (in the particular non-limiting example of WQE0 107) the host memory address 122 stored in WQE0 107, and stores data (generally comprising all of, but possibly comprising only a portion of) packet0 132 in the indicated address location of the host memory 103; the location for storage of the data from packet0, based on the host memory address 122, is indicated in FIG. 1 by reference numeral 142.

[0067] When a next packet, packet1 134 arrives, the next WQE, namely WQE1 109, is accessed by the network element 105; and the data of packet1 134 is then stored in the indicated address location of the host memory 103, based on the host memory address 124 in WQE1 109. The location for storage of the data from packet1 is indicated in FIG. 1 by reference numeral 144.

[0068] Similarly, data of further incoming packets (depicted in FIG. 1 as packet2 136, packet3 138, and packetn 140) is stored in indicated address locations of the host memory 103 (designated in FIG. 1 by reference numerals 146, 148 and 150), based on the host memory addresses 126, 128, and 130 in the corresponding WQEs.

[0069] As depicted in FIG. 1, it is appreciated that the order of host memory addresses for storage of data of packets is not necessarily the same as the order of WQEs; for example, in FIG. 1, the host memory address 148 associated with WQE3 113 is shown as being between the host memory address 142 associated with WQE0 107 and the host memory address 144 associated with WQE1 109.

[0070] As described above, it is appreciated that, in the exemplary embodiment of FIG. 1, it may be the case, particularly if the network element 105 implements a high-speed network in which a portion of the network traffic generates events (corresponding in the exemplary embodiment of FIG. 1 to the packets 132, 134, 136, 138, and 140), that events (which, by way of non-limiting example, may comprise: packets with errors; a certain fixed percentage of received packets; etc.) may be sent at a high rate to the host (not shown) for storage in the host memory 103.

[0071] In the described case of a high rate of incoming packets, it is appreciated that memory consumption in the host memory 103 is high and, as a result, allocated memory for received data (indicated in FIG. 1 by reference numerals 142, 144, 146, 148, and 150) may quickly fill. Once allocated memory for received data in the host memory 103 is full, additional WQEs in the RDQ 120 and additional allocated memory for received data will be allocated by the host (not shown) in order to allow additional packets to be received. In such a situation, if additional WQEs in the RDQ 120 and additional allocated memory for received data is not provided quickly enough (quickly enough in light of a rate of received packets), in general the network element 105 will not be able to write further data to the host memory 103, such that incoming packets will be lost, by being discarded by the network element 105. Alternatively, the network element 105 may prevent packet loss by storing packets to the extent possible until a WQE becomes available, but since the number of packets which can be stored in the network element 105 is limited, such a scenario may cause back pressure, which can cause spreading network congestion, as is known in the art in cases of back pressure.

[0072] Reference is now made to FIG. 2, which is a simplified block-diagram illustration of an input-output queueing system, constructed and operative in accordance with another exemplary embodiment of the present invention.

[0073] The system of FIG. 2, generally designated 201, comprises the following:

[0074] a host memory 203, comprised in a host device (not shown); the host device may be similar to the host device described above with reference to FIG. 1; and

[0075] a network element 205, which may, for example, comprise a switch or a network interface controller (NIC), which may be similar to those described above with reference to FIG. 1.

[0076] The host memory 203 stores a plurality of work queue elements (WQE), shown in FIG. 2 as WQE0 207, WQE1 209, WQE2 211, WQE3 213, and (further WQEs not shown through) WQEn 215, it being appreciated that the particular number of WQEs shown in FIG. 2 is not meant to be limiting, and that in some cases there may be, by way of non-limiting example, a few hundred or a few thousand WQEs.

[0077] The plurality of WQEs are maintained in a received data queue (RDQ) 220. It is appreciated that, for the sake of simplicity of depiction, the plurality of WQEs are depicted as being in a single RDQ 220; in certain exemplary embodiments there may be a plurality of RDQs instead of a single RDQ.

[0078] Each of the plurality of WQEs comprises a host memory address; in the simplified depiction of FIG. 2:

[0079] the WQE0 127 stores a WQE0 host memory address 222;

[0080] the WQE1 209 stores a WQE1 host memory address 224;

[0081] the WQE2 211 stores a WQE2 host memory address 226;

[0082] the WQE3 213 stores a WQE3 host memory address 228; and

[0083] the WQEn 215 stores a WQEn host memory address 230.

Each of the host memory addresses 222, 224, 226, 228, and 230 can be viewed as a pointer into a location in the host memory 203.

[0084] An exemplary mode of operation of the exemplary embodiment of FIG. 2 is now briefly described. A plurality of incoming packets is received at the network element 205. For simplicity of depiction and description, in FIG. 2 the plurality of incoming packets are shown as: [0085] packet0 232; [0086] packet1 234; [0087] packet2 236; [0088] packet3 238;

[0089] (other packets not shown, through) packetn 140; and

[0090] packetn+1 252.

It is appreciated that, in practice, a much larger number of packets may be received.

[0091] When a given packet, such as packet0 232, is received at the network element 205, the network element 205 accesses a next WQE in the RDQ 220; in the particular example of packet0 232, the next WQE is the first WQE, WQE0 207. The network element 205 then determines (in the particular non-limiting example of WQE0 207) the host memory address 222 stored in WQE0 207, and stores (similarly to the mechanism described above with reference to FIG. 1) data of packet0 232 in the indicated address location of the host memory 203; the location for storage of the data from packet0, based on the host memory address 222, is indicated in FIG. 2 by reference numeral 242 (as will be explained in more detail below, for purposes of simplicity of depiction and description, the host memory address 242 is shown as if the host memory address 242 were outside the host memory 203, while in fact the host memory address 242 is comprised in the host memory 203).

[0092] When a next packet, packet1 234 arrives, the next WQE, namely WQE1 209, is accessed by the network element 205; and the data of packet1 234 is then stored in the indicated address location of the host memory 203, based on the host memory address 224 in WQE1 209. The location for storage of the data of packet1 is indicated in FIG. 2 by reference numeral 244.

[0093] Similarly, data of further incoming packets (depicted in FIG. 2 as packet2 236, packet3 238, and packetn 240) is stored in indicated address locations of the host memory 203 (designated in FIG. 2 by reference numerals 246, 248 and 250), based on the host memory addresses 226, 228, and 230 in the corresponding WQEs.

[0094] As depicted in FIG. 2, it is appreciated that the order of host memory addresses for storage of data portions of packets is not necessarily the same as the order of WQEs; for example, in FIG. 2, the host memory address 244 associated with WQE2 209 is shown as being between the host memory address 248 associated with WQE3 213 and the host memory address 246 associated with WQE2 226.

[0095] As described above, it is appreciated that, in the exemplary embodiment of FIG. 2, it may be the case, particularly if the network element 205 implements a high-speed network in which a portion of the network traffic generates events (corresponding in the exemplary embodiment of FIG. 2 to the packets 232, 234, 236, 238, and 240), that events may be sent at a high rate to the host (not shown) for storage in the host memory 203. In the described case of a high rate of incoming packets, it is appreciated that the rate of memory consumption in the host memory 203 is high and, as a result, allocated memory for received data (indicated in FIG. 1 by reference numerals 242, 244, 246, 248, and 250) may quickly fill. Once allocated memory for received data in the host memory 203 is full and an additional packet such as packetn+1 252 is received, the network element 205 accesses the RDQ 220 in a circular fashion, so that after having accessed WQEn 215, the next WQE accessed, for packetn+1 252, is WQE0 207, such that the data portion of packetn+1 252 is stored in a host memory address 254 (which is actually the same as host memory address 242), replacing data formerly held in that location (in the exemplary embodiment of FIG. 2, the data formerly held in that location was the data of packet 0 232).

[0096] It will be appreciated that the circular fashion of access to WQEs in the RDQ 220 may continue indefinitely, with WQEs being reused repeatedly (indefinitely), with locations for storage of data in the host memory 203 being reused repeatedly (indefinitely). In this way, the issue described above with reference to FIG. 1, in which the network element 105 will not be able to write further data to the host memory 103, such that incoming packets will be lost (or such that network congestion will occur), has been overcome, albeit at the price of overwriting older data stored in the host memory 103. In the exemplary embodiment of FIG. 2, it is appreciated that the latest (newest) packet will generally overwrite the oldest packet in the memory of the host. This may allow maintaining in memory the latest (generally the most relevant) data, while consuming less memory than would be consumed if a very large amount of memory were to be allocated to handle large numbers of incoming packets, and reducing CPU load relative to a situation in which more and more WQEs and more and more memory locations were to be allocated to handle large numbers of incoming packets.

[0097] In other exemplary embodiments of the present invention, an operation similar to the operation described above with reference to FIG. 1 may first take place, until all WQEs in the RDQ 120 have been used; and then an operation similar to the operation described above with reference to FIG. 2 may take place, using the WQEs in the RDQ 220 of FIG. 2 in a circular fashion. In this manner, in addition to maintaining data from the latest (newest) packets received, data from the first (oldest) packets received is also maintained. In a further exemplary embodiment, more than one RDQ such as the RDQ 120 of FIG. 1 may be provided, with the operation described above with reference to FIG. 1 taking place once for each RDQ; and then an operation similar to the operation described above with reference to FIG. 2 may take place, using the WQEs in the RDQ 220 of FIG. 2 in a circular fashion.

[0098] In a still further exemplary embodiment, more than one RDQ such as the RDQ 120 of FIG. 1 may be provided, with the operation described above with reference to FIG. 1 taking place once for each RDQ. In this exemplary embodiment, if a sufficient number of RDQs are provided, similar advantages to those mentioned with the system of FIG. 2 may be obtained, even without using an RDQ, such as the RDQ 220 of FIG. 2, in a circular fashion.

[0099] Reference is now made to FIG. 3, which is a simplified block-diagram illustration of an exemplary implementation of the system of FIG. 2.

[0100] The exemplary implementation of FIG. 3 comprises the following:

[0101] a network element 305, which may be as described above with reference to FIG. 2; and

[0102] an external device 310 comprising a memory 315, both of which may be as described above with reference to FIG. 2.

[0103] The network element 305 is depicted in FIG. 3 as comprising the following elements, it being appreciated that other elements (not shown, which may comprise conventional elements of a conventional network element) may also be comprised in the network element 305:

[0104] buffer address control circuitry 320;

[0105] output circuitry 325; and

[0106] next entry assignment circuitry 330.

It is appreciated that the buffer address control circuitry 320, the output circuitry 325, and the next entry assignment circuitry 330, while shown as separate, may in an actual implementation be combined in various ways; by way of non-limiting example, the buffer address control circuitry 320 and the next entry assignment circuitry 330 may be combined into a single element.

[0107] An exemplary mode of operation of the exemplary implementation of FIG. 3 is now briefly described.

[0108] Packets (shown for simplicity as a single packet 335, it being appreciated as described above with reference to FIG. 2 that a large plurality of packets may be processed) are received at the network element 305 from a source external thereto.

[0109] The buffer address control circuitry 320 and the next entry assignment circuitry 330 are together configured to access WQEs in one or more RDQs (not shown in FIG. 3) in the memory 315, as described above with reference to FIGS. 1 and 2. For example, the buffer address control circuitry 320 may be configured to access a given WQE in an RDQ and to supply a memory address comprised in the WQE to the output circuitry 325. The next entry assignment circuitry 330 may be configured to choose a next WQE (either in the manner described above with reference to FIG. 1 or in the circular manner described above with reference to FIG. 2).

[0110] When accessing an RDQ, zero, one, or more RDQs may be accessed in the manner described above with reference to FIG. 1, followed by accessing one or more RDQs in the circular manner described above with refence to FIG. 2. Alternatively, a plurality of RDQs may be accessed in the manner described above with reference to FIG. 1, without accessing any RDQs in the circular manner described above with reference to FIG. 2.

[0111] The output circuitry 325 is configured to write data from incoming packets (such as the packet 335) into the memory 315, in accordance with addresses in WQEs in RDQs (neither shown in FIG. 3); the addresses are supplied by the by the buffer address control circuitry, as described above.

[0112] Reference is now made to FIG. 4, which is a simplified flowchart illustration of an exemplary method of operation of the system of FIG. 2. The method of FIG. 4 may include the following steps:

[0113] A network element, including at least buffer address control circuitry and output circuitry, is provided (step 405).

[0114] A packet including data is received from external to the network element (step 410).

[0115] The buffer address control circuitry reads a given entry from a (first) queue maintained in memory of a device external to the network element. The queue has at least a first entry and a last entry. It is appreciated that whenever a queue is indicated herein to have a first entry and a last entry, it is alternatively possible for the queue to have only one entry, which would be both the first entry and the last entry in the queue; thus, recitation of a first entry and a last entry in a queue is not limiting, and such a queue could have only one entry. The given entry includes a destination address in the memory (step 415).

[0116] The output circuitry writes the data to the destination address in the memory, in accordance with the given entry (step 420).

[0117] A next entry is assigned by the buffer address control circuitry as follows: when the given entry is other than the last entry in the (first) queue, a next entry is assigned as an entry in the (first) queue after the given entry; when the given entry is the last entry in the (first) queue, the next entry is assigned as the first entry in the (first) queue (step 425).

[0118] The next entry (as assigned in step 425) is used as the given entry (step 430). Processing then proceeds with step 420.

[0119] Reference is now made to FIG. 5, which is a simplified flowchart illustration of another exemplary method of operation of the system of FIG. 2. The method of FIG. 5 may include the following steps:

[0120] A network element, including at least buffer address control circuitry and output circuitry, is provided (step 505).

[0121] A packet including data is received from external to the network element (step 510).

[0122] From a plurality of queues provided, a queue is chosen, and the buffer address control circuitry reads a given entry from the chosen queue maintained in memory of a device external to the network element. The chosen queue has at least a first entry and a last entry. The given entry includes a destination address in the memory (step 515).

[0123] The output circuitry writes the data to the destination address in the memory, in accordance with the given entry (step 520).

[0124] A next entry is assigned by the buffer address control circuitry as follows: when the given entry is other than the last entry in the given queue, a next entry is assigned as an entry in the given queue after the given entry; when the given entry is the last entry in the given queue, another one of the plurality of queues is chosen as the given queue, and the next entry is assigned as the first entry in the (new) given queue (steps 525 and 530). Processing then proceeds with step 520.

[0125] It is appreciated that software components of the present invention may, if desired, be implemented in ROM (read only memory) form. The software components may, generally, be implemented in hardware, if desired, using conventional techniques. It is further appreciated that the software components may be instantiated, for example: as a computer program product or on a tangible medium. In some cases, it may be possible to instantiate the software components as a signal interpretable by an appropriate computer, although such an instantiation may be excluded in certain embodiments of the present invention.

[0126] It is appreciated that various features of the invention which are, for clarity, described in the contexts of separate embodiments may also be provided in combination in a single embodiment. Conversely, various features of the invention which are, for brevity, described in the context of a single embodiment may also be provided separately or in any suitable subcombination.

[0127] It will be appreciated by persons skilled in the art that the present invention is not limited by what has been particularly shown and described hereinabove. Rather the scope of the invention is defined by the appended claims and equivalents thereof;