Token bucket flow-rate limiter

Abstract

A token bucket flow rate limiter is provided for a data transmission, comprising a token counter configured to be incremented at a rate determining the average flow rate of the transmission; a frequency divider connected to control incrementing of the token counter from a clock, the divider having an integer division factor; and a modulator configured to alternate the division factor between two different integers so as to make the resulting average flow rate tend to a programmed flow rate comprised between two boundary flow rates respectively corresponding to the two integers.

Claims

1. A token bucket flow rate limiter for data transmission, comprising: a token counter configured to be incremented at a rate determining the average flow rate of the transmission; a frequency divider connected to control incrementing of the token counter from a clock, the divider having an integer division factor; and a modulator configured to alternate the division factor between two different integers so that the resulting average flow rate tends to a programmed flow rate comprised between two boundary flow rates respectively corresponding to the two integers.

2. The flow rate limiter of claim 1, wherein the modulator comprises an overflow counter connected to be incremented at the clock frequency by a fixed increment based on the difference between the programmed flow rate and the flow rate determined by a base division factor selected between the two integers, the overflow counter being connected to the divider to alternate the division ratio at each overflow.

3. The flow rate limiter of claim 2, wherein the frequency divider is configured to apply by default the smallest of the two integers as the division factor and transitorily increase the division factor at each overflow of the overflow counter.

4. The flow rate limiter of claim 3, wherein the frequency divider comprises an auxiliary counter connected to be incremented at the clock frequency and count up to the smallest of the two integers, the overflow counter being connected to make the auxiliary counter skip one increment at each overflow.

5. The flow rate limiter of claim 2, wherein: the programmed flow rate is represented by a normalized flow rate between 0 and 1, the base division factor is chosen equal to the integer part of the ratio between the token counter increment and the normalized flow rate, and the increment of the overflow counter is based on the fractional part of the ratio between the token counter increment and the normalized flow rate.

6. The flow rate limiter of claim 5, wherein the increment of the overflow counter is expressed by the relationship:
2.sup.k{c/}.Math./c rounded to the lower or higher integer, where is the normalized flow rate, c is the increment of the token counter, k is the number of bits of the overflow counter, and {. . .} designates a fractional part.

7. A method of limiting the flow rate of a data transmission based on a token bucket, comprising the steps of: incrementing a token counter at consecutive intervals having durations that are variable between two discrete values that are multiples of the period of a clock; selecting the two discrete values according to a programmed flow rate; and applying the two discrete values to the token counter according to respective frequencies selected so that the average token counter incrementing rate corresponds to the programmed flow rate.

8. The method of claim 7, comprising the steps of: applying by default a first of the two discrete values for the incrementing interval of the token counter; incrementing an overflow counter at the frequency of the clock by an increment based on the difference between the programmed flow rate and the flow rate corresponding to the first discrete value; and transitorily applying to the token counter the second discrete value as the incrementing interval at each overflow of the overflow counter.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

(1) Other advantages and features will become more clearly apparent from the following description of particular embodiments of the invention provided for exemplary purposes only and represented in the appended drawings, in which:

(2) FIG. 1 is a block diagram of an exemplary token bucket flow limiter;

(3) FIG. 2 is a block diagram of an embodiment of a flow limiter offering accurate adjustment;

(4) FIG. 3 is a block diagram of a detailed embodiment of a flow limiter based on the principle of FIG. 2;

(5) FIGS. 4A to 4D are graphs showing the evolution of counters of the flow limiter of FIG. 3, in the context of an example;

(6) FIG. 5 is a graph showing a long-term trend of the amount of data transmitted in the context of the example of FIG. 4; and

(7) FIG. 6 schematically shows a system for transmitting multiple competing data flows on a shared network link, in which flow limiters can be implemented.

DETAILED DESCRIPTION

(8) In FIG. 1, a token bucket flow rate limiter usually includes a token counter 10 connected to be incremented at the rate of pulses INC arriving at a clock input. The content of the token counter is limited to a maximum value defined by a parameter b, often programmed in a register. The increment to be applied may be an integer c at least equal to 1, programmable in another register. Thus, at each pulse INC, the content of counter 10 increases by value c, up to saturation at value b.

(9) The pulses INC are supplied from a clock CK through a frequency divider 12. The division factor N may be programmed in a register. The divider 12 is generally in the form of a counter connected to be reset as soon as its content reaches N. Thus, the divider 12 produces a pulse INC every N pulses of clock CK.

(10) Each time a data packet is available for transmission, the packet size is compared to the content of the counter 10. If the number of flits in the packet is at most equal to the content of the counter, the packet is transmitted and the counter is decremented by the size of the packet. Otherwise, the packet may be put on hold and the counter is not modified.

(11) Such a flow limiter allows the programming of normalized rates by discrete values =c/N, where N and c are integers such that cN. If a fine adjustment granularity is desired between the programmable flow rates, N is chosen relatively large. However, when N is large, there is a correspondingly large time delay between two increments of the token counter, which increases latency in the packet transmission. A relatively high latency may be acceptable when the programmed rate is low (e.g. c/N=1/100), but may be unacceptable when the programmed rate is high (e.g. c/N=99/100, where increments of the token counter take place only once every 100 clock cycles).

(12) Thus, for higher flow rates, it is desired that N be close to 1, which substantially reduces the granularity for .

(13) FIG. 2 is a block diagram of an embodiment of a flow rate limiter that offers a fine granularity whatever the value programmed for flow rate , and even allows programming a flow rate value that is not a ratio of integers.

(14) The divider-by-N of the flow rate limiter of FIG. 1 is replaced by a fractional divider 12, configured for example to apply one or the other of two integer division factors, here consecutive integers N and N+1. The integer N may be the integer part of c/, denoted [c/], whereby the flow rate determined by the sole factor N is greater than or equal to the programmed rate , itself less than the flow rate determined by the sole factor N+1.

(15) In general, instead of two consecutive integers N and N+1, any two distinct integers N1 and N2 may be used, chosen so that the programmed rate lies between the boundary rates determined by these two integers.

(16) The divider 12 is controlled by a modulator 14 configured to select the division factor to apply for each increment based on the fractional part of c/, denoted {c/}. Here the modulation is such that the ratio between the frequency of use of factor N+1 and the number of increments tends to {c/}, producing an increment rate of the token counter equal on average to the programmed flow rate.

(17) A modulation is used, because it is sought to alternate the two division factors as much as possible, to smooth the token counter increments and the resulting actual flow rate. Such a modulator could be borrowed from phase locked loops or delta-sigma modulators, but these systems are complex in that they are designed to follow varying analog signals.

(18) FIG. 3 is a block-diagram of a flow rate limiter of the type of FIG. 2, illustrating in detail an example of a simple modulator satisfying the needs of the flow rate limiter. The modulator 14 comprises a counter 16 connected to be incremented at each clock cycle CK by an integer a, programmable in a register. Incrementing is achieved in an open loop, that is to say without controlling the counter overflow.

(19) The counter 16 having a limited size, say k bits, its content varies modulo 2.sup.k. In other words, counter 16 regularly overflows to start counting anew, which is why it will be referred to hereunder as overflow counter. If 2.sup.k is divisible by a, the content of the counter 16 changes according to ramps having a period equal to 2.sup.k/a clock cycles, starting at 0 and ending at 2.sup.k1. If 2.sup.k if is not divisible by a, the content of counter 16 changes according to aperiodic ramps that start at a value between 0 and a1, and end at a value between 2.sup.k and 2.sup.ka1.

(20) The overflow events of the counter 16 are taken as a reference to produce the desired modulation. The counter 16 may be designed to produce a pulse OF at every overflow. Then the fractional divider 12 may be configured by default to apply the division factor N and to respond to each pulse OF to transitorily apply the division factor N+1.

(21) The value of increment a thus determines the frequency of application of the factor N+1. To achieve the modulation sought here, the increment a is expressed by the equation:
2.sup.k{c/}.Math./c=2.sup.k(1[c/].Math./c)
rounded to the upper or lower integer value.

(22) The fractional divider 12 may be realized, as shown, by a simple divider-by-N 12, whose clock input is preceded by an AND gate 18. One of the inputs of the AND gate receives the clock signal CK and the other input receives the complement of the overflow pulses OF. With this configuration, the AND gate is transparent to clock CK as long as the counter 16 does not overflow, causing division by N by default. At each overflow event, the AND gate 18 masks the current clock pulse CK from the input of divider 12. The divider skips the pulse. This divider typically being a counter, the counter freezes during one cycle, causing the delay of one cycle in the production of the next incrementing pulse INC for the token counter. In other words, the pulse INC is transitorily produced after N+1 cycles instead of N cycles.

(23) FIGS. 4A to 4D are graphs illustrating this operation more clearly in the context of an example. In this example, for a packet size of 32 flits, the following values were used:

(24) =0.15,

(25) b==58,

(26) c=1,

(27) k=8, 2.sup.k=256.

(28) These settings produce the following values, calculated according to the previously mentioned equations:

(29) N=6,

(30) a=25.

(31) FIG. 4A shows the evolution of the overflow counter 16. This counter freewheels and the first overflows occur in cycles 12, 22, 32 and 42.

(32) FIG. 4B shows the evolution of a counter embodying the divider 12, here a divider by six. If the divider 12 operated in freewheel mode, it would produce periodic ramps of 6 clock cycles and an amplitude of 5, like the first ramp shown. At the times when the counter 16 (FIG. 4A) overflows, as identified by vertical lines, the counter 12 freezes during one clock cycle. Thus, the counter 12 remains at 4 during both cycles 11 and 12, remains at 1 during both cycles 21 and 22, remains at 4 during both cycles 31 and 32, and remains at 1 during both cycles 41 and 42.

(33) It follows that a division factor of seven (6+1) is applied to the second, fourth, fifth, and seventh ramps of FIG. 4B. As soon as these first cycles, it can be noted that for 7 increments of the token counter, 4 were achieved over an interval of 7 cycles, and 3 were achieved over an interval of 6 cycles, resulting in an average interval of 6.57 cycles. This value is already close to the programmed interval of 1/0.15=6.67, and will get even closer over time.

(34) FIG. 4C shows in bold line the evolution of token counter 10, set up with b=58, assuming a first packet of 32 flits becomes available for transmission in cycle 2.

(35) FIG. 4D, to which reference will be made at the same time as FIG. 4C, shows the accumulated transmitted flits.

(36) The packet transmission starts at cycle 3, as the token counter is decremented by 32, these operations being identified by a pulse in dashed line having the packet size as amplitude.

(37) The token counter is then incremented by one in each of the cycles 6, 13, 19, 26, 33, 39 . . . defined by the divider (FIG. 4B).

(38) The last flit of the first packet is transmitted in cycle 33. It is assumed that a new packet is available as of cycle 34, but the token counter contains 31, insufficient units to transmit the packet. The transmission of the new packet starts at cycle 40, after the token counter has reached 32 at cycle 39.

(39) FIG. 5 is a long-term extension, over 2,500 cycles, of the graph of FIG. 4C, assuming that packets are continuously available for transmission. The dashed straight lines represent boundaries for the evolution of the accumulated transmitted flits complying with a - profile.

(40) The lower boundary line is defined by y=t=0.15t, where t is the time in number of clock cycles.

(41) The upper boundary line is defined by y=t+=0.15t+58.

(42) These boundary lines are defined according to the exact programmed value of . It shall be noted that the accumulated transmitted flits fully respect these boundaries, although 1/ is not an integer.

(43) The resolution with which the flow rate can be programmed depends on the size k of the overflow counter, N, and c. More specifically, it is equal to 2kc/N for N<2.sup.k. When N is greater than 2.sup.k, the increment a is always zero and the resolution is c/(N (N+1)).

(44) FIG. 6 schematically shows an example of a system for transmitting multiple competing flows over a shared network link L, as discussed in US patent application 2014-0301206. Several flow rate limiters as described herein may be used in such a system.

(45) The system includes a CPU, a memory MEM, and a direct memory access circuit DMA, interconnected by a system bus B. A network interface NI is connected receive data through the DMA circuit and to send the data over the network link L. This network interface includes a plurality of queues 20, for example eight, each of which may be associated with a different connection, or virtual channel, or more generally a different transmission session.

(46) An arbiter ARB may be provided to manage the filling of the queues, for example using weighted fair queuing (WFQ). The writing of the queues to the network link L is managed by a flow regulator REG. This regulator may include a flow rate limiter for each queue, the parameters of which are programmed by the processor upon powering-on the system. The flow rate limiters may be configured so that each is enabled in turn in a round-robin fashion. The active limiter then transmits packets taken from its queue until the token counter is exhausted or the queue is empty.

(47) Many variations and modifications of the embodiments described herein will be apparent to the skilled person. For example, although a fractional divider was disclosed that operates by default with the smallest division factor (N) and is transitorily controlled by the modulator to apply the higher division factor (N+1), a reciprocal configuration may be used, where the divider operates by default with the higher division factor (N+1) and where this factor is transitorily switched to the smaller factor (N) by the modulator. In this case, the value 1{c/} may be used instead of {c/} in the various disclosed equations.