In network routing, CoDel (pronounced "coddle") for controlled delay is a scheduling algorithm for the network scheduler developed by Van Jacobson and Kathleen Nichols. It is designed to overcome bufferbloat in network links (such as routers) by setting limits on the delay network packets suffer due to passing through the buffer being managed by CoDel.
CoDel aims at improving on the overall performance of the RED algorithm by addressing some fundamental misconceptions in the algorithm (as perceived by Jacobson) and by being easier to manage (since, unlike RED, CoDel does not require manual configuration).
An implementation of CoDel was written by Dave Täht and Eric Dumazet for the Linux kernel and dual licensed under the GNU General Public License and the 3-clause BSD license. Dave Täht back-ported CoDel to Linux kernel 3.3 for project CeroWrt, which concerns itself among other things with bufferbloat, where it was exhaustively tested. It was then pushed into OpenWrt. Dumazet's variant of CoDel is called fq_codel, standing for "fair queuing controlled delay"; in 2012 it was adopted as the standard active queue management (AQM) and packet scheduling solution in the OpenWrt "Barrier Breaker". From there, CoDel and fq_codel have migrated into various downstream projects such as dd-wrt, IpFire, and technologies like StreamBoost.
The theory behind CoDel is based on a number of observations of packet behavior in packet-switched networks under the influence of data buffers. Some of these observations are about the fundamental nature of queueing and the causes of bufferbloat, others relate to weaknesses of alternative queue management algorithms. CoDel was developed as an attempt to address the problem of bufferbloat.
The flow of packets slows down while travelling through a network link between a fast and a slow network, especially at the start of a TCP session, when there is a sudden burst of packets and link to the slower network may not be able to process the burst quickly enough. Buffers exist to ease this problem by giving the fast network a place to push packets, to be read by the slower network as fast as it can. In other words, buffers act like shock absorbers to convert bursty arrivals into smooth, steady departures. However, a buffer has a finite size, and it can hold only a specific maximum number of packets. The ideal buffer is sized so it can handle a sudden burst of communication and match the speed of that burst to the speed of the slower network. Ideally, the "shock absorbing" situation is characterized by a temporary delay for packets in the buffer during the transmission burst, after which the delay rapidly disappears and the network reaches a balance in offering and handling packets.
The TCP congestion avoidance algorithm relies on packet drops to determine the available bandwidth. It speeds up the data transfer until packets start to drop, and then slows down the transmission rate. Ideally it keeps speeding up and slowing down the transmission rate, until it finds an equilibrium to the speed of the link. However, for this to work the packet drops must occur in a timely manner, so that the algorithm can select a suitable transfer speed. With a large buffer that has been filled, the packets will arrive at their destination, but with a higher latency. The packet is not dropped, so TCP does not slow down once the uplink has been saturated, further filling the buffer. Newly arriving packets are dropped only when the buffer is fully saturated. TCP may even decide that the path of the connection has changed, and again go into the more aggressive search for a new operating point.
Similarly, routing protocols such as IGRP use available bandwidth as one of the key metrics in path selection. If an intermediary device has a tendency to buffer large amounts of data to establish a massive backlog it can give the appears of having a large amount of bandwidth available, causing more packets to be queued down that path until more packet drops start happening. If bandwidth were given a realistic appearance, an alternative network route could be selected (depending on network topology). In this case, large buffers interfere with TCP on the end nodes as well as the routing decisions between them.
In the problematic situation, packets queued in a network buffer are only dropped if the buffer is full. Having a big and constantly full buffer which causes increased transmission delays and reduced interactivity, especially when looking at two or more simultaneous transmissions over the same channel, is called bufferbloat. Available channel bandwidth can also end up being unused, as some fast destinations may not be reached due to buffers clogged with data awaiting delivery to slow destinations, what is caused by contention between simultaneous transmissions competing for some space in an already full buffer.
Even in a situation where no packets end up being dropped, buffer bloat can still degrade the user's experience. One instance would be for applications where quality of service is sensitive to per-user latency. One example would be a web server, where the Tx buffer for an interface is shared amongst several users. If the buffer is set to large, a user downloading a large ISO can degrade the experience of most of the other users who are only requesting small web pages. The reason is because the Tx buffer will end up being mostly filled with the resource-intensive user's requested file, creating a large buffer that must be worked through before other users' data can be reached. By contrast, a smaller buffer could allow for the other users' requests to more fairly interleave with the resource intensive user's request. The resource hog experience less throughput and less data may be moved overall but the data that is delivered will be "fairer."
Good and bad queues
- Good queue
- Defined as a queue that exhibits no buffer bloat, i.e. catches and handles communications bursts with no more than a temporary increase in queue delay and which maximizes utilization of the network link.
- Bad queue
- Defined as a queue that exhibits buffer bloat, i.e. where a communications burst has caused the buffer to fill up and stay filled, resulting in low utilization and a constantly high buffer delay.
In order to be effective against bufferbloat, a solution in form of an Active queue management (AQM) algorithm must be able to recognize an occurrence of bufferbloat and react with deploying effective countermeasures.
Regarding the recognition of an unwanted situation, Van Jacobson asserted in 2006 that existing algorithms have been using incorrect means of recognizing bufferbloat. In an attempt to recognize the telltale standing queue of bufferbloat, algorithms like RED measure the average queue length (in packets or stored bytes) and consider it a case of bufferbloat if the average grows too large. However, Jacobson demonstrated in 2006 that this measurement is not a good metric, as the average queue length rises sharply in case of a communications burst. But this can then dissipate quickly (good queue) or develop a standing queue (bad queue). Also, other factors in network traffic can cause false positives or negatives, causing countermeasures to be deployed unnecessarily; Jacobson suggested therefore that average queue length actually contains no information at all about packet demand or network load. He then suggested that a better metric might be the minimum amount of delay experienced by any packet in the sliding window of the buffer.
The CoDel algorithm
Based on Jacobson's notion from 2006, CoDel was developed to manage queues under control of the minimum delay experienced by packets in the running buffer window. The goal is to keep this minimum delay below 5 milliseconds. If the minimum delay rises to too high a value, packets are dropped from the window until the delay drops below the maximum level.
Nichols and Jacobson cite several advantages to using nothing more than this metric:
- CoDel is parameterless. One of the weaknesses in the RED algorithm (according to Jacobson) is that it is too difficult to configure (and too difficult to configure correctly, especially in an environment with dynamic link rates). CoDel has no parameters to set at all.
- CoDel treats good queue and bad queue differently. A good queue has low delays by nature, so the management algorithm can ignore it, while a bad queue is susceptible to management intervention in the form of dropping packets.
- CoDel works off of a parameter that is determined completely locally, so it is independent of round-trip delays, link rates, traffic loads and other factors that cannot be controlled or predicted by the local buffer.
- The local minimum delay can only be determined when a packet leaves the buffer, so no extra delay is needed to run the queue to collect statistics to manage the queue.
- CoDel adapts to dynamically changing link rates with no negative impact on utilization.
- CoDel can be implemented relatively simply and therefore can span the spectrum from low-end home routers to high-end routing solutions.
CoDel does nothing to manage the buffer if the minimum delay for the buffer window is below the maximum allowed value. It also does nothing if the buffer is relatively empty (if there are fewer than one MTU's worth of bytes in the buffer). If these conditions do not hold, then CoDel drops packets probabilistically.
The algorithm is independently computed at each hop. The algorithm operates over an interval. The initial interval is 100 milliseconds. Per-packet queuing delay is monitored through the hop. As each packet is dequeued for forwarding, the queuing delay (amount of time the packet spent waiting in the queue) is calculated. The lowest queuing delay for the interval is stored. When the last packet of the interval is dequeued, if the lowest queuing delay for the interval is greater than 5 milliseconds, this single packet is dropped and the interval used for the next group of packets is shortened. If the lowest queuing delay for the interval is less than 5 milliseconds, the packet is forwarded and interval is reset to 100 milliseconds.
When the interval is shortened, it is done so in accordance with the inverse square root of the number of successive intervals in which packets were dropped due to excessive queuing delay. The sequence of intervals is , , , , ...
- In comparison to RED, CoDel keeps the packet delay closer to the target value across the full range of bandwidths (from 3 to 100 Mbit/s). This seems to result in good queue, since the measured link utilizations are consistently near 100% of link bandwidth.
- At lower MTU, packet delays are lower than at higher MTU. Higher MTU results in good link utilization, lower MTU results in good link utilizations at lower bandwidth, degrading to fair utilization at high bandwidth.
CoDel in use
A full implementation of CoDel was realized in May 2012 and is available as open-source software to all interested parties. This implementation will be used by different parties to study CoDel in actual use.
CoDel began to appear as an option in some proprietary/turnkey bandwidth management platforms in 2013.
- Nichols, Kathleen; Jacobson, Van (6 May 2012). "Controlling Queue Delay". ACM Queue. ACM Publishing. doi:10.1145/2209249.2209264. Retrieved 12 August 2012.
- Gettys, Jim (May 8, 2012). "Fundamental Progress Solving Bufferbloat". jg's Ramblings. Wordpress. Retrieved 12 August 2012.
- "CoDel Linux kernel network scheduler module". kernel.org. Retrieved 2013-09-07.
- "Cerowrt - Overview". Bufferbloat. Retrieved 2014-01-24.
- "Backport of CoDel and FQ_CoDel to Linux 3.3". OpenWrt. 2012-05-16. Retrieved 2013-09-07.
- "Adoption of CoDel in OpenWrt default QoS scripts". OpenWrt.
- Joe Brockmeier (2012-05-08). "Good News for Solving Bufferbloat: CoDel Provides "No Knobs" Solution". ReadWriteWeb. Retrieved 2012-08-16.
- Jacobson, Van; Karels, MJ (1988). "Congestion avoidance and control". ACM SIGCOMM Computer Communication Review 18 (4).
- Jacobson, Van (2006). "A rant on queues. A talk presented at MIT Lincoln Labs, Lexington, MA". Retrieved 12 August 2012.
- Iljitsch van Beijnum (2012-05-10). "CoDel buffer management could solve the Internet’s bufferbloat jams". Ars Technica. Retrieved 2012-08-16.
- Nichols, Kathleen (July 2012). "Controlled Delay (CoDel) Active Queue Management". Pollere Inc. Retrieved 12 August 2012.
- Greg White; Joey Padden (November 2012). "PRELIMINARY STUDY OF CODEL AQM IN A DOCSIS NETWORK". Retrieved 2012-11-26.
- Gettys, Jim (22 May 2012). "A Milestone Reached: CoDel is in Linux!". jg's Ramblings. Retrieved 12 August 2012.
- ["http://download.proceranetworks.com/client-bin/14.0/11/changelog.txt" "Procera Packetlogic Changelog"].